2023-09-12T15:42:17.138 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-12T15:42:17.139 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100 --name pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi --verbose --owner scheduled_pdonnell@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} -- /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/orig.config.yaml 2023-09-12T15:42:17.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T15:42:17.282 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100 branch: wip-batrick-testing-20230912.122437 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7395100' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20230912.122437 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20230912.122437 sha1: 3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9684 sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 sleep_before_teardown: 0 subset: 236/256 suite: fs suite_branch: wip-batrick-testing-20230912.122437 suite_path: /home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: 3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 500 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-12_14:07:50 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1394 2023-09-12T15:42:17.283 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa; will attempt to use it 2023-09-12T15:42:17.283 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks 2023-09-12T15:42:17.284 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-12T15:42:17.285 INFO:teuthology.task.internal:Checking packages... 2023-09-12T15:42:17.310 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2110e00747b31a0d73768c9e5229da18b26b8aa0' 2023-09-12T15:42:17.311 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-12T15:42:17.311 INFO:teuthology.packaging:ref: None 2023-09-12T15:42:17.312 INFO:teuthology.packaging:tag: None 2023-09-12T15:42:17.312 INFO:teuthology.packaging:branch: wip-batrick-testing-20230912.122437 2023-09-12T15:42:17.312 INFO:teuthology.packaging:sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:42:17.313 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20230912.122437 2023-09-12T15:42:17.451 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6088.g2110e007 2023-09-12T15:42:17.452 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-12T15:42:17.459 INFO:teuthology.task.internal:no buildpackages task found 2023-09-12T15:42:17.459 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-12T15:42:17.474 INFO:teuthology.task.internal:Saving configuration 2023-09-12T15:42:17.494 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-12T15:42:17.500 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-12T15:42:17.525 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 15:36:47.897367', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-12T15:42:17.547 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 15:36:47.896221', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-12T15:42:17.571 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 15:36:47.894868', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-12T15:42:17.571 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-12T15:42:17.578 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-09-12T15:42:17.579 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-09-12T15:42:17.579 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-09-12T15:42:17.579 INFO:teuthology.run_tasks:Running task console_log... 2023-09-12T15:42:17.681 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f33ed506ca0>, signals=[15]) 2023-09-12T15:42:17.682 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-12T15:42:17.688 INFO:teuthology.task.internal:Opening connections... 2023-09-12T15:42:17.688 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2023-09-12T15:42:17.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T15:42:17.759 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2023-09-12T15:42:17.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T15:42:17.826 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2023-09-12T15:42:17.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T15:42:17.891 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-12T15:42:17.898 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2023-09-12T15:42:17.920 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2023-09-12T15:42:17.920 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2023-09-12T15:42:17.980 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="Red Hat Enterprise Linux" 2023-09-12T15:42:17.980 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="8.6 (Ootpa)" 2023-09-12T15:42:17.981 INFO:teuthology.orchestra.run.smithi130.stdout:ID="rhel" 2023-09-12T15:42:17.981 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="fedora" 2023-09-12T15:42:17.981 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="8.6" 2023-09-12T15:42:17.981 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el8" 2023-09-12T15:42:17.981 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-12T15:42:17.982 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2023-09-12T15:42:17.982 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-12T15:42:17.982 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://www.redhat.com/" 2023-09-12T15:42:17.982 INFO:teuthology.orchestra.run.smithi130.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-12T15:42:17.982 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-12T15:42:17.983 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:42:17.983 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-12T15:42:17.983 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-12T15:42:17.983 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-12T15:42:17.983 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-12T15:42:17.984 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2023-09-12T15:42:18.009 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2023-09-12T15:42:18.029 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2023-09-12T15:42:18.030 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2023-09-12T15:42:18.090 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Red Hat Enterprise Linux" 2023-09-12T15:42:18.090 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8.6 (Ootpa)" 2023-09-12T15:42:18.090 INFO:teuthology.orchestra.run.smithi159.stdout:ID="rhel" 2023-09-12T15:42:18.090 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="fedora" 2023-09-12T15:42:18.091 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8.6" 2023-09-12T15:42:18.092 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2023-09-12T15:42:18.092 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-12T15:42:18.092 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2023-09-12T15:42:18.092 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-12T15:42:18.093 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.redhat.com/" 2023-09-12T15:42:18.093 INFO:teuthology.orchestra.run.smithi159.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-12T15:42:18.093 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-12T15:42:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:42:18.094 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-12T15:42:18.094 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-12T15:42:18.094 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-12T15:42:18.094 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-12T15:42:18.095 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2023-09-12T15:42:18.117 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2023-09-12T15:42:18.141 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2023-09-12T15:42:18.142 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2023-09-12T15:42:18.199 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="Red Hat Enterprise Linux" 2023-09-12T15:42:18.199 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="8.6 (Ootpa)" 2023-09-12T15:42:18.199 INFO:teuthology.orchestra.run.smithi203.stdout:ID="rhel" 2023-09-12T15:42:18.199 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="fedora" 2023-09-12T15:42:18.200 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="8.6" 2023-09-12T15:42:18.200 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el8" 2023-09-12T15:42:18.200 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-12T15:42:18.200 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2023-09-12T15:42:18.200 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-12T15:42:18.200 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://www.redhat.com/" 2023-09-12T15:42:18.201 INFO:teuthology.orchestra.run.smithi203.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-12T15:42:18.201 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-12T15:42:18.201 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:42:18.201 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-12T15:42:18.201 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-12T15:42:18.202 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-12T15:42:18.202 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-12T15:42:18.202 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2023-09-12T15:42:18.225 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-12T15:42:18.237 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-12T15:42:18.243 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-12T15:42:18.244 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2023-09-12T15:42:18.246 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2023-09-12T15:42:18.249 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2023-09-12T15:42:18.268 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-12T15:42:18.275 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-12T15:42:18.275 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2023-09-12T15:42:18.305 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2023-09-12T15:42:18.308 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2023-09-12T15:42:18.339 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-12T15:42:18.413 INFO:teuthology.run_tasks:Running task kernel... 2023-09-12T15:42:18.427 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-09-12T15:42:18.428 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-09-12T15:42:18.428 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-09-12T15:42:18.429 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-12T15:42:18.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T15:42:18.448 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2023-09-12T15:42:18.507 INFO:teuthology.orchestra.run.smithi130.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-12T15:42:18.507 INFO:teuthology.task.kernel:Running kernel on smithi130: 4.18.0-372.9.1.el8.x86_64 2023-09-12T15:42:18.508 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum install -y kernel 2023-09-12T15:42:19.703 INFO:teuthology.orchestra.run.smithi130.stdout:Updating Subscription Management repositories. 2023-09-12T15:42:19.703 INFO:teuthology.orchestra.run.smithi130.stdout:Unable to read consumer identity 2023-09-12T15:42:21.066 INFO:teuthology.orchestra.run.smithi130.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-09-12T15:42:21.800 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-09-12T15:42:25.275 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 498 kB/s | 24 kB 00:00 2023-09-12T15:42:25.936 INFO:teuthology.orchestra.run.smithi130.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-09-12T15:42:25.992 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2023-09-12T15:42:25.993 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2023-09-12T15:42:25.993 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2023-09-12T15:42:26.117 DEBUG:teuthology.orchestra.run.smithi130:> echo no | sudo yum reinstall kernel || true 2023-09-12T15:42:26.448 INFO:teuthology.orchestra.run.smithi130.stdout:Updating Subscription Management repositories. 2023-09-12T15:42:26.448 INFO:teuthology.orchestra.run.smithi130.stdout:Unable to read consumer identity 2023-09-12T15:42:26.711 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:01 ago on Tue 12 Sep 2023 03:42:25 PM UTC. 2023-09-12T15:42:26.752 INFO:teuthology.orchestra.run.smithi130.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-12T15:42:26.785 INFO:teuthology.orchestra.run.smithi130.stderr:Error: No packages marked for reinstall. 2023-09-12T15:42:26.831 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum reinstall -y kernel || true 2023-09-12T15:42:27.161 INFO:teuthology.orchestra.run.smithi130.stdout:Updating Subscription Management repositories. 2023-09-12T15:42:27.161 INFO:teuthology.orchestra.run.smithi130.stdout:Unable to read consumer identity 2023-09-12T15:42:27.421 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:02 ago on Tue 12 Sep 2023 03:42:25 PM UTC. 2023-09-12T15:42:27.460 INFO:teuthology.orchestra.run.smithi130.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-12T15:42:27.484 INFO:teuthology.orchestra.run.smithi130.stderr:Error: No packages marked for reinstall. 2023-09-12T15:42:27.533 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q kernel | sort -rV | head -n 1 2023-09-12T15:42:27.613 INFO:teuthology.orchestra.run.smithi130.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-09-12T15:42:27.613 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-09-12T15:42:27.614 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-09-12T15:42:27.614 DEBUG:teuthology.parallel:result is None 2023-09-12T15:42:27.614 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-12T15:42:27.621 INFO:teuthology.task.internal:Creating test directory... 2023-09-12T15:42:27.622 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-12T15:42:27.624 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-12T15:42:27.627 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-12T15:42:27.649 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-12T15:42:27.656 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-12T15:42:27.663 INFO:teuthology.task.internal:Creating archive directory... 2023-09-12T15:42:27.663 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-12T15:42:27.681 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-12T15:42:27.688 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-12T15:42:27.754 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-12T15:42:27.761 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-12T15:42:27.761 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-12T15:42:27.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T15:42:27.780 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-12T15:42:27.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T15:42:27.800 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-12T15:42:27.802 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-12T15:42:27.823 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-12T15:42:27.840 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T15:42:27.853 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T15:42:27.860 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T15:42:27.869 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T15:42:27.891 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T15:42:27.909 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T15:42:27.911 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-12T15:42:27.918 INFO:teuthology.task.internal:Configuring sudo... 2023-09-12T15:42:27.918 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-12T15:42:27.921 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-12T15:42:27.923 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-12T15:42:27.952 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-12T15:42:27.961 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-12T15:42:27.962 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-12T15:42:27.989 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-12T15:42:27.992 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-12T15:42:28.010 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-12T15:42:28.087 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-12T15:42:28.136 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-12T15:42:28.213 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-12T15:42:28.261 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:42:28.261 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-12T15:42:28.328 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-12T15:42:28.373 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-12T15:42:28.428 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-12T15:42:28.513 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-12T15:42:28.557 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:42:28.557 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-12T15:42:28.625 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-12T15:42:28.672 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-12T15:42:28.727 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-12T15:42:28.811 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-12T15:42:28.856 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:42:28.856 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-12T15:42:28.926 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2023-09-12T15:42:28.928 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2023-09-12T15:42:28.931 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2023-09-12T15:42:28.990 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-12T15:42:28.990 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-12T15:42:29.025 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-12T15:42:29.256 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-12T15:42:29.265 INFO:teuthology.task.internal:Starting timer... 2023-09-12T15:42:29.265 INFO:teuthology.run_tasks:Running task pcp... 2023-09-12T15:42:29.275 INFO:teuthology.run_tasks:Running task selinux... 2023-09-12T15:42:29.284 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-09-12T15:42:29.284 DEBUG:teuthology.orchestra.run.smithi130:> sudo service auditd rotate 2023-09-12T15:42:29.350 INFO:teuthology.orchestra.run.smithi130.stdout:Rotating logs: 2023-09-12T15:42:29.351 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2023-09-12T15:42:29.413 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2023-09-12T15:42:29.414 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2023-09-12T15:42:29.480 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2023-09-12T15:42:29.481 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-12T15:42:29.482 DEBUG:teuthology.orchestra.run.smithi130:> /usr/sbin/getenforce 2023-09-12T15:42:29.509 INFO:teuthology.orchestra.run.smithi130.stdout:Permissive 2023-09-12T15:42:29.510 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2023-09-12T15:42:29.535 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2023-09-12T15:42:29.535 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2023-09-12T15:42:29.563 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2023-09-12T15:42:29.563 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi130.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2023-09-12T15:42:29.564 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-12T15:42:29.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T15:42:29.591 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-12T15:42:29.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T15:42:29.621 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-12T15:42:29.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T15:42:29.649 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-12T15:42:29.650 DEBUG:teuthology.orchestra.run.smithi130:> sudo /usr/sbin/setenforce permissive 2023-09-12T15:42:29.677 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2023-09-12T15:42:29.706 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2023-09-12T15:42:29.734 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-12T15:42:29.744 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-12T15:42:29.749 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-12T15:42:29.880 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-12T15:42:29.896 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-12T15:42:29.898 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi130.front.sepia.ceph.com,smithi159.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-12T15:51:30.389 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi130.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2023-09-12T15:51:30.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2023-09-12T15:51:30.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T15:51:30.460 DEBUG:teuthology.orchestra.run.smithi130:> true 2023-09-12T15:51:30.533 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2023-09-12T15:51:30.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-09-12T15:51:30.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T15:51:30.603 DEBUG:teuthology.orchestra.run.smithi159:> true 2023-09-12T15:51:30.677 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-09-12T15:51:30.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-09-12T15:51:30.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T15:51:30.749 DEBUG:teuthology.orchestra.run.smithi203:> true 2023-09-12T15:51:30.816 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-09-12T15:51:30.816 INFO:teuthology.run_tasks:Running task clock... 2023-09-12T15:51:30.827 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-12T15:51:30.827 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-12T15:51:30.828 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T15:51:30.830 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-12T15:51:30.831 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T15:51:30.833 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-12T15:51:30.833 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T15:51:30.859 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-12T15:51:30.864 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-12T15:51:30.874 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-12T15:51:30.879 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-12T15:51:30.891 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-12T15:51:30.906 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-12T15:51:30.941 INFO:teuthology.orchestra.run.smithi130.stderr:sudo: ntpd: command not found 2023-09-12T15:51:30.952 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2023-09-12T15:51:30.952 INFO:teuthology.orchestra.run.smithi130.stdout:506 Cannot talk to daemon 2023-09-12T15:51:30.964 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2023-09-12T15:51:30.966 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-12T15:51:30.979 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-12T15:51:30.980 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-12T15:51:30.984 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2023-09-12T15:51:30.993 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-12T15:51:30.996 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2023-09-12T15:51:31.010 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-12T15:51:31.024 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-12T15:51:31.052 INFO:teuthology.orchestra.run.smithi130.stderr:bash: ntpq: command not found 2023-09-12T15:51:31.057 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-12T15:51:31.057 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2023-09-12T15:51:31.057 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.057 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.057 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.058 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.079 INFO:teuthology.orchestra.run.smithi159.stderr:bash: ntpq: command not found 2023-09-12T15:51:31.082 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-12T15:51:31.082 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2023-09-12T15:51:31.082 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.082 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.083 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.083 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.104 INFO:teuthology.orchestra.run.smithi203.stderr:bash: ntpq: command not found 2023-09-12T15:51:31.108 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-12T15:51:31.108 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2023-09-12T15:51:31.108 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.108 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.108 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.108 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T15:51:31.109 INFO:teuthology.run_tasks:Running task pexec... 2023-09-12T15:51:31.118 INFO:teuthology.task.pexec:Executing custom commands... 2023-09-12T15:51:31.119 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi130.front.sepia.ceph.com 2023-09-12T15:51:31.119 DEBUG:teuthology.orchestra.run.smithi130:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-12T15:51:31.119 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi159.front.sepia.ceph.com 2023-09-12T15:51:31.120 DEBUG:teuthology.orchestra.run.smithi159:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-12T15:51:31.120 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi203.front.sepia.ceph.com 2023-09-12T15:51:31.121 DEBUG:teuthology.orchestra.run.smithi203:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-12T15:51:31.490 INFO:teuthology.orchestra.run.smithi130.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:31.498 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:31.517 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:32.304 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:41 ago on Tue 12 Sep 2023 03:49:50 PM UTC. 2023-09-12T15:51:32.319 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:29 ago on Tue 12 Sep 2023 03:50:02 PM UTC. 2023-09-12T15:51:32.329 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:29 ago on Tue 12 Sep 2023 03:50:02 PM UTC. 2023-09-12T15:51:34.082 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-09-12T15:51:34.083 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2023-09-12T15:51:34.083 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2023-09-12T15:51:34.083 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2023-09-12T15:51:34.083 INFO:teuthology.orchestra.run.smithi203.stdout:Resetting modules: 2023-09-12T15:51:34.084 INFO:teuthology.orchestra.run.smithi203.stdout: container-tools 2023-09-12T15:51:34.084 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:51:34.084 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2023-09-12T15:51:34.084 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2023-09-12T15:51:34.084 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:51:34.090 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-09-12T15:51:34.091 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-09-12T15:51:34.091 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2023-09-12T15:51:34.091 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-09-12T15:51:34.092 INFO:teuthology.orchestra.run.smithi159.stdout:Resetting modules: 2023-09-12T15:51:34.092 INFO:teuthology.orchestra.run.smithi159.stdout: container-tools 2023-09-12T15:51:34.092 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:51:34.092 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-09-12T15:51:34.092 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-09-12T15:51:34.092 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:51:34.186 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2023-09-12T15:51:34.187 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2023-09-12T15:51:34.187 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2023-09-12T15:51:34.188 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2023-09-12T15:51:34.188 INFO:teuthology.orchestra.run.smithi130.stdout:Resetting modules: 2023-09-12T15:51:34.188 INFO:teuthology.orchestra.run.smithi130.stdout: container-tools 2023-09-12T15:51:34.188 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:51:34.188 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2023-09-12T15:51:34.188 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2023-09-12T15:51:34.189 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:51:35.047 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:32 ago on Tue 12 Sep 2023 03:50:02 PM UTC. 2023-09-12T15:51:35.066 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:32 ago on Tue 12 Sep 2023 03:50:02 PM UTC. 2023-09-12T15:51:35.127 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:44 ago on Tue 12 Sep 2023 03:49:50 PM UTC. 2023-09-12T15:51:37.460 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-09-12T15:51:37.931 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:38.044 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-09-12T15:51:38.093 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2023-09-12T15:51:38.513 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:38.709 INFO:teuthology.orchestra.run.smithi130.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:38.728 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:36 ago on Tue 12 Sep 2023 03:50:02 PM UTC. 2023-09-12T15:51:39.289 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:36 ago on Tue 12 Sep 2023 03:50:02 PM UTC. 2023-09-12T15:51:39.484 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:49 ago on Tue 12 Sep 2023 03:49:50 PM UTC. 2023-09-12T15:51:40.471 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-09-12T15:51:40.563 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================================================ 2023-09-12T15:51:40.563 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2023-09-12T15:51:40.563 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================================================ 2023-09-12T15:51:40.563 INFO:teuthology.orchestra.run.smithi203.stdout:Installing group/module packages: 2023-09-12T15:51:40.563 INFO:teuthology.orchestra.run.smithi203.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-12T15:51:40.564 INFO:teuthology.orchestra.run.smithi203.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-12T15:51:40.564 INFO:teuthology.orchestra.run.smithi203.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-12T15:51:40.564 INFO:teuthology.orchestra.run.smithi203.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-12T15:51:40.564 INFO:teuthology.orchestra.run.smithi203.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-12T15:51:40.564 INFO:teuthology.orchestra.run.smithi203.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-12T15:51:40.565 INFO:teuthology.orchestra.run.smithi203.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-12T15:51:40.565 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2023-09-12T15:51:40.565 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-12T15:51:40.565 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-12T15:51:40.565 INFO:teuthology.orchestra.run.smithi203.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-12T15:51:40.566 INFO:teuthology.orchestra.run.smithi203.stdout:Installing module profiles: 2023-09-12T15:51:40.566 INFO:teuthology.orchestra.run.smithi203.stdout: container-tools/common 2023-09-12T15:51:40.566 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:51:40.566 INFO:teuthology.orchestra.run.smithi203.stdout:Enabling module streams: 2023-09-12T15:51:40.566 INFO:teuthology.orchestra.run.smithi203.stdout: container-tools rhel8 2023-09-12T15:51:40.567 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:51:40.567 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2023-09-12T15:51:40.567 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================================================ 2023-09-12T15:51:40.567 INFO:teuthology.orchestra.run.smithi203.stdout:Install 10 Packages 2023-09-12T15:51:40.567 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:51:40.568 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 18 M 2023-09-12T15:51:40.568 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 65 M 2023-09-12T15:51:40.568 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2023-09-12T15:51:40.651 INFO:teuthology.orchestra.run.smithi203.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 475 kB/s | 41 kB 00:00 2023-09-12T15:51:40.676 INFO:teuthology.orchestra.run.smithi203.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 223 kB/s | 25 kB 00:00 2023-09-12T15:51:40.702 INFO:teuthology.orchestra.run.smithi203.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 687 kB/s | 94 kB 00:00 2023-09-12T15:51:40.727 INFO:teuthology.orchestra.run.smithi203.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-09-12T15:51:40.769 INFO:teuthology.orchestra.run.smithi203.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.5 MB/s | 524 kB 00:00 2023-09-12T15:51:40.786 INFO:teuthology.orchestra.run.smithi203.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-12T15:51:40.820 INFO:teuthology.orchestra.run.smithi203.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-09-12T15:51:40.962 INFO:teuthology.orchestra.run.smithi203.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 15 MB/s | 2.2 MB 00:00 2023-09-12T15:51:41.070 INFO:teuthology.orchestra.run.smithi203.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 22 MB/s | 8.0 MB 00:00 2023-09-12T15:51:41.153 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-09-12T15:51:41.154 INFO:teuthology.orchestra.run.smithi203.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 19 MB/s | 7.0 MB 00:00 2023-09-12T15:51:41.154 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2023-09-12T15:51:41.155 INFO:teuthology.orchestra.run.smithi203.stdout:Total 31 MB/s | 18 MB 00:00 2023-09-12T15:51:41.193 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2023-09-12T15:51:41.246 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================================ 2023-09-12T15:51:41.246 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-09-12T15:51:41.246 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================================ 2023-09-12T15:51:41.246 INFO:teuthology.orchestra.run.smithi159.stdout:Installing group/module packages: 2023-09-12T15:51:41.247 INFO:teuthology.orchestra.run.smithi159.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-12T15:51:41.247 INFO:teuthology.orchestra.run.smithi159.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-12T15:51:41.247 INFO:teuthology.orchestra.run.smithi159.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-12T15:51:41.247 INFO:teuthology.orchestra.run.smithi159.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-12T15:51:41.247 INFO:teuthology.orchestra.run.smithi159.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-12T15:51:41.248 INFO:teuthology.orchestra.run.smithi159.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-12T15:51:41.248 INFO:teuthology.orchestra.run.smithi159.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-12T15:51:41.248 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-09-12T15:51:41.248 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-12T15:51:41.248 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-12T15:51:41.249 INFO:teuthology.orchestra.run.smithi159.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-12T15:51:41.249 INFO:teuthology.orchestra.run.smithi159.stdout:Installing module profiles: 2023-09-12T15:51:41.249 INFO:teuthology.orchestra.run.smithi159.stdout: container-tools/common 2023-09-12T15:51:41.249 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:51:41.249 INFO:teuthology.orchestra.run.smithi159.stdout:Enabling module streams: 2023-09-12T15:51:41.249 INFO:teuthology.orchestra.run.smithi159.stdout: container-tools rhel8 2023-09-12T15:51:41.250 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:51:41.250 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-09-12T15:51:41.250 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================================ 2023-09-12T15:51:41.250 INFO:teuthology.orchestra.run.smithi159.stdout:Install 10 Packages 2023-09-12T15:51:41.250 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:51:41.251 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 18 M 2023-09-12T15:51:41.251 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 65 M 2023-09-12T15:51:41.251 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-09-12T15:51:41.285 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================================================ 2023-09-12T15:51:41.286 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2023-09-12T15:51:41.286 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================================================ 2023-09-12T15:51:41.286 INFO:teuthology.orchestra.run.smithi130.stdout:Installing group/module packages: 2023-09-12T15:51:41.286 INFO:teuthology.orchestra.run.smithi130.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-12T15:51:41.286 INFO:teuthology.orchestra.run.smithi130.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-12T15:51:41.287 INFO:teuthology.orchestra.run.smithi130.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-12T15:51:41.287 INFO:teuthology.orchestra.run.smithi130.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-12T15:51:41.287 INFO:teuthology.orchestra.run.smithi130.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-12T15:51:41.287 INFO:teuthology.orchestra.run.smithi130.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-12T15:51:41.287 INFO:teuthology.orchestra.run.smithi130.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-12T15:51:41.288 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2023-09-12T15:51:41.288 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-12T15:51:41.288 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-12T15:51:41.288 INFO:teuthology.orchestra.run.smithi130.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-12T15:51:41.288 INFO:teuthology.orchestra.run.smithi130.stdout:Installing module profiles: 2023-09-12T15:51:41.289 INFO:teuthology.orchestra.run.smithi130.stdout: container-tools/common 2023-09-12T15:51:41.289 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:51:41.289 INFO:teuthology.orchestra.run.smithi130.stdout:Enabling module streams: 2023-09-12T15:51:41.289 INFO:teuthology.orchestra.run.smithi130.stdout: container-tools rhel8 2023-09-12T15:51:41.289 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:51:41.289 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2023-09-12T15:51:41.290 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================================================ 2023-09-12T15:51:41.290 INFO:teuthology.orchestra.run.smithi130.stdout:Install 10 Packages 2023-09-12T15:51:41.290 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:51:41.291 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 18 M 2023-09-12T15:51:41.291 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 65 M 2023-09-12T15:51:41.291 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2023-09-12T15:51:41.317 INFO:teuthology.orchestra.run.smithi159.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 364 kB/s | 25 kB 00:00 2023-09-12T15:51:41.342 INFO:teuthology.orchestra.run.smithi159.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 436 kB/s | 41 kB 00:00 2023-09-12T15:51:41.355 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2023-09-12T15:51:41.362 INFO:teuthology.orchestra.run.smithi130.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-09-12T15:51:41.367 INFO:teuthology.orchestra.run.smithi159.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 790 kB/s | 94 kB 00:00 2023-09-12T15:51:41.374 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2023-09-12T15:51:41.375 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2023-09-12T15:51:41.387 INFO:teuthology.orchestra.run.smithi130.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 250 kB/s | 25 kB 00:00 2023-09-12T15:51:41.392 INFO:teuthology.orchestra.run.smithi159.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-09-12T15:51:41.412 INFO:teuthology.orchestra.run.smithi130.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 328 kB/s | 41 kB 00:00 2023-09-12T15:51:41.426 INFO:teuthology.orchestra.run.smithi159.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-12T15:51:41.437 INFO:teuthology.orchestra.run.smithi130.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-09-12T15:51:41.451 INFO:teuthology.orchestra.run.smithi159.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-12T15:51:41.471 INFO:teuthology.orchestra.run.smithi130.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-12T15:51:41.476 INFO:teuthology.orchestra.run.smithi159.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-09-12T15:51:41.497 INFO:teuthology.orchestra.run.smithi130.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-12T15:51:41.522 INFO:teuthology.orchestra.run.smithi130.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.2 MB/s | 213 kB 00:00 2023-09-12T15:51:41.559 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2023-09-12T15:51:41.560 INFO:teuthology.orchestra.run.smithi159.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 26 MB/s | 2.2 MB 00:00 2023-09-12T15:51:41.563 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2023-09-12T15:51:41.614 INFO:teuthology.orchestra.run.smithi130.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-09-12T15:51:41.677 INFO:teuthology.orchestra.run.smithi159.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 31 MB/s | 7.0 MB 00:00 2023-09-12T15:51:41.722 INFO:teuthology.orchestra.run.smithi130.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 31 MB/s | 7.0 MB 00:00 2023-09-12T15:51:41.760 INFO:teuthology.orchestra.run.smithi159.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 20 MB/s | 8.0 MB 00:00 2023-09-12T15:51:41.761 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-09-12T15:51:41.761 INFO:teuthology.orchestra.run.smithi159.stdout:Total 36 MB/s | 18 MB 00:00 2023-09-12T15:51:41.848 INFO:teuthology.orchestra.run.smithi130.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-12T15:51:41.848 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2023-09-12T15:51:41.849 INFO:teuthology.orchestra.run.smithi130.stdout:Total 33 MB/s | 18 MB 00:00 2023-09-12T15:51:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-09-12T15:51:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-09-12T15:51:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-09-12T15:51:42.048 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2023-09-12T15:51:42.067 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2023-09-12T15:51:42.067 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2023-09-12T15:51:42.093 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2023-09-12T15:51:42.164 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-09-12T15:51:42.168 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-09-12T15:51:42.248 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2023-09-12T15:51:42.252 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2023-09-12T15:51:42.299 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-12T15:51:42.459 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-12T15:51:42.678 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-12T15:51:42.716 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-09-12T15:51:42.736 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2023-09-12T15:51:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-12T15:51:42.896 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-12T15:51:42.901 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-12T15:51:43.070 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-12T15:51:43.101 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-12T15:51:43.247 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-12T15:51:43.295 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-12T15:51:43.321 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-12T15:51:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-12T15:51:43.498 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-12T15:51:43.807 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-12T15:51:43.830 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-12T15:51:44.145 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-12T15:51:44.592 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-12T15:51:44.597 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-12T15:51:45.065 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-12T15:51:45.283 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-12T15:51:45.441 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-12T15:51:45.470 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-12T15:51:45.483 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-12T15:51:45.644 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-12T15:51:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-12T15:51:45.678 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-12T15:51:45.800 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-12T15:51:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-12T15:51:45.909 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-12T15:51:45.909 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-12T15:51:45.909 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-12T15:51:45.909 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-12T15:51:45.910 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-12T15:51:45.910 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-12T15:51:45.910 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-12T15:51:45.910 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-12T15:51:45.910 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-12T15:51:45.911 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-12T15:51:45.960 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-12T15:51:46.054 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-12T15:51:46.227 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-12T15:51:46.227 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-12T15:51:46.228 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-12T15:51:46.228 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-12T15:51:46.228 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-12T15:51:46.228 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-12T15:51:46.228 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-12T15:51:46.229 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-12T15:51:46.229 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-12T15:51:46.229 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-12T15:51:46.323 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-12T15:51:46.323 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-12T15:51:46.324 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-12T15:51:46.324 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-12T15:51:46.324 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-12T15:51:46.324 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-12T15:51:46.324 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-12T15:51:46.325 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-12T15:51:46.325 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-12T15:51:46.325 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-12T15:51:47.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-12T15:51:47.099 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:44 ago on Tue 12 Sep 2023 03:50:02 PM UTC. 2023-09-12T15:51:47.360 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-12T15:51:47.360 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:56 ago on Tue 12 Sep 2023 03:49:50 PM UTC. 2023-09-12T15:51:47.452 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-12T15:51:47.452 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:44 ago on Tue 12 Sep 2023 03:50:02 PM UTC. 2023-09-12T15:51:50.199 INFO:teuthology.orchestra.run.smithi203.stdout:Installed products updated. 2023-09-12T15:51:50.390 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:51:50.390 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2023-09-12T15:51:50.391 INFO:teuthology.orchestra.run.smithi203.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:50.391 INFO:teuthology.orchestra.run.smithi203.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-12T15:51:50.391 INFO:teuthology.orchestra.run.smithi203.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:50.391 INFO:teuthology.orchestra.run.smithi203.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-12T15:51:50.391 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-12T15:51:50.392 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-12T15:51:50.392 INFO:teuthology.orchestra.run.smithi203.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:50.392 INFO:teuthology.orchestra.run.smithi203.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:50.392 INFO:teuthology.orchestra.run.smithi203.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-12T15:51:50.392 INFO:teuthology.orchestra.run.smithi203.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-12T15:51:50.392 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:51:50.393 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-09-12T15:51:50.523 DEBUG:teuthology.parallel:result is None 2023-09-12T15:51:51.005 INFO:teuthology.orchestra.run.smithi130.stdout:Installed products updated. 2023-09-12T15:51:51.065 INFO:teuthology.orchestra.run.smithi159.stdout:Installed products updated. 2023-09-12T15:51:51.197 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:51:51.198 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2023-09-12T15:51:51.198 INFO:teuthology.orchestra.run.smithi130.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:51.198 INFO:teuthology.orchestra.run.smithi130.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-12T15:51:51.198 INFO:teuthology.orchestra.run.smithi130.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:51.199 INFO:teuthology.orchestra.run.smithi130.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-12T15:51:51.199 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-12T15:51:51.199 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-12T15:51:51.199 INFO:teuthology.orchestra.run.smithi130.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:51.199 INFO:teuthology.orchestra.run.smithi130.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:51.200 INFO:teuthology.orchestra.run.smithi130.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-12T15:51:51.200 INFO:teuthology.orchestra.run.smithi130.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-12T15:51:51.200 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:51:51.200 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2023-09-12T15:51:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:51:51.268 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-09-12T15:51:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-12T15:51:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-12T15:51:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-12T15:51:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-12T15:51:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-12T15:51:51.270 INFO:teuthology.orchestra.run.smithi159.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-12T15:51:51.270 INFO:teuthology.orchestra.run.smithi159.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-12T15:51:51.270 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:51:51.270 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-09-12T15:51:51.331 DEBUG:teuthology.parallel:result is None 2023-09-12T15:51:51.415 DEBUG:teuthology.parallel:result is None 2023-09-12T15:51:51.416 INFO:teuthology.run_tasks:Running task install... 2023-09-12T15:51:51.425 DEBUG:teuthology.task.install:project ceph 2023-09-12T15:51:51.425 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0'}} 2023-09-12T15:51:51.425 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0'} 2023-09-12T15:51:51.426 INFO:teuthology.task.install:Using flavor: default 2023-09-12T15:51:51.432 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-12T15:51:51.433 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-12T15:51:51.433 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'tag': None, 'wait_for_package': False} 2023-09-12T15:51:51.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:51:51.436 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'tag': None, 'wait_for_package': False} 2023-09-12T15:51:51.436 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:51:51.437 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'tag': None, 'wait_for_package': False} 2023-09-12T15:51:51.438 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:51:51.612 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/ 2023-09-12T15:51:51.612 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6088.g2110e007 2023-09-12T15:51:51.620 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/ 2023-09-12T15:51:51.620 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6088.g2110e007 2023-09-12T15:51:51.723 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-12T15:51:51.724 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:51:51.724 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-12T15:51:51.741 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/ 2023-09-12T15:51:51.741 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6088.g2110e007 2023-09-12T15:51:51.752 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-12T15:51:51.752 DEBUG:teuthology.orchestra.run.smithi130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2110e00747b31a0d73768c9e5229da18b26b8aa0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-12T15:51:51.820 DEBUG:teuthology.orchestra.run.smithi130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-12T15:51:51.860 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-12T15:51:51.860 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:51:51.861 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-12T15:51:51.890 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-12T15:51:51.890 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2110e00747b31a0d73768c9e5229da18b26b8aa0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-12T15:51:51.897 DEBUG:teuthology.orchestra.run.smithi130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-12T15:51:51.959 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-12T15:51:51.964 INFO:teuthology.orchestra.run.smithi130.stdout:check_obsoletes = 1 2023-09-12T15:51:51.965 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum clean all 2023-09-12T15:51:52.000 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-12T15:51:52.000 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:51:52.000 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-12T15:51:52.033 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-12T15:51:52.033 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2110e00747b31a0d73768c9e5229da18b26b8aa0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-12T15:51:52.037 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-12T15:51:52.103 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2023-09-12T15:51:52.105 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-12T15:51:52.105 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2023-09-12T15:51:52.186 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-12T15:51:52.254 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2023-09-12T15:51:52.255 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2023-09-12T15:51:52.334 INFO:teuthology.orchestra.run.smithi130.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:52.479 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:52.589 INFO:teuthology.orchestra.run.smithi130.stdout:44 files removed 2023-09-12T15:51:52.629 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:52.635 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-12T15:51:52.755 INFO:teuthology.orchestra.run.smithi159.stdout:44 files removed 2023-09-12T15:51:52.802 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-12T15:51:52.905 INFO:teuthology.orchestra.run.smithi203.stdout:44 files removed 2023-09-12T15:51:52.953 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-12T15:51:52.965 INFO:teuthology.orchestra.run.smithi130.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:53.136 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:53.285 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-09-12T15:51:53.872 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 326 kB/s | 81 kB 00:00 2023-09-12T15:51:53.886 INFO:teuthology.orchestra.run.smithi130.stdout:ceph packages for x86_64 176 kB/s | 81 kB 00:00 2023-09-12T15:51:54.303 INFO:teuthology.orchestra.run.smithi130.stdout:ceph noarch packages 24 kB/s | 9.4 kB 00:00 2023-09-12T15:51:54.314 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-09-12T15:51:54.342 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 126 kB/s | 81 kB 00:00 2023-09-12T15:51:54.735 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-09-12T15:51:54.879 INFO:teuthology.orchestra.run.smithi130.stdout:ceph source packages 2.7 kB/s | 1.5 kB 00:00 2023-09-12T15:51:54.890 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 2.7 kB/s | 1.5 kB 00:00 2023-09-12T15:51:55.155 INFO:teuthology.orchestra.run.smithi130.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-09-12T15:51:55.236 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 3.1 kB/s | 1.5 kB 00:00 2023-09-12T15:51:55.316 INFO:teuthology.orchestra.run.smithi159.stdout:Copr repo for python3-asyncssh owned by ceph 8.5 kB/s | 3.5 kB 00:00 2023-09-12T15:51:55.461 INFO:teuthology.orchestra.run.smithi203.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2023-09-12T15:51:56.916 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 16 MB 00:01 2023-09-12T15:51:57.341 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 16 MB 00:01 2023-09-12T15:51:57.466 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 16 MB 00:02 2023-09-12T15:52:00.364 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 391 kB/s | 24 kB 00:00 2023-09-12T15:52:00.787 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 579 kB/s | 24 kB 00:00 2023-09-12T15:52:00.925 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 387 kB/s | 24 kB 00:00 2023-09-12T15:52:01.282 INFO:teuthology.orchestra.run.smithi130.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-09-12T15:52:01.832 INFO:teuthology.orchestra.run.smithi203.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 54 MB/s | 53 MB 00:00 2023-09-12T15:52:02.085 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-09-12T15:52:09.683 INFO:teuthology.orchestra.run.smithi130.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-09-12T15:52:10.236 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 53 MB 00:01 2023-09-12T15:52:10.274 INFO:teuthology.orchestra.run.smithi203.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-09-12T15:52:20.734 INFO:teuthology.orchestra.run.smithi130.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-12T15:52:20.734 INFO:teuthology.orchestra.run.smithi130.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-12T15:52:20.749 INFO:teuthology.orchestra.run.smithi130.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-12T15:52:20.750 INFO:teuthology.orchestra.run.smithi130.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-12T15:52:20.751 INFO:teuthology.orchestra.run.smithi130.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-12T15:52:20.752 INFO:teuthology.orchestra.run.smithi130.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-12T15:52:20.754 INFO:teuthology.orchestra.run.smithi130.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-12T15:52:20.758 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-12T15:52:20.759 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-12T15:52:20.890 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2023-09-12T15:52:20.902 INFO:teuthology.orchestra.run.smithi130.stdout:=========================================================================================================================== 2023-09-12T15:52:20.906 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2023-09-12T15:52:20.906 INFO:teuthology.orchestra.run.smithi130.stdout:=========================================================================================================================== 2023-09-12T15:52:20.906 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2023-09-12T15:52:20.906 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-12T15:52:20.907 INFO:teuthology.orchestra.run.smithi130.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-12T15:52:20.907 INFO:teuthology.orchestra.run.smithi130.stdout: ceph x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.4 k 2023-09-12T15:52:20.907 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base x86_64 2:18.0.0-6088.g2110e007.el8 ceph 5.4 M 2023-09-12T15:52:20.907 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 907 k 2023-09-12T15:52:20.907 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6088.g2110e007.el8 ceph 167 k 2023-09-12T15:52:20.907 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr x86_64 2:18.0.0-6088.g2110e007.el8 ceph 1.6 M 2023-09-12T15:52:20.908 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 132 k 2023-09-12T15:52:20.908 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 1.7 M 2023-09-12T15:52:20.908 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 7.4 M 2023-09-12T15:52:20.908 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 47 k 2023-09-12T15:52:20.908 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 14 M 2023-09-12T15:52:20.909 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test x86_64 2:18.0.0-6088.g2110e007.el8 ceph 49 M 2023-09-12T15:52:20.909 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 261 k 2023-09-12T15:52:20.909 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 177 k 2023-09-12T15:52:20.909 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 261 k 2023-09-12T15:52:20.909 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 18 k 2023-09-12T15:52:20.910 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-12T15:52:20.910 INFO:teuthology.orchestra.run.smithi130.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-12T15:52:20.910 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-12T15:52:20.910 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 31 k 2023-09-12T15:52:20.911 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 827 k 2023-09-12T15:52:20.911 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 139 k 2023-09-12T15:52:20.911 INFO:teuthology.orchestra.run.smithi130.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-12T15:52:20.911 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-12T15:52:20.911 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-12T15:52:20.912 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-12T15:52:20.912 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-12T15:52:20.912 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-12T15:52:20.912 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs x86_64 2:18.0.0-6088.g2110e007.el8 ceph 197 k 2023-09-12T15:52:20.912 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados x86_64 2:18.0.0-6088.g2110e007.el8 ceph 371 k 2023-09-12T15:52:20.912 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 364 k 2023-09-12T15:52:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 106 k 2023-09-12T15:52:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 94 k 2023-09-12T15:52:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.6 M 2023-09-12T15:52:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 179 k 2023-09-12T15:52:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-12T15:52:20.914 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrading: 2023-09-12T15:52:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-12T15:52:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-12T15:52:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-12T15:52:20.914 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2023-09-12T15:52:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-12T15:52:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 22 M 2023-09-12T15:52:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 23 k 2023-09-12T15:52:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds x86_64 2:18.0.0-6088.g2110e007.el8 ceph 2.3 M 2023-09-12T15:52:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 242 k 2023-09-12T15:52:20.916 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon x86_64 2:18.0.0-6088.g2110e007.el8 ceph 4.6 M 2023-09-12T15:52:20.916 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 18 M 2023-09-12T15:52:20.916 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 15 k 2023-09-12T15:52:20.916 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux x86_64 2:18.0.0-6088.g2110e007.el8 ceph 24 k 2023-09-12T15:52:20.916 INFO:teuthology.orchestra.run.smithi130.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-12T15:52:20.917 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-12T15:52:20.917 INFO:teuthology.orchestra.run.smithi130.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-12T15:52:20.917 INFO:teuthology.orchestra.run.smithi130.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-12T15:52:20.917 INFO:teuthology.orchestra.run.smithi130.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-12T15:52:20.917 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite x86_64 2:18.0.0-6088.g2110e007.el8 ceph 184 k 2023-09-12T15:52:20.918 INFO:teuthology.orchestra.run.smithi130.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-12T15:52:20.918 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-12T15:52:20.918 INFO:teuthology.orchestra.run.smithi130.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-12T15:52:20.918 INFO:teuthology.orchestra.run.smithi130.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-12T15:52:20.918 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-12T15:52:20.919 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-12T15:52:20.919 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 519 k 2023-09-12T15:52:20.919 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-12T15:52:20.919 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.8 M 2023-09-12T15:52:20.919 INFO:teuthology.orchestra.run.smithi130.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-12T15:52:20.920 INFO:teuthology.orchestra.run.smithi130.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-12T15:52:20.920 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-12T15:52:20.920 INFO:teuthology.orchestra.run.smithi130.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-12T15:52:20.920 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-12T15:52:20.920 INFO:teuthology.orchestra.run.smithi130.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-12T15:52:20.921 INFO:teuthology.orchestra.run.smithi130.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-12T15:52:20.921 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-12T15:52:20.921 INFO:teuthology.orchestra.run.smithi130.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-12T15:52:20.921 INFO:teuthology.orchestra.run.smithi130.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-12T15:52:20.921 INFO:teuthology.orchestra.run.smithi130.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-12T15:52:20.921 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-12T15:52:20.922 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-12T15:52:20.922 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-12T15:52:20.922 INFO:teuthology.orchestra.run.smithi130.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-12T15:52:20.922 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-12T15:52:20.922 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 45 k 2023-09-12T15:52:20.923 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 125 k 2023-09-12T15:52:20.923 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-12T15:52:20.923 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-12T15:52:20.923 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-12T15:52:20.923 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-12T15:52:20.924 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-12T15:52:20.924 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-12T15:52:20.924 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-12T15:52:20.924 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-12T15:52:20.924 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-12T15:52:20.925 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-12T15:52:20.925 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-12T15:52:20.925 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-12T15:52:20.925 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-12T15:52:20.925 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-12T15:52:20.926 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-12T15:52:20.926 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-12T15:52:20.926 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-12T15:52:20.926 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-12T15:52:20.926 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-12T15:52:20.927 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-12T15:52:20.927 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-12T15:52:20.927 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-12T15:52:20.927 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-12T15:52:20.927 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-12T15:52:20.928 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-12T15:52:20.928 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-12T15:52:20.928 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-12T15:52:20.928 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-12T15:52:20.928 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-12T15:52:20.928 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-12T15:52:20.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-12T15:52:20.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-12T15:52:20.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-12T15:52:20.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-12T15:52:20.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-12T15:52:20.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-12T15:52:20.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-12T15:52:20.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-12T15:52:20.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-12T15:52:20.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-12T15:52:20.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-12T15:52:20.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-12T15:52:20.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-12T15:52:20.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-12T15:52:20.931 INFO:teuthology.orchestra.run.smithi130.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-12T15:52:20.932 INFO:teuthology.orchestra.run.smithi130.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-12T15:52:20.932 INFO:teuthology.orchestra.run.smithi130.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-12T15:52:20.932 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-12T15:52:20.932 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-12T15:52:20.932 INFO:teuthology.orchestra.run.smithi130.stdout:Enabling module streams: 2023-09-12T15:52:20.932 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql 10 2023-09-12T15:52:20.933 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:52:20.933 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2023-09-12T15:52:20.933 INFO:teuthology.orchestra.run.smithi130.stdout:=========================================================================================================================== 2023-09-12T15:52:20.933 INFO:teuthology.orchestra.run.smithi130.stdout:Install 128 Packages 2023-09-12T15:52:20.933 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrade 3 Packages 2023-09-12T15:52:20.934 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:52:20.934 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 208 M 2023-09-12T15:52:20.935 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2023-09-12T15:52:21.142 INFO:teuthology.orchestra.run.smithi130.stdout:(1/131): ceph-18.0.0-6088.g2110e007.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2023-09-12T15:52:21.297 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-12T15:52:21.298 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-12T15:52:21.314 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-12T15:52:21.315 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-12T15:52:21.315 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-12T15:52:21.316 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-12T15:52:21.317 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-12T15:52:21.320 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-12T15:52:21.321 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-12T15:52:21.373 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-12T15:52:21.374 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-12T15:52:21.389 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-12T15:52:21.391 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-12T15:52:21.392 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-12T15:52:21.393 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-12T15:52:21.395 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-12T15:52:21.399 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-12T15:52:21.400 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-12T15:52:21.426 INFO:teuthology.orchestra.run.smithi130.stdout:(2/131): ceph-fuse-18.0.0-6088.g2110e007.el8.x8 3.1 MB/s | 907 kB 00:00 2023-09-12T15:52:21.452 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-09-12T15:52:21.464 INFO:teuthology.orchestra.run.smithi159.stdout:=========================================================================================================================== 2023-09-12T15:52:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-09-12T15:52:21.465 INFO:teuthology.orchestra.run.smithi159.stdout:=========================================================================================================================== 2023-09-12T15:52:21.465 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-09-12T15:52:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-12T15:52:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-12T15:52:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.4 k 2023-09-12T15:52:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:18.0.0-6088.g2110e007.el8 ceph 5.4 M 2023-09-12T15:52:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 907 k 2023-09-12T15:52:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6088.g2110e007.el8 ceph 167 k 2023-09-12T15:52:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:18.0.0-6088.g2110e007.el8 ceph 1.6 M 2023-09-12T15:52:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 132 k 2023-09-12T15:52:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 1.7 M 2023-09-12T15:52:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 7.4 M 2023-09-12T15:52:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 47 k 2023-09-12T15:52:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 14 M 2023-09-12T15:52:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:18.0.0-6088.g2110e007.el8 ceph 49 M 2023-09-12T15:52:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 261 k 2023-09-12T15:52:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 177 k 2023-09-12T15:52:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 261 k 2023-09-12T15:52:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 18 k 2023-09-12T15:52:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-12T15:52:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-12T15:52:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-12T15:52:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 31 k 2023-09-12T15:52:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 827 k 2023-09-12T15:52:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 139 k 2023-09-12T15:52:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-12T15:52:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-12T15:52:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-12T15:52:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-12T15:52:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-12T15:52:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-12T15:52:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:18.0.0-6088.g2110e007.el8 ceph 197 k 2023-09-12T15:52:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:18.0.0-6088.g2110e007.el8 ceph 371 k 2023-09-12T15:52:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 364 k 2023-09-12T15:52:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 106 k 2023-09-12T15:52:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 94 k 2023-09-12T15:52:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.6 M 2023-09-12T15:52:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 179 k 2023-09-12T15:52:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-12T15:52:21.474 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2023-09-12T15:52:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-12T15:52:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-12T15:52:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-12T15:52:21.474 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-09-12T15:52:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-12T15:52:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 22 M 2023-09-12T15:52:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 23 k 2023-09-12T15:52:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:18.0.0-6088.g2110e007.el8 ceph 2.3 M 2023-09-12T15:52:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 242 k 2023-09-12T15:52:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:18.0.0-6088.g2110e007.el8 ceph 4.6 M 2023-09-12T15:52:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 18 M 2023-09-12T15:52:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 15 k 2023-09-12T15:52:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:18.0.0-6088.g2110e007.el8 ceph 24 k 2023-09-12T15:52:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-12T15:52:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-12T15:52:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-12T15:52:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-12T15:52:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-12T15:52:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:18.0.0-6088.g2110e007.el8 ceph 184 k 2023-09-12T15:52:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-12T15:52:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-12T15:52:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-12T15:52:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-12T15:52:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-12T15:52:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-12T15:52:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 519 k 2023-09-12T15:52:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-12T15:52:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.8 M 2023-09-12T15:52:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-12T15:52:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-12T15:52:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-12T15:52:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-12T15:52:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-12T15:52:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-12T15:52:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-12T15:52:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-12T15:52:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-12T15:52:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-12T15:52:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-12T15:52:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-12T15:52:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-12T15:52:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-12T15:52:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-12T15:52:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-12T15:52:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 45 k 2023-09-12T15:52:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 125 k 2023-09-12T15:52:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-12T15:52:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-12T15:52:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-12T15:52:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-12T15:52:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-12T15:52:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-12T15:52:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-12T15:52:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-12T15:52:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-12T15:52:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-12T15:52:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-12T15:52:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-12T15:52:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-12T15:52:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-12T15:52:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-12T15:52:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-12T15:52:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-12T15:52:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-12T15:52:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-12T15:52:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-12T15:52:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-12T15:52:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-12T15:52:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-12T15:52:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-12T15:52:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-12T15:52:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-12T15:52:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-12T15:52:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-12T15:52:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-12T15:52:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-12T15:52:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-12T15:52:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-12T15:52:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-12T15:52:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-12T15:52:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-12T15:52:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-12T15:52:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-12T15:52:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-12T15:52:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-12T15:52:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-12T15:52:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-12T15:52:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-12T15:52:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-12T15:52:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-12T15:52:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-12T15:52:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-12T15:52:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-12T15:52:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-12T15:52:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-12T15:52:21.498 INFO:teuthology.orchestra.run.smithi159.stdout:Enabling module streams: 2023-09-12T15:52:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql 10 2023-09-12T15:52:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:52:21.499 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-09-12T15:52:21.499 INFO:teuthology.orchestra.run.smithi159.stdout:=========================================================================================================================== 2023-09-12T15:52:21.499 INFO:teuthology.orchestra.run.smithi159.stdout:Install 128 Packages 2023-09-12T15:52:21.500 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 3 Packages 2023-09-12T15:52:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:52:21.500 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 208 M 2023-09-12T15:52:21.501 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-09-12T15:52:21.518 INFO:teuthology.orchestra.run.smithi130.stdout:(3/131): ceph-immutable-object-cache-18.0.0-608 1.8 MB/s | 167 kB 00:00 2023-09-12T15:52:21.535 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-09-12T15:52:21.547 INFO:teuthology.orchestra.run.smithi203.stdout:=========================================================================================================================== 2023-09-12T15:52:21.548 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2023-09-12T15:52:21.548 INFO:teuthology.orchestra.run.smithi203.stdout:=========================================================================================================================== 2023-09-12T15:52:21.548 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2023-09-12T15:52:21.548 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-12T15:52:21.548 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-12T15:52:21.549 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.4 k 2023-09-12T15:52:21.549 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:18.0.0-6088.g2110e007.el8 ceph 5.4 M 2023-09-12T15:52:21.549 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 907 k 2023-09-12T15:52:21.549 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6088.g2110e007.el8 ceph 167 k 2023-09-12T15:52:21.549 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:18.0.0-6088.g2110e007.el8 ceph 1.6 M 2023-09-12T15:52:21.549 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 132 k 2023-09-12T15:52:21.550 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 1.7 M 2023-09-12T15:52:21.550 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 7.4 M 2023-09-12T15:52:21.550 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 47 k 2023-09-12T15:52:21.550 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 14 M 2023-09-12T15:52:21.550 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:18.0.0-6088.g2110e007.el8 ceph 49 M 2023-09-12T15:52:21.551 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 261 k 2023-09-12T15:52:21.551 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 177 k 2023-09-12T15:52:21.551 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 261 k 2023-09-12T15:52:21.551 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 18 k 2023-09-12T15:52:21.551 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-12T15:52:21.552 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-12T15:52:21.552 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-12T15:52:21.552 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 31 k 2023-09-12T15:52:21.552 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 827 k 2023-09-12T15:52:21.552 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 139 k 2023-09-12T15:52:21.552 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-12T15:52:21.553 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-12T15:52:21.553 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-12T15:52:21.553 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-12T15:52:21.553 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-12T15:52:21.554 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-12T15:52:21.554 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:18.0.0-6088.g2110e007.el8 ceph 197 k 2023-09-12T15:52:21.554 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:18.0.0-6088.g2110e007.el8 ceph 371 k 2023-09-12T15:52:21.554 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 364 k 2023-09-12T15:52:21.554 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 106 k 2023-09-12T15:52:21.554 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 94 k 2023-09-12T15:52:21.555 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.6 M 2023-09-12T15:52:21.555 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 179 k 2023-09-12T15:52:21.555 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-12T15:52:21.555 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2023-09-12T15:52:21.555 INFO:teuthology.orchestra.run.smithi203.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-12T15:52:21.556 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-12T15:52:21.556 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-12T15:52:21.556 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2023-09-12T15:52:21.556 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-12T15:52:21.556 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 22 M 2023-09-12T15:52:21.557 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 23 k 2023-09-12T15:52:21.557 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:18.0.0-6088.g2110e007.el8 ceph 2.3 M 2023-09-12T15:52:21.557 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 242 k 2023-09-12T15:52:21.557 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:18.0.0-6088.g2110e007.el8 ceph 4.6 M 2023-09-12T15:52:21.557 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 18 M 2023-09-12T15:52:21.557 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 15 k 2023-09-12T15:52:21.558 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:18.0.0-6088.g2110e007.el8 ceph 24 k 2023-09-12T15:52:21.558 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-12T15:52:21.558 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-12T15:52:21.558 INFO:teuthology.orchestra.run.smithi203.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-12T15:52:21.558 INFO:teuthology.orchestra.run.smithi203.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-12T15:52:21.559 INFO:teuthology.orchestra.run.smithi203.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-12T15:52:21.559 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:18.0.0-6088.g2110e007.el8 ceph 184 k 2023-09-12T15:52:21.559 INFO:teuthology.orchestra.run.smithi203.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-12T15:52:21.559 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-12T15:52:21.559 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-12T15:52:21.559 INFO:teuthology.orchestra.run.smithi203.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-12T15:52:21.560 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-12T15:52:21.560 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-12T15:52:21.560 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 519 k 2023-09-12T15:52:21.560 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-12T15:52:21.560 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.8 M 2023-09-12T15:52:21.561 INFO:teuthology.orchestra.run.smithi203.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-12T15:52:21.561 INFO:teuthology.orchestra.run.smithi203.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-12T15:52:21.561 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-12T15:52:21.561 INFO:teuthology.orchestra.run.smithi203.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-12T15:52:21.561 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-12T15:52:21.562 INFO:teuthology.orchestra.run.smithi203.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-12T15:52:21.562 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-12T15:52:21.562 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-12T15:52:21.562 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-12T15:52:21.562 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-12T15:52:21.562 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-12T15:52:21.563 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-12T15:52:21.563 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-12T15:52:21.563 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-12T15:52:21.563 INFO:teuthology.orchestra.run.smithi203.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-12T15:52:21.563 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-12T15:52:21.564 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 45 k 2023-09-12T15:52:21.564 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 125 k 2023-09-12T15:52:21.564 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-12T15:52:21.564 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-12T15:52:21.564 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-12T15:52:21.565 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-12T15:52:21.565 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-12T15:52:21.565 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-12T15:52:21.565 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-12T15:52:21.565 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-12T15:52:21.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-12T15:52:21.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-12T15:52:21.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-12T15:52:21.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-12T15:52:21.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-12T15:52:21.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-12T15:52:21.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-12T15:52:21.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-12T15:52:21.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-12T15:52:21.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-12T15:52:21.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-12T15:52:21.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-12T15:52:21.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-12T15:52:21.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-12T15:52:21.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-12T15:52:21.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-12T15:52:21.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-12T15:52:21.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-12T15:52:21.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-12T15:52:21.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-12T15:52:21.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-12T15:52:21.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-12T15:52:21.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-12T15:52:21.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-12T15:52:21.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-12T15:52:21.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-12T15:52:21.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-12T15:52:21.571 INFO:teuthology.orchestra.run.smithi203.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-12T15:52:21.571 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-12T15:52:21.571 INFO:teuthology.orchestra.run.smithi203.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-12T15:52:21.571 INFO:teuthology.orchestra.run.smithi203.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-12T15:52:21.571 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-12T15:52:21.571 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-12T15:52:21.572 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-12T15:52:21.572 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-12T15:52:21.572 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-12T15:52:21.572 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-12T15:52:21.572 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-12T15:52:21.573 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-12T15:52:21.573 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-12T15:52:21.573 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-12T15:52:21.573 INFO:teuthology.orchestra.run.smithi203.stdout:Enabling module streams: 2023-09-12T15:52:21.573 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql 10 2023-09-12T15:52:21.573 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:52:21.574 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2023-09-12T15:52:21.574 INFO:teuthology.orchestra.run.smithi203.stdout:=========================================================================================================================== 2023-09-12T15:52:21.574 INFO:teuthology.orchestra.run.smithi203.stdout:Install 128 Packages 2023-09-12T15:52:21.574 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 3 Packages 2023-09-12T15:52:21.574 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:52:21.575 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 208 M 2023-09-12T15:52:21.575 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2023-09-12T15:52:21.803 INFO:teuthology.orchestra.run.smithi159.stdout:(1/131): ceph-18.0.0-6088.g2110e007.el8.x86_64. 36 kB/s | 6.4 kB 00:00 2023-09-12T15:52:21.851 INFO:teuthology.orchestra.run.smithi130.stdout:(4/131): ceph-base-18.0.0-6088.g2110e007.el8.x8 6.0 MB/s | 5.4 MB 00:00 2023-09-12T15:52:21.875 INFO:teuthology.orchestra.run.smithi203.stdout:(1/131): ceph-18.0.0-6088.g2110e007.el8.x86_64. 46 kB/s | 6.4 kB 00:00 2023-09-12T15:52:21.902 INFO:teuthology.orchestra.run.smithi130.stdout:(5/131): ceph-mds-18.0.0-6088.g2110e007.el8.x86 6.1 MB/s | 2.3 MB 00:00 2023-09-12T15:52:22.111 INFO:teuthology.orchestra.run.smithi130.stdout:(6/131): ceph-mgr-18.0.0-6088.g2110e007.el8.x86 6.2 MB/s | 1.6 MB 00:00 2023-09-12T15:52:22.220 INFO:teuthology.orchestra.run.smithi159.stdout:(2/131): ceph-fuse-18.0.0-6088.g2110e007.el8.x8 2.1 MB/s | 907 kB 00:00 2023-09-12T15:52:22.329 INFO:teuthology.orchestra.run.smithi159.stdout:(3/131): ceph-immutable-object-cache-18.0.0-608 1.5 MB/s | 167 kB 00:00 2023-09-12T15:52:22.786 INFO:teuthology.orchestra.run.smithi130.stdout:(7/131): ceph-mon-18.0.0-6088.g2110e007.el8.x86 5.2 MB/s | 4.6 MB 00:00 2023-09-12T15:52:22.975 INFO:teuthology.orchestra.run.smithi203.stdout:(2/131): ceph-fuse-18.0.0-6088.g2110e007.el8.x8 825 kB/s | 907 kB 00:01 2023-09-12T15:52:23.250 INFO:teuthology.orchestra.run.smithi203.stdout:(3/131): ceph-immutable-object-cache-18.0.0-608 607 kB/s | 167 kB 00:00 2023-09-12T15:52:23.387 INFO:teuthology.orchestra.run.smithi159.stdout:(4/131): ceph-mds-18.0.0-6088.g2110e007.el8.x86 2.2 MB/s | 2.3 MB 00:01 2023-09-12T15:52:23.719 INFO:teuthology.orchestra.run.smithi130.stdout:(8/131): ceph-common-18.0.0-6088.g2110e007.el8. 8.1 MB/s | 22 MB 00:02 2023-09-12T15:52:23.795 INFO:teuthology.orchestra.run.smithi130.stdout:(9/131): ceph-selinux-18.0.0-6088.g2110e007.el8 324 kB/s | 24 kB 00:00 2023-09-12T15:52:23.863 INFO:teuthology.orchestra.run.smithi159.stdout:(5/131): ceph-mgr-18.0.0-6088.g2110e007.el8.x86 3.4 MB/s | 1.6 MB 00:00 2023-09-12T15:52:24.753 INFO:teuthology.orchestra.run.smithi130.stdout:(10/131): ceph-osd-18.0.0-6088.g2110e007.el8.x8 6.7 MB/s | 18 MB 00:02 2023-09-12T15:52:24.853 INFO:teuthology.orchestra.run.smithi130.stdout:(11/131): cephfs-mirror-18.0.0-6088.g2110e007.e 2.6 MB/s | 261 kB 00:00 2023-09-12T15:52:24.945 INFO:teuthology.orchestra.run.smithi130.stdout:(12/131): libcephfs-devel-18.0.0-6088.g2110e007 338 kB/s | 31 kB 00:00 2023-09-12T15:52:25.146 INFO:teuthology.orchestra.run.smithi130.stdout:(13/131): libcephfs2-18.0.0-6088.g2110e007.el8. 4.0 MB/s | 827 kB 00:00 2023-09-12T15:52:25.254 INFO:teuthology.orchestra.run.smithi130.stdout:(14/131): libcephsqlite-18.0.0-6088.g2110e007.e 1.7 MB/s | 184 kB 00:00 2023-09-12T15:52:25.275 INFO:teuthology.orchestra.run.smithi203.stdout:(4/131): ceph-mds-18.0.0-6088.g2110e007.el8.x86 1.2 MB/s | 2.3 MB 00:02 2023-09-12T15:52:25.384 INFO:teuthology.orchestra.run.smithi203.stdout:(5/131): ceph-base-18.0.0-6088.g2110e007.el8.x8 1.5 MB/s | 5.4 MB 00:03 2023-09-12T15:52:25.421 INFO:teuthology.orchestra.run.smithi130.stdout:(15/131): ceph-radosgw-18.0.0-6088.g2110e007.el 5.2 MB/s | 14 MB 00:02 2023-09-12T15:52:25.446 INFO:teuthology.orchestra.run.smithi130.stdout:(16/131): librados-devel-18.0.0-6088.g2110e007. 724 kB/s | 139 kB 00:00 2023-09-12T15:52:25.647 INFO:teuthology.orchestra.run.smithi130.stdout:(17/131): libradosstriper1-18.0.0-6088.g2110e00 2.3 MB/s | 519 kB 00:00 2023-09-12T15:52:25.713 INFO:teuthology.orchestra.run.smithi159.stdout:(6/131): ceph-mon-18.0.0-6088.g2110e007.el8.x86 2.5 MB/s | 4.6 MB 00:01 2023-09-12T15:52:25.747 INFO:teuthology.orchestra.run.smithi130.stdout:(18/131): python3-ceph-argparse-18.0.0-6088.g21 447 kB/s | 45 kB 00:00 2023-09-12T15:52:25.873 INFO:teuthology.orchestra.run.smithi130.stdout:(19/131): python3-ceph-common-18.0.0-6088.g2110 1.0 MB/s | 125 kB 00:00 2023-09-12T15:52:25.981 INFO:teuthology.orchestra.run.smithi130.stdout:(20/131): python3-cephfs-18.0.0-6088.g2110e007. 1.8 MB/s | 197 kB 00:00 2023-09-12T15:52:26.107 INFO:teuthology.orchestra.run.smithi130.stdout:(21/131): python3-rados-18.0.0-6088.g2110e007.e 2.9 MB/s | 371 kB 00:00 2023-09-12T15:52:26.205 INFO:teuthology.orchestra.run.smithi159.stdout:(7/131): ceph-base-18.0.0-6088.g2110e007.el8.x8 1.2 MB/s | 5.4 MB 00:04 2023-09-12T15:52:26.215 INFO:teuthology.orchestra.run.smithi130.stdout:(22/131): python3-rbd-18.0.0-6088.g2110e007.el8 3.3 MB/s | 364 kB 00:00 2023-09-12T15:52:26.299 INFO:teuthology.orchestra.run.smithi130.stdout:(23/131): python3-rgw-18.0.0-6088.g2110e007.el8 1.2 MB/s | 106 kB 00:00 2023-09-12T15:52:26.351 INFO:teuthology.orchestra.run.smithi203.stdout:(6/131): ceph-mgr-18.0.0-6088.g2110e007.el8.x86 1.5 MB/s | 1.6 MB 00:01 2023-09-12T15:52:26.391 INFO:teuthology.orchestra.run.smithi130.stdout:(24/131): rbd-fuse-18.0.0-6088.g2110e007.el8.x8 1.0 MB/s | 94 kB 00:00 2023-09-12T15:52:26.599 INFO:teuthology.orchestra.run.smithi130.stdout:(25/131): librgw2-18.0.0-6088.g2110e007.el8.x86 5.9 MB/s | 6.8 MB 00:01 2023-09-12T15:52:26.692 INFO:teuthology.orchestra.run.smithi130.stdout:(26/131): rbd-nbd-18.0.0-6088.g2110e007.el8.x86 1.9 MB/s | 179 kB 00:00 2023-09-12T15:52:26.783 INFO:teuthology.orchestra.run.smithi130.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6088.g 252 kB/s | 23 kB 00:00 2023-09-12T15:52:26.892 INFO:teuthology.orchestra.run.smithi130.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6088.g2110e00 1.2 MB/s | 132 kB 00:00 2023-09-12T15:52:27.184 INFO:teuthology.orchestra.run.smithi130.stdout:(29/131): rbd-mirror-18.0.0-6088.g2110e007.el8. 4.5 MB/s | 3.6 MB 00:00 2023-09-12T15:52:27.268 INFO:teuthology.orchestra.run.smithi130.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6088.g2110e 4.7 MB/s | 1.7 MB 00:00 2023-09-12T15:52:27.376 INFO:teuthology.orchestra.run.smithi130.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6088.g21 2.2 MB/s | 242 kB 00:00 2023-09-12T15:52:27.477 INFO:teuthology.orchestra.run.smithi130.stdout:(32/131): ceph-mgr-rook-18.0.0-6088.g2110e007.e 474 kB/s | 47 kB 00:00 2023-09-12T15:52:27.568 INFO:teuthology.orchestra.run.smithi130.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6088.g2 158 kB/s | 15 kB 00:00 2023-09-12T15:52:27.660 INFO:teuthology.orchestra.run.smithi130.stdout:(34/131): ceph-volume-18.0.0-6088.g2110e007.el8 2.8 MB/s | 261 kB 00:00 2023-09-12T15:52:27.744 INFO:teuthology.orchestra.run.smithi130.stdout:(35/131): cephadm-18.0.0-6088.g2110e007.el8.noa 2.1 MB/s | 177 kB 00:00 2023-09-12T15:52:27.819 INFO:teuthology.orchestra.run.smithi130.stdout:(36/131): cephfs-top-18.0.0-6088.g2110e007.el8. 239 kB/s | 18 kB 00:00 2023-09-12T15:52:27.860 INFO:teuthology.orchestra.run.smithi203.stdout:(7/131): ceph-common-18.0.0-6088.g2110e007.el8. 3.7 MB/s | 22 MB 00:06 2023-09-12T15:52:28.109 INFO:teuthology.orchestra.run.smithi203.stdout:(8/131): ceph-mon-18.0.0-6088.g2110e007.el8.x86 1.7 MB/s | 4.6 MB 00:02 2023-09-12T15:52:28.186 INFO:teuthology.orchestra.run.smithi203.stdout:(9/131): ceph-selinux-18.0.0-6088.g2110e007.el8 320 kB/s | 24 kB 00:00 2023-09-12T15:52:28.311 INFO:teuthology.orchestra.run.smithi130.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 623 kB/s | 306 kB 00:00 2023-09-12T15:52:28.487 INFO:teuthology.orchestra.run.smithi130.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 5.7 MB/s | 7.4 MB 00:01 2023-09-12T15:52:28.512 INFO:teuthology.orchestra.run.smithi130.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 293 kB/s | 59 kB 00:00 2023-09-12T15:52:28.704 INFO:teuthology.orchestra.run.smithi130.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.4 MB/s | 476 kB 00:00 2023-09-12T15:52:28.804 INFO:teuthology.orchestra.run.smithi130.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 436 kB/s | 44 kB 00:00 2023-09-12T15:52:28.830 INFO:teuthology.orchestra.run.smithi130.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 219 kB/s | 75 kB 00:00 2023-09-12T15:52:28.913 INFO:teuthology.orchestra.run.smithi130.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-09-12T15:52:28.939 INFO:teuthology.orchestra.run.smithi130.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 303 kB/s | 33 kB 00:00 2023-09-12T15:52:29.014 INFO:teuthology.orchestra.run.smithi130.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 162 kB/s | 16 kB 00:00 2023-09-12T15:52:29.123 INFO:teuthology.orchestra.run.smithi130.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 940 kB/s | 173 kB 00:00 2023-09-12T15:52:29.173 INFO:teuthology.orchestra.run.smithi130.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.4 MB/s | 384 kB 00:00 2023-09-12T15:52:29.223 INFO:teuthology.orchestra.run.smithi130.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 810 kB/s | 81 kB 00:00 2023-09-12T15:52:29.273 INFO:teuthology.orchestra.run.smithi130.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-09-12T15:52:29.315 INFO:teuthology.orchestra.run.smithi130.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2023-09-12T15:52:29.407 INFO:teuthology.orchestra.run.smithi130.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 532 kB/s | 49 kB 00:00 2023-09-12T15:52:29.466 INFO:teuthology.orchestra.run.smithi130.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 6.1 MB/s | 1.2 MB 00:00 2023-09-12T15:52:29.500 INFO:teuthology.orchestra.run.smithi130.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 636 kB/s | 59 kB 00:00 2023-09-12T15:52:29.558 INFO:teuthology.orchestra.run.smithi130.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 654 kB/s | 60 kB 00:00 2023-09-12T15:52:29.650 INFO:teuthology.orchestra.run.smithi130.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-09-12T15:52:29.652 INFO:teuthology.orchestra.run.smithi203.stdout:(10/131): ceph-radosgw-18.0.0-6088.g2110e007.el 7.6 MB/s | 14 MB 00:01 2023-09-12T15:52:29.692 INFO:teuthology.orchestra.run.smithi130.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-09-12T15:52:29.744 INFO:teuthology.orchestra.run.smithi203.stdout:(11/131): cephfs-mirror-18.0.0-6088.g2110e007.e 2.8 MB/s | 261 kB 00:00 2023-09-12T15:52:29.750 INFO:teuthology.orchestra.run.smithi130.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 335 kB/s | 34 kB 00:00 2023-09-12T15:52:29.793 INFO:teuthology.orchestra.run.smithi130.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-09-12T15:52:29.819 INFO:teuthology.orchestra.run.smithi203.stdout:(12/131): libcephfs-devel-18.0.0-6088.g2110e007 415 kB/s | 31 kB 00:00 2023-09-12T15:52:29.851 INFO:teuthology.orchestra.run.smithi130.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 640 kB/s | 64 kB 00:00 2023-09-12T15:52:29.893 INFO:teuthology.orchestra.run.smithi130.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2023-09-12T15:52:29.952 INFO:teuthology.orchestra.run.smithi130.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-09-12T15:52:29.978 INFO:teuthology.orchestra.run.smithi203.stdout:(13/131): libcephfs2-18.0.0-6088.g2110e007.el8. 5.1 MB/s | 827 kB 00:00 2023-09-12T15:52:29.994 INFO:teuthology.orchestra.run.smithi130.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-09-12T15:52:30.053 INFO:teuthology.orchestra.run.smithi130.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-09-12T15:52:30.061 INFO:teuthology.orchestra.run.smithi203.stdout:(14/131): libcephsqlite-18.0.0-6088.g2110e007.e 2.2 MB/s | 184 kB 00:00 2023-09-12T15:52:30.128 INFO:teuthology.orchestra.run.smithi130.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-09-12T15:52:30.154 INFO:teuthology.orchestra.run.smithi203.stdout:(15/131): librados-devel-18.0.0-6088.g2110e007. 1.5 MB/s | 139 kB 00:00 2023-09-12T15:52:30.187 INFO:teuthology.orchestra.run.smithi130.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2023-09-12T15:52:30.237 INFO:teuthology.orchestra.run.smithi203.stdout:(16/131): libradosstriper1-18.0.0-6088.g2110e00 6.1 MB/s | 519 kB 00:00 2023-09-12T15:52:30.671 INFO:teuthology.orchestra.run.smithi130.stdout:(66/131): ceph-test-18.0.0-6088.g2110e007.el8.x 7.1 MB/s | 49 MB 00:06 2023-09-12T15:52:30.696 INFO:teuthology.orchestra.run.smithi130.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 108 kB/s | 61 kB 00:00 2023-09-12T15:52:30.721 INFO:teuthology.orchestra.run.smithi130.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 160 kB/s | 85 kB 00:00 2023-09-12T15:52:30.746 INFO:teuthology.orchestra.run.smithi130.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-12T15:52:30.980 INFO:teuthology.orchestra.run.smithi203.stdout:(17/131): librgw2-18.0.0-6088.g2110e007.el8.x86 9.2 MB/s | 6.8 MB 00:00 2023-09-12T15:52:30.988 INFO:teuthology.orchestra.run.smithi130.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 193 kB/s | 47 kB 00:00 2023-09-12T15:52:31.014 INFO:teuthology.orchestra.run.smithi130.stdout:(71/131): python3-zc-lockfile-2.0-2.el8.noarch. 67 kB/s | 23 kB 00:00 2023-09-12T15:52:31.039 INFO:teuthology.orchestra.run.smithi130.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2023-09-12T15:52:31.046 INFO:teuthology.orchestra.run.smithi203.stdout:(18/131): python3-ceph-argparse-18.0.0-6088.g21 674 kB/s | 45 kB 00:00 2023-09-12T15:52:31.081 INFO:teuthology.orchestra.run.smithi130.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 4.5 MB/s | 1.7 MB 00:00 2023-09-12T15:52:31.106 INFO:teuthology.orchestra.run.smithi130.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 462 kB/s | 43 kB 00:00 2023-09-12T15:52:31.122 INFO:teuthology.orchestra.run.smithi203.stdout:(19/131): python3-ceph-common-18.0.0-6088.g2110 1.6 MB/s | 125 kB 00:00 2023-09-12T15:52:31.131 INFO:teuthology.orchestra.run.smithi130.stdout:(75/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.5 MB/s | 220 kB 00:00 2023-09-12T15:52:31.157 INFO:teuthology.orchestra.run.smithi130.stdout:(76/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 638 kB/s | 48 kB 00:00 2023-09-12T15:52:31.182 INFO:teuthology.orchestra.run.smithi130.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.2 MB/s | 87 kB 00:00 2023-09-12T15:52:31.189 INFO:teuthology.orchestra.run.smithi203.stdout:(20/131): python3-cephfs-18.0.0-6088.g2110e007. 2.9 MB/s | 197 kB 00:00 2023-09-12T15:52:31.208 INFO:teuthology.orchestra.run.smithi130.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 564 kB/s | 43 kB 00:00 2023-09-12T15:52:31.233 INFO:teuthology.orchestra.run.smithi130.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-12T15:52:31.258 INFO:teuthology.orchestra.run.smithi130.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-09-12T15:52:31.264 INFO:teuthology.orchestra.run.smithi203.stdout:(21/131): python3-rados-18.0.0-6088.g2110e007.e 4.8 MB/s | 371 kB 00:00 2023-09-12T15:52:31.284 INFO:teuthology.orchestra.run.smithi130.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 372 kB/s | 19 kB 00:00 2023-09-12T15:52:31.305 INFO:teuthology.orchestra.run.smithi159.stdout:(8/131): ceph-osd-18.0.0-6088.g2110e007.el8.x86 3.2 MB/s | 18 MB 00:05 2023-09-12T15:52:31.309 INFO:teuthology.orchestra.run.smithi130.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-09-12T15:52:31.348 INFO:teuthology.orchestra.run.smithi203.stdout:(22/131): python3-rbd-18.0.0-6088.g2110e007.el8 4.3 MB/s | 364 kB 00:00 2023-09-12T15:52:31.350 INFO:teuthology.orchestra.run.smithi130.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2023-09-12T15:52:31.363 INFO:teuthology.orchestra.run.smithi159.stdout:(9/131): ceph-selinux-18.0.0-6088.g2110e007.el8 419 kB/s | 24 kB 00:00 2023-09-12T15:52:31.367 INFO:teuthology.orchestra.run.smithi130.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-09-12T15:52:31.410 INFO:teuthology.orchestra.run.smithi130.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.3 MB/s | 559 kB 00:00 2023-09-12T15:52:31.415 INFO:teuthology.orchestra.run.smithi203.stdout:(23/131): python3-rgw-18.0.0-6088.g2110e007.el8 1.6 MB/s | 106 kB 00:00 2023-09-12T15:52:31.435 INFO:teuthology.orchestra.run.smithi130.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-09-12T15:52:31.460 INFO:teuthology.orchestra.run.smithi130.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-09-12T15:52:31.482 INFO:teuthology.orchestra.run.smithi203.stdout:(24/131): rbd-fuse-18.0.0-6088.g2110e007.el8.x8 1.4 MB/s | 94 kB 00:00 2023-09-12T15:52:31.485 INFO:teuthology.orchestra.run.smithi130.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-09-12T15:52:31.502 INFO:teuthology.orchestra.run.smithi130.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-09-12T15:52:31.528 INFO:teuthology.orchestra.run.smithi130.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-09-12T15:52:31.553 INFO:teuthology.orchestra.run.smithi130.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-09-12T15:52:31.578 INFO:teuthology.orchestra.run.smithi130.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-12T15:52:31.612 INFO:teuthology.orchestra.run.smithi130.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 658 kB/s | 39 kB 00:00 2023-09-12T15:52:31.654 INFO:teuthology.orchestra.run.smithi130.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 8.3 MB/s | 644 kB 00:00 2023-09-12T15:52:31.794 INFO:teuthology.orchestra.run.smithi130.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-09-12T15:52:31.794 INFO:teuthology.orchestra.run.smithi130.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-09-12T15:52:31.795 INFO:teuthology.orchestra.run.smithi130.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.4 MB/s | 2.3 MB 00:00 2023-09-12T15:52:31.797 INFO:teuthology.orchestra.run.smithi130.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2023-09-12T15:52:31.822 INFO:teuthology.orchestra.run.smithi130.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-09-12T15:52:31.847 INFO:teuthology.orchestra.run.smithi130.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-09-12T15:52:31.906 INFO:teuthology.orchestra.run.smithi130.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2023-09-12T15:52:31.907 INFO:teuthology.orchestra.run.smithi203.stdout:(25/131): rbd-mirror-18.0.0-6088.g2110e007.el8. 8.4 MB/s | 3.6 MB 00:00 2023-09-12T15:52:31.931 INFO:teuthology.orchestra.run.smithi130.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 464 kB/s | 39 kB 00:00 2023-09-12T15:52:31.973 INFO:teuthology.orchestra.run.smithi130.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-09-12T15:52:31.974 INFO:teuthology.orchestra.run.smithi203.stdout:(26/131): rbd-nbd-18.0.0-6088.g2110e007.el8.x86 2.6 MB/s | 179 kB 00:00 2023-09-12T15:52:32.015 INFO:teuthology.orchestra.run.smithi130.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-09-12T15:52:32.032 INFO:teuthology.orchestra.run.smithi203.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6088.g 396 kB/s | 23 kB 00:00 2023-09-12T15:52:32.040 INFO:teuthology.orchestra.run.smithi130.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-09-12T15:52:32.066 INFO:teuthology.orchestra.run.smithi130.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-09-12T15:52:32.096 INFO:teuthology.orchestra.run.smithi159.stdout:(10/131): ceph-radosgw-18.0.0-6088.g2110e007.el 2.3 MB/s | 14 MB 00:05 2023-09-12T15:52:32.108 INFO:teuthology.orchestra.run.smithi130.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-09-12T15:52:32.133 INFO:teuthology.orchestra.run.smithi203.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6088.g2110e00 1.3 MB/s | 132 kB 00:00 2023-09-12T15:52:32.149 INFO:teuthology.orchestra.run.smithi130.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 640 kB/s | 27 kB 00:00 2023-09-12T15:52:32.200 INFO:teuthology.orchestra.run.smithi130.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-09-12T15:52:32.242 INFO:teuthology.orchestra.run.smithi130.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-09-12T15:52:32.255 INFO:teuthology.orchestra.run.smithi159.stdout:(11/131): cephfs-mirror-18.0.0-6088.g2110e007.e 1.6 MB/s | 261 kB 00:00 2023-09-12T15:52:32.309 INFO:teuthology.orchestra.run.smithi130.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-09-12T15:52:32.316 INFO:teuthology.orchestra.run.smithi203.stdout:(29/131): ceph-osd-18.0.0-6088.g2110e007.el8.x8 3.0 MB/s | 18 MB 00:05 2023-09-12T15:52:32.359 INFO:teuthology.orchestra.run.smithi130.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-09-12T15:52:32.375 INFO:teuthology.orchestra.run.smithi159.stdout:(12/131): libcephfs-devel-18.0.0-6088.g2110e007 260 kB/s | 31 kB 00:00 2023-09-12T15:52:32.409 INFO:teuthology.orchestra.run.smithi130.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 4.4 MB/s | 225 kB 00:00 2023-09-12T15:52:32.450 INFO:teuthology.orchestra.run.smithi203.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6088.g2110e 5.5 MB/s | 1.7 MB 00:00 2023-09-12T15:52:32.476 INFO:teuthology.orchestra.run.smithi130.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 16 MB/s | 3.7 MB 00:00 2023-09-12T15:52:32.502 INFO:teuthology.orchestra.run.smithi130.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-09-12T15:52:32.527 INFO:teuthology.orchestra.run.smithi130.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-12T15:52:32.552 INFO:teuthology.orchestra.run.smithi130.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-09-12T15:52:32.559 INFO:teuthology.orchestra.run.smithi203.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6088.g21 2.2 MB/s | 242 kB 00:00 2023-09-12T15:52:32.577 INFO:teuthology.orchestra.run.smithi130.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-09-12T15:52:32.620 INFO:teuthology.orchestra.run.smithi130.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-09-12T15:52:32.651 INFO:teuthology.orchestra.run.smithi203.stdout:(32/131): ceph-mgr-rook-18.0.0-6088.g2110e007.e 517 kB/s | 47 kB 00:00 2023-09-12T15:52:32.739 INFO:teuthology.orchestra.run.smithi159.stdout:(13/131): libcephfs2-18.0.0-6088.g2110e007.el8. 2.2 MB/s | 827 kB 00:00 2023-09-12T15:52:32.743 INFO:teuthology.orchestra.run.smithi203.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6088.g2 158 kB/s | 15 kB 00:00 2023-09-12T15:52:32.762 INFO:teuthology.orchestra.run.smithi130.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-09-12T15:52:32.796 INFO:teuthology.orchestra.run.smithi130.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-09-12T15:52:32.814 INFO:teuthology.orchestra.run.smithi159.stdout:(14/131): libcephsqlite-18.0.0-6088.g2110e007.e 2.4 MB/s | 184 kB 00:00 2023-09-12T15:52:32.837 INFO:teuthology.orchestra.run.smithi130.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-09-12T15:52:32.843 INFO:teuthology.orchestra.run.smithi203.stdout:(34/131): ceph-volume-18.0.0-6088.g2110e007.el8 2.6 MB/s | 261 kB 00:00 2023-09-12T15:52:32.888 INFO:teuthology.orchestra.run.smithi130.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-09-12T15:52:32.906 INFO:teuthology.orchestra.run.smithi159.stdout:(15/131): librados-devel-18.0.0-6088.g2110e007. 1.5 MB/s | 139 kB 00:00 2023-09-12T15:52:32.952 INFO:teuthology.orchestra.run.smithi203.stdout:(35/131): cephadm-18.0.0-6088.g2110e007.el8.noa 1.6 MB/s | 177 kB 00:00 2023-09-12T15:52:32.988 INFO:teuthology.orchestra.run.smithi130.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-09-12T15:52:33.036 INFO:teuthology.orchestra.run.smithi203.stdout:(36/131): cephfs-top-18.0.0-6088.g2110e007.el8. 216 kB/s | 18 kB 00:00 2023-09-12T15:52:33.055 INFO:teuthology.orchestra.run.smithi130.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-09-12T15:52:33.130 INFO:teuthology.orchestra.run.smithi130.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-09-12T15:52:33.189 INFO:teuthology.orchestra.run.smithi130.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-09-12T15:52:33.272 INFO:teuthology.orchestra.run.smithi130.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-09-12T15:52:33.331 INFO:teuthology.orchestra.run.smithi130.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-12T15:52:33.374 INFO:teuthology.orchestra.run.smithi159.stdout:(16/131): ceph-common-18.0.0-6088.g2110e007.el8 1.9 MB/s | 22 MB 00:11 2023-09-12T15:52:33.407 INFO:teuthology.orchestra.run.smithi159.stdout:(17/131): libradosstriper1-18.0.0-6088.g2110e00 1.0 MB/s | 519 kB 00:00 2023-09-12T15:52:33.491 INFO:teuthology.orchestra.run.smithi159.stdout:(18/131): python3-ceph-argparse-18.0.0-6088.g21 537 kB/s | 45 kB 00:00 2023-09-12T15:52:33.528 INFO:teuthology.orchestra.run.smithi203.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 6.1 MB/s | 7.4 MB 00:01 2023-09-12T15:52:33.641 INFO:teuthology.orchestra.run.smithi159.stdout:(19/131): python3-ceph-common-18.0.0-6088.g2110 838 kB/s | 125 kB 00:00 2023-09-12T15:52:33.816 INFO:teuthology.orchestra.run.smithi159.stdout:(20/131): python3-cephfs-18.0.0-6088.g2110e007. 1.1 MB/s | 197 kB 00:00 2023-09-12T15:52:34.008 INFO:teuthology.orchestra.run.smithi159.stdout:(21/131): python3-rados-18.0.0-6088.g2110e007.e 1.9 MB/s | 371 kB 00:00 2023-09-12T15:52:34.081 INFO:teuthology.orchestra.run.smithi130.stdout:(130/131): librados2-18.0.0-6088.g2110e007.el8. 3.9 MB/s | 3.7 MB 00:00 2023-09-12T15:52:34.131 INFO:teuthology.orchestra.run.smithi130.stdout:(131/131): librbd1-18.0.0-6088.g2110e007.el8.x8 3.9 MB/s | 3.7 MB 00:00 2023-09-12T15:52:34.133 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2023-09-12T15:52:34.134 INFO:teuthology.orchestra.run.smithi130.stdout:Total 16 MB/s | 208 MB 00:13 2023-09-12T15:52:34.200 INFO:teuthology.orchestra.run.smithi159.stdout:(22/131): python3-rbd-18.0.0-6088.g2110e007.el8 1.9 MB/s | 364 kB 00:00 2023-09-12T15:52:34.309 INFO:teuthology.orchestra.run.smithi159.stdout:(23/131): python3-rgw-18.0.0-6088.g2110e007.el8 985 kB/s | 106 kB 00:00 2023-09-12T15:52:34.426 INFO:teuthology.orchestra.run.smithi159.stdout:(24/131): rbd-fuse-18.0.0-6088.g2110e007.el8.x8 803 kB/s | 94 kB 00:00 2023-09-12T15:52:34.645 INFO:teuthology.orchestra.run.smithi203.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 53 kB/s | 59 kB 00:01 2023-09-12T15:52:34.670 INFO:teuthology.orchestra.run.smithi203.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 187 kB/s | 306 kB 00:01 2023-09-12T15:52:34.958 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2023-09-12T15:52:35.029 INFO:teuthology.orchestra.run.smithi203.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 196 kB/s | 75 kB 00:00 2023-09-12T15:52:35.077 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2023-09-12T15:52:35.077 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2023-09-12T15:52:35.187 INFO:teuthology.orchestra.run.smithi203.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 921 kB/s | 476 kB 00:00 2023-09-12T15:52:35.338 INFO:teuthology.orchestra.run.smithi203.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 141 kB/s | 44 kB 00:00 2023-09-12T15:52:35.455 INFO:teuthology.orchestra.run.smithi203.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 693 kB/s | 185 kB 00:00 2023-09-12T15:52:35.560 INFO:teuthology.orchestra.run.smithi159.stdout:(25/131): rbd-mirror-18.0.0-6088.g2110e007.el8. 3.2 MB/s | 3.6 MB 00:01 2023-09-12T15:52:35.630 INFO:teuthology.orchestra.run.smithi203.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 113 kB/s | 33 kB 00:00 2023-09-12T15:52:35.676 INFO:teuthology.orchestra.run.smithi159.stdout:(26/131): rbd-nbd-18.0.0-6088.g2110e007.el8.x86 1.5 MB/s | 179 kB 00:00 2023-09-12T15:52:35.688 INFO:teuthology.orchestra.run.smithi203.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 70 kB/s | 16 kB 00:00 2023-09-12T15:52:35.785 INFO:teuthology.orchestra.run.smithi159.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6088.g 213 kB/s | 23 kB 00:00 2023-09-12T15:52:35.919 INFO:teuthology.orchestra.run.smithi159.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6088.g2110e00 990 kB/s | 132 kB 00:00 2023-09-12T15:52:36.014 INFO:teuthology.orchestra.run.smithi203.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-09-12T15:52:36.131 INFO:teuthology.orchestra.run.smithi203.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 345 kB/s | 173 kB 00:00 2023-09-12T15:52:36.189 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2023-09-12T15:52:36.193 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2023-09-12T15:52:36.273 INFO:teuthology.orchestra.run.smithi203.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 314 kB/s | 81 kB 00:00 2023-09-12T15:52:36.311 INFO:teuthology.orchestra.run.smithi159.stdout:(29/131): librgw2-18.0.0-6088.g2110e007.el8.x86 2.3 MB/s | 6.8 MB 00:02 2023-09-12T15:52:36.373 INFO:teuthology.orchestra.run.smithi203.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 46 kB/s | 11 kB 00:00 2023-09-12T15:52:36.478 INFO:teuthology.orchestra.run.smithi159.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6088.g2110e 3.1 MB/s | 1.7 MB 00:00 2023-09-12T15:52:36.507 INFO:teuthology.orchestra.run.smithi203.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 75 kB/s | 18 kB 00:00 2023-09-12T15:52:36.595 INFO:teuthology.orchestra.run.smithi159.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6088.g21 2.0 MB/s | 242 kB 00:00 2023-09-12T15:52:36.662 INFO:teuthology.orchestra.run.smithi159.stdout:(32/131): ceph-mgr-rook-18.0.0-6088.g2110e007.e 712 kB/s | 47 kB 00:00 2023-09-12T15:52:36.720 INFO:teuthology.orchestra.run.smithi159.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6088.g2 249 kB/s | 15 kB 00:00 2023-09-12T15:52:36.749 INFO:teuthology.orchestra.run.smithi203.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 201 kB/s | 49 kB 00:00 2023-09-12T15:52:36.795 INFO:teuthology.orchestra.run.smithi159.stdout:(34/131): ceph-volume-18.0.0-6088.g2110e007.el8 3.4 MB/s | 261 kB 00:00 2023-09-12T15:52:36.873 INFO:teuthology.orchestra.run.smithi159.stdout:(35/131): cephadm-18.0.0-6088.g2110e007.el8.noa 2.3 MB/s | 177 kB 00:00 2023-09-12T15:52:36.938 INFO:teuthology.orchestra.run.smithi159.stdout:(36/131): cephfs-top-18.0.0-6088.g2110e007.el8. 270 kB/s | 18 kB 00:00 2023-09-12T15:52:36.991 INFO:teuthology.orchestra.run.smithi203.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 243 kB/s | 59 kB 00:00 2023-09-12T15:52:37.191 INFO:teuthology.orchestra.run.smithi203.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 1.4 MB/s | 1.2 MB 00:00 2023-09-12T15:52:37.250 INFO:teuthology.orchestra.run.smithi203.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 233 kB/s | 60 kB 00:00 2023-09-12T15:52:37.290 INFO:teuthology.orchestra.run.smithi159.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 876 kB/s | 306 kB 00:00 2023-09-12T15:52:37.355 INFO:teuthology.orchestra.run.smithi159.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 886 kB/s | 59 kB 00:00 2023-09-12T15:52:37.422 INFO:teuthology.orchestra.run.smithi159.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-09-12T15:52:37.623 INFO:teuthology.orchestra.run.smithi159.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.3 MB/s | 476 kB 00:00 2023-09-12T15:52:37.656 INFO:teuthology.orchestra.run.smithi159.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2023-09-12T15:52:37.715 INFO:teuthology.orchestra.run.smithi159.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-09-12T15:52:37.748 INFO:teuthology.orchestra.run.smithi159.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 993 kB/s | 33 kB 00:00 2023-09-12T15:52:37.782 INFO:teuthology.orchestra.run.smithi159.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 488 kB/s | 16 kB 00:00 2023-09-12T15:52:37.866 INFO:teuthology.orchestra.run.smithi159.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-09-12T15:52:37.892 INFO:teuthology.orchestra.run.smithi203.stdout:(55/131): ceph-test-18.0.0-6088.g2110e007.el8.x 5.0 MB/s | 49 MB 00:09 2023-09-12T15:52:37.917 INFO:teuthology.orchestra.run.smithi203.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 23 kB/s | 16 kB 00:00 2023-09-12T15:52:37.942 INFO:teuthology.orchestra.run.smithi203.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 376 kB/s | 283 kB 00:00 2023-09-12T15:52:38.008 INFO:teuthology.orchestra.run.smithi159.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.7 MB/s | 384 kB 00:00 2023-09-12T15:52:38.134 INFO:teuthology.orchestra.run.smithi203.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 139 kB/s | 34 kB 00:00 2023-09-12T15:52:38.168 INFO:teuthology.orchestra.run.smithi203.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 285 kB/s | 64 kB 00:00 2023-09-12T15:52:38.283 INFO:teuthology.orchestra.run.smithi159.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 295 kB/s | 81 kB 00:00 2023-09-12T15:52:38.368 INFO:teuthology.orchestra.run.smithi203.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 82 kB/s | 19 kB 00:00 2023-09-12T15:52:38.416 INFO:teuthology.orchestra.run.smithi159.stdout:(48/131): ceph-mgr-diskprediction-local-18.0.0- 3.5 MB/s | 7.4 MB 00:02 2023-09-12T15:52:38.471 INFO:teuthology.orchestra.run.smithi159.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 59 kB/s | 11 kB 00:00 2023-09-12T15:52:38.494 INFO:teuthology.orchestra.run.smithi203.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 73 kB/s | 24 kB 00:00 2023-09-12T15:52:38.575 INFO:teuthology.orchestra.run.smithi159.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 111 kB/s | 18 kB 00:00 2023-09-12T15:52:38.634 INFO:teuthology.orchestra.run.smithi159.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 837 kB/s | 49 kB 00:00 2023-09-12T15:52:38.676 INFO:teuthology.orchestra.run.smithi159.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-09-12T15:52:38.713 INFO:teuthology.orchestra.run.smithi203.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 85 kB/s | 29 kB 00:00 2023-09-12T15:52:38.718 INFO:teuthology.orchestra.run.smithi159.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-09-12T15:52:38.744 INFO:teuthology.orchestra.run.smithi203.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 110 kB/s | 27 kB 00:00 2023-09-12T15:52:38.835 INFO:teuthology.orchestra.run.smithi159.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-09-12T15:52:38.877 INFO:teuthology.orchestra.run.smithi159.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 375 kB/s | 16 kB 00:00 2023-09-12T15:52:38.911 INFO:teuthology.orchestra.run.smithi159.stdout:(56/131): python3-kubernetes-11.0.0-6.el8.noarc 2.7 MB/s | 1.2 MB 00:00 2023-09-12T15:52:38.936 INFO:teuthology.orchestra.run.smithi159.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 575 kB/s | 34 kB 00:00 2023-09-12T15:52:39.011 INFO:teuthology.orchestra.run.smithi159.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 855 kB/s | 64 kB 00:00 2023-09-12T15:52:39.028 INFO:teuthology.orchestra.run.smithi203.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 813 kB/s | 255 kB 00:00 2023-09-12T15:52:39.053 INFO:teuthology.orchestra.run.smithi203.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 814 kB/s | 251 kB 00:00 2023-09-12T15:52:39.095 INFO:teuthology.orchestra.run.smithi159.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.0 MB/s | 196 kB 00:00 2023-09-12T15:52:39.120 INFO:teuthology.orchestra.run.smithi159.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-09-12T15:52:39.187 INFO:teuthology.orchestra.run.smithi159.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 260 kB/s | 24 kB 00:00 2023-09-12T15:52:39.213 INFO:teuthology.orchestra.run.smithi159.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-09-12T15:52:39.312 INFO:teuthology.orchestra.run.smithi203.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 216 kB/s | 61 kB 00:00 2023-09-12T15:52:39.329 INFO:teuthology.orchestra.run.smithi203.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 310 kB/s | 85 kB 00:00 2023-09-12T15:52:39.446 INFO:teuthology.orchestra.run.smithi203.stdout:(68/131): python3-routes-2.4.1-12.el8.noarch.rp 128 kB/s | 196 kB 00:01 2023-09-12T15:52:39.471 INFO:teuthology.orchestra.run.smithi203.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-12T15:52:39.541 INFO:teuthology.orchestra.run.smithi203.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 702 kB/s | 47 kB 00:00 2023-09-12T15:52:39.564 INFO:teuthology.orchestra.run.smithi203.stdout:(71/131): python3-zc-lockfile-2.0-2.el8.noarch. 91 kB/s | 23 kB 00:00 2023-09-12T15:52:39.589 INFO:teuthology.orchestra.run.smithi203.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2023-09-12T15:52:39.622 INFO:teuthology.orchestra.run.smithi203.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 727 kB/s | 43 kB 00:00 2023-09-12T15:52:39.638 INFO:teuthology.orchestra.run.smithi159.stdout:(63/131): ceph-test-18.0.0-6088.g2110e007.el8.x 5.9 MB/s | 49 MB 00:08 2023-09-12T15:52:39.648 INFO:teuthology.orchestra.run.smithi203.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.1 MB/s | 220 kB 00:00 2023-09-12T15:52:39.663 INFO:teuthology.orchestra.run.smithi159.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 58 kB/s | 27 kB 00:00 2023-09-12T15:52:39.665 INFO:teuthology.orchestra.run.smithi203.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-09-12T15:52:39.682 INFO:teuthology.orchestra.run.smithi203.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.5 MB/s | 87 kB 00:00 2023-09-12T15:52:39.688 INFO:teuthology.orchestra.run.smithi159.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 537 kB/s | 255 kB 00:00 2023-09-12T15:52:39.699 INFO:teuthology.orchestra.run.smithi203.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.2 MB/s | 43 kB 00:00 2023-09-12T15:52:39.716 INFO:teuthology.orchestra.run.smithi203.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 3.2 MB/s | 109 kB 00:00 2023-09-12T15:52:39.750 INFO:teuthology.orchestra.run.smithi203.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-12T15:52:39.791 INFO:teuthology.orchestra.run.smithi203.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-09-12T15:52:39.817 INFO:teuthology.orchestra.run.smithi203.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 280 kB/s | 19 kB 00:00 2023-09-12T15:52:39.822 INFO:teuthology.orchestra.run.smithi159.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 386 kB/s | 61 kB 00:00 2023-09-12T15:52:39.847 INFO:teuthology.orchestra.run.smithi159.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 538 kB/s | 85 kB 00:00 2023-09-12T15:52:39.859 INFO:teuthology.orchestra.run.smithi203.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 3.3 MB/s | 1.7 MB 00:00 2023-09-12T15:52:39.873 INFO:teuthology.orchestra.run.smithi159.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 459 kB/s | 23 kB 00:00 2023-09-12T15:52:39.893 INFO:teuthology.orchestra.run.smithi203.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2023-09-12T15:52:39.898 INFO:teuthology.orchestra.run.smithi159.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-12T15:52:39.960 INFO:teuthology.orchestra.run.smithi203.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 3.8 MB/s | 559 kB 00:00 2023-09-12T15:52:39.973 INFO:teuthology.orchestra.run.smithi159.stdout:(70/131): python3-webob-1.8.5-1.el8.1.noarch.rp 751 kB/s | 251 kB 00:00 2023-09-12T15:52:39.985 INFO:teuthology.orchestra.run.smithi203.stdout:(85/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 142 kB/s | 18 kB 00:00 2023-09-12T15:52:40.010 INFO:teuthology.orchestra.run.smithi203.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2023-09-12T15:52:40.032 INFO:teuthology.orchestra.run.smithi159.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.9 MB/s | 110 kB 00:00 2023-09-12T15:52:40.052 INFO:teuthology.orchestra.run.smithi203.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.8 MB/s | 605 kB 00:00 2023-09-12T15:52:40.074 INFO:teuthology.orchestra.run.smithi159.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-12T15:52:40.077 INFO:teuthology.orchestra.run.smithi203.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.1 MB/s | 72 kB 00:00 2023-09-12T15:52:40.103 INFO:teuthology.orchestra.run.smithi203.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-12T15:52:40.115 INFO:teuthology.orchestra.run.smithi159.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-09-12T15:52:40.128 INFO:teuthology.orchestra.run.smithi203.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-09-12T15:52:40.145 INFO:teuthology.orchestra.run.smithi203.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-09-12T15:52:40.166 INFO:teuthology.orchestra.run.smithi159.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-09-12T15:52:40.170 INFO:teuthology.orchestra.run.smithi203.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-09-12T15:52:40.196 INFO:teuthology.orchestra.run.smithi203.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 769 kB/s | 39 kB 00:00 2023-09-12T15:52:40.199 INFO:teuthology.orchestra.run.smithi159.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-09-12T15:52:40.238 INFO:teuthology.orchestra.run.smithi203.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-09-12T15:52:40.250 INFO:teuthology.orchestra.run.smithi159.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-09-12T15:52:40.263 INFO:teuthology.orchestra.run.smithi203.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-09-12T15:52:40.297 INFO:teuthology.orchestra.run.smithi203.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-09-12T15:52:40.300 INFO:teuthology.orchestra.run.smithi159.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-09-12T15:52:40.364 INFO:teuthology.orchestra.run.smithi203.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.1 MB/s | 2.3 MB 00:00 2023-09-12T15:52:40.389 INFO:teuthology.orchestra.run.smithi203.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2023-09-12T15:52:40.414 INFO:teuthology.orchestra.run.smithi203.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-09-12T15:52:40.440 INFO:teuthology.orchestra.run.smithi203.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-09-12T15:52:40.459 INFO:teuthology.orchestra.run.smithi159.stdout:(78/131): python3-scipy-1.0.0-21.module+el8.5.0 24 MB/s | 14 MB 00:00 2023-09-12T15:52:40.473 INFO:teuthology.orchestra.run.smithi203.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-09-12T15:52:40.484 INFO:teuthology.orchestra.run.smithi159.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 854 kB/s | 157 kB 00:00 2023-09-12T15:52:40.490 INFO:teuthology.orchestra.run.smithi203.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 773 kB/s | 39 kB 00:00 2023-09-12T15:52:40.509 INFO:teuthology.orchestra.run.smithi159.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-09-12T15:52:40.540 INFO:teuthology.orchestra.run.smithi203.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-09-12T15:52:40.574 INFO:teuthology.orchestra.run.smithi203.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-09-12T15:52:40.576 INFO:teuthology.orchestra.run.smithi159.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-09-12T15:52:40.600 INFO:teuthology.orchestra.run.smithi203.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-12T15:52:40.602 INFO:teuthology.orchestra.run.smithi159.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 1.0 MB/s | 27 kB 00:00 2023-09-12T15:52:40.625 INFO:teuthology.orchestra.run.smithi203.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-09-12T15:52:40.644 INFO:teuthology.orchestra.run.smithi159.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-12T15:52:40.667 INFO:teuthology.orchestra.run.smithi203.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-09-12T15:52:40.700 INFO:teuthology.orchestra.run.smithi203.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 798 kB/s | 27 kB 00:00 2023-09-12T15:52:40.727 INFO:teuthology.orchestra.run.smithi159.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-09-12T15:52:40.759 INFO:teuthology.orchestra.run.smithi203.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-09-12T15:52:40.769 INFO:teuthology.orchestra.run.smithi159.stdout:(85/131): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-09-12T15:52:40.793 INFO:teuthology.orchestra.run.smithi203.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-09-12T15:52:40.795 INFO:teuthology.orchestra.run.smithi159.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 292 kB/s | 44 kB 00:00 2023-09-12T15:52:40.845 INFO:teuthology.orchestra.run.smithi159.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 4.9 MB/s | 225 kB 00:00 2023-09-12T15:52:40.877 INFO:teuthology.orchestra.run.smithi203.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-09-12T15:52:40.887 INFO:teuthology.orchestra.run.smithi159.stdout:(88/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2023-09-12T15:52:40.912 INFO:teuthology.orchestra.run.smithi159.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 958 kB/s | 64 kB 00:00 2023-09-12T15:52:40.926 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2023-09-12T15:52:40.927 INFO:teuthology.orchestra.run.smithi203.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-09-12T15:52:40.937 INFO:teuthology.orchestra.run.smithi159.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.4 MB/s | 126 kB 00:00 2023-09-12T15:52:40.963 INFO:teuthology.orchestra.run.smithi159.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-09-12T15:52:40.969 INFO:teuthology.orchestra.run.smithi203.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-09-12T15:52:40.988 INFO:teuthology.orchestra.run.smithi159.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-09-12T15:52:41.022 INFO:teuthology.orchestra.run.smithi159.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-12T15:52:41.036 INFO:teuthology.orchestra.run.smithi203.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-09-12T15:52:41.061 INFO:teuthology.orchestra.run.smithi203.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 697 kB/s | 64 kB 00:00 2023-09-12T15:52:41.080 INFO:teuthology.orchestra.run.smithi159.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2023-09-12T15:52:41.086 INFO:teuthology.orchestra.run.smithi203.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-12T15:52:41.105 INFO:teuthology.orchestra.run.smithi159.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-09-12T15:52:41.145 INFO:teuthology.orchestra.run.smithi203.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 239 kB/s | 20 kB 00:00 2023-09-12T15:52:41.170 INFO:teuthology.orchestra.run.smithi203.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.2 MB/s | 103 kB 00:00 2023-09-12T15:52:41.172 INFO:teuthology.orchestra.run.smithi159.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2023-09-12T15:52:41.204 INFO:teuthology.orchestra.run.smithi203.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-12T15:52:41.214 INFO:teuthology.orchestra.run.smithi159.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-12T15:52:41.298 INFO:teuthology.orchestra.run.smithi159.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-09-12T15:52:41.346 INFO:teuthology.orchestra.run.smithi203.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-09-12T15:52:41.365 INFO:teuthology.orchestra.run.smithi159.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-09-12T15:52:41.371 INFO:teuthology.orchestra.run.smithi203.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-09-12T15:52:41.398 INFO:teuthology.orchestra.run.smithi159.stdout:(100/131): postgresql-contrib-10.21-2.module+el 7.9 MB/s | 808 kB 00:00 2023-09-12T15:52:41.413 INFO:teuthology.orchestra.run.smithi203.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-09-12T15:52:41.440 INFO:teuthology.orchestra.run.smithi159.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-09-12T15:52:41.464 INFO:teuthology.orchestra.run.smithi203.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-09-12T15:52:41.474 INFO:teuthology.orchestra.run.smithi159.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-09-12T15:52:41.478 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-12T15:52:41.499 INFO:teuthology.orchestra.run.smithi159.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.3 MB/s | 200 kB 00:00 2023-09-12T15:52:41.525 INFO:teuthology.orchestra.run.smithi159.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-09-12T15:52:41.550 INFO:teuthology.orchestra.run.smithi159.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-12T15:52:41.575 INFO:teuthology.orchestra.run.smithi159.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 956 kB/s | 48 kB 00:00 2023-09-12T15:52:41.597 INFO:teuthology.orchestra.run.smithi203.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 11 MB/s | 4.9 MB 00:00 2023-09-12T15:52:41.600 INFO:teuthology.orchestra.run.smithi159.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-12T15:52:41.626 INFO:teuthology.orchestra.run.smithi159.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 848 kB/s | 43 kB 00:00 2023-09-12T15:52:41.651 INFO:teuthology.orchestra.run.smithi159.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-12T15:52:41.673 INFO:teuthology.orchestra.run.smithi203.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-09-12T15:52:41.676 INFO:teuthology.orchestra.run.smithi159.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-12T15:52:41.687 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-12T15:52:41.710 INFO:teuthology.orchestra.run.smithi159.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-09-12T15:52:41.714 INFO:teuthology.orchestra.run.smithi203.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-09-12T15:52:41.735 INFO:teuthology.orchestra.run.smithi159.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 318 kB/s | 19 kB 00:00 2023-09-12T15:52:41.773 INFO:teuthology.orchestra.run.smithi203.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-09-12T15:52:41.778 INFO:teuthology.orchestra.run.smithi159.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.2 MB/s | 155 kB 00:00 2023-09-12T15:52:41.845 INFO:teuthology.orchestra.run.smithi159.stdout:(114/131): postgresql-server-10.21-2.module+el8 8.0 MB/s | 5.1 MB 00:00 2023-09-12T15:52:41.846 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-12T15:52:41.857 INFO:teuthology.orchestra.run.smithi203.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-09-12T15:52:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 196 kB/s | 18 kB 00:00 2023-09-12T15:52:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.3 MB/s | 559 kB 00:00 2023-09-12T15:52:41.937 INFO:teuthology.orchestra.run.smithi203.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 9.8 MB/s | 798 kB 00:00 2023-09-12T15:52:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-09-12T15:52:41.959 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-12T15:52:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-09-12T15:52:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-09-12T15:52:42.013 INFO:teuthology.orchestra.run.smithi159.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-12T15:52:42.038 INFO:teuthology.orchestra.run.smithi159.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-09-12T15:52:42.063 INFO:teuthology.orchestra.run.smithi159.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-09-12T15:52:42.089 INFO:teuthology.orchestra.run.smithi159.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-09-12T15:52:42.114 INFO:teuthology.orchestra.run.smithi159.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 765 kB/s | 39 kB 00:00 2023-09-12T15:52:42.156 INFO:teuthology.orchestra.run.smithi159.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-09-12T15:52:42.181 INFO:teuthology.orchestra.run.smithi159.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-09-12T15:52:42.188 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-12T15:52:42.209 INFO:teuthology.orchestra.run.smithi159.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-12T15:52:42.257 INFO:teuthology.orchestra.run.smithi159.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.6 MB/s | 2.3 MB 00:00 2023-09-12T15:52:42.307 INFO:teuthology.orchestra.run.smithi159.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-09-12T15:52:42.591 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-12T15:52:42.780 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 5/134 2023-09-12T15:52:42.924 INFO:teuthology.orchestra.run.smithi203.stdout:(130/131): librbd1-18.0.0-6088.g2110e007.el8.x8 3.2 MB/s | 3.7 MB 00:01 2023-09-12T15:52:43.212 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 5/134 2023-09-12T15:52:43.266 INFO:teuthology.orchestra.run.smithi159.stdout:(130/131): librados2-18.0.0-6088.g2110e007.el8. 3.4 MB/s | 3.7 MB 00:01 2023-09-12T15:52:43.371 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-12T15:52:43.486 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-12T15:52:43.541 INFO:teuthology.orchestra.run.smithi159.stdout:(131/131): librbd1-18.0.0-6088.g2110e007.el8.x8 2.8 MB/s | 3.7 MB 00:01 2023-09-12T15:52:43.545 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-09-12T15:52:43.545 INFO:teuthology.orchestra.run.smithi159.stdout:Total 9.4 MB/s | 208 MB 00:22 2023-09-12T15:52:43.649 INFO:teuthology.orchestra.run.smithi203.stdout:(131/131): librados2-18.0.0-6088.g2110e007.el8. 1.9 MB/s | 3.7 MB 00:01 2023-09-12T15:52:43.654 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2023-09-12T15:52:43.654 INFO:teuthology.orchestra.run.smithi203.stdout:Total 9.4 MB/s | 208 MB 00:22 2023-09-12T15:52:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-12T15:52:43.777 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-12T15:52:43.926 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-12T15:52:44.116 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-12T15:52:44.332 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-12T15:52:44.356 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-09-12T15:52:44.477 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-09-12T15:52:44.478 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-09-12T15:52:44.506 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2023-09-12T15:52:44.548 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-12T15:52:44.627 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2023-09-12T15:52:44.627 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2023-09-12T15:52:44.939 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-12T15:52:45.153 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-12T15:52:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-12T15:52:45.421 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-12T15:52:45.571 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-12T15:52:45.615 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-09-12T15:52:45.619 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-09-12T15:52:45.717 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-12T15:52:45.745 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2023-09-12T15:52:45.750 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2023-09-12T15:52:45.941 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-12T15:52:46.106 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-12T15:52:46.276 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-12T15:52:46.395 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-12T15:52:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-12T15:52:46.929 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-12T15:52:47.545 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-12T15:52:47.720 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-12T15:52:48.316 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-12T15:52:48.486 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-12T15:52:49.181 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-12T15:52:49.419 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-12T15:52:49.601 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-12T15:52:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-12T15:52:50.010 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 26/134 2023-09-12T15:52:50.156 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 27/134 2023-09-12T15:52:50.376 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 28/134 2023-09-12T15:52:50.579 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-12T15:52:50.675 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-09-12T15:52:50.819 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-12T15:52:50.969 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-12T15:52:51.019 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2023-09-12T15:52:51.186 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-12T15:52:51.192 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-12T15:52:51.372 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-12T15:52:51.416 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-12T15:52:51.462 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-12T15:52:51.569 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-12T15:52:51.675 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-12T15:52:51.705 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-12T15:52:51.902 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-12T15:52:52.090 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-12T15:52:52.280 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-12T15:52:52.289 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-12T15:52:52.341 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-12T15:52:52.510 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 5/134 2023-09-12T15:52:52.689 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-12T15:52:52.863 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-12T15:52:52.924 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 5/134 2023-09-12T15:52:52.978 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-12T15:52:53.102 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-12T15:52:53.204 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-12T15:52:53.246 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-12T15:52:53.313 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-12T15:52:53.400 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-12T15:52:53.452 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-12T15:52:53.482 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 9/134 2023-09-12T15:52:53.541 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-12T15:52:53.700 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-12T15:52:53.709 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-12T15:52:53.723 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-12T15:52:53.877 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-12T15:52:53.941 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-12T15:52:53.996 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-12T15:52:54.130 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-12T15:52:54.216 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-12T15:52:54.333 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-12T15:52:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-12T15:52:54.414 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-12T15:52:54.613 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-12T15:52:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-12T15:52:54.763 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-12T15:52:54.774 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-12T15:52:54.783 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 36/134 2023-09-12T15:52:54.872 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-12T15:52:54.934 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 37/134 2023-09-12T15:52:55.004 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-12T15:52:55.009 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-12T15:52:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-12T15:52:55.093 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-12T15:52:55.128 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-12T15:52:55.237 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-12T15:52:55.241 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-12T15:52:55.254 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-12T15:52:55.407 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-12T15:52:55.410 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-12T15:52:55.419 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-12T15:52:55.486 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-12T15:52:55.554 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-12T15:52:55.579 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-12T15:52:55.701 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-12T15:52:55.797 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-12T15:52:55.914 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-12T15:52:55.984 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-12T15:52:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-12T15:52:56.131 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-12T15:52:56.191 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-12T15:52:56.192 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-12T15:52:56.318 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-12T15:52:56.357 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-12T15:52:56.396 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-12T15:52:56.447 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-12T15:52:56.498 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-12T15:52:56.610 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-12T15:52:56.668 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-12T15:52:56.854 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-12T15:52:57.008 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-12T15:52:57.038 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-12T15:52:57.189 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-12T15:52:57.220 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-12T15:52:57.229 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-12T15:52:57.376 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-12T15:52:57.410 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-12T15:52:57.609 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-12T15:52:57.934 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-12T15:52:57.969 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-12T15:52:58.006 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-12T15:52:58.193 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-12T15:52:58.202 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-12T15:52:58.279 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-12T15:52:58.422 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-12T15:52:58.496 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-12T15:52:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-12T15:52:58.730 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-12T15:52:58.835 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-12T15:52:58.888 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-12T15:52:58.981 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-12T15:52:59.034 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-12T15:52:59.133 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-12T15:52:59.289 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-12T15:52:59.294 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-12T15:52:59.332 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-12T15:52:59.460 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 26/134 2023-09-12T15:52:59.485 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-12T15:52:59.573 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 27/134 2023-09-12T15:52:59.594 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-12T15:52:59.653 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-12T15:52:59.815 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 26/134 2023-09-12T15:52:59.907 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-12T15:53:00.064 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-12T15:53:00.238 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-12T15:53:00.254 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 27/134 2023-09-12T15:53:00.475 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 28/134 2023-09-12T15:53:00.492 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 28/134 2023-09-12T15:53:00.535 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-12T15:53:00.684 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-12T15:53:00.799 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-12T15:53:00.803 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-12T15:53:01.040 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-12T15:53:01.065 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-12T15:53:01.295 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-12T15:53:01.315 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-12T15:53:01.451 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-12T15:53:01.503 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-12T15:53:01.621 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-12T15:53:01.878 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-12T15:53:02.116 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-12T15:53:02.238 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-12T15:53:02.403 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-12T15:53:02.456 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-12T15:53:02.600 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-12T15:53:02.794 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-12T15:53:03.002 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-12T15:53:03.179 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-12T15:53:03.266 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-12T15:53:03.306 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-12T15:53:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-12T15:53:03.513 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-12T15:53:03.563 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-12T15:53:03.753 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-12T15:53:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-12T15:53:03.789 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-12T15:53:03.914 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-12T15:53:03.922 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-12T15:53:04.103 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-12T15:53:04.108 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 36/134 2023-09-12T15:53:04.132 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-12T15:53:04.298 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 37/134 2023-09-12T15:53:04.341 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-12T15:53:04.372 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-12T15:53:04.477 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-12T15:53:04.489 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-12T15:53:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-12T15:53:04.646 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-12T15:53:04.696 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-12T15:53:04.802 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-12T15:53:04.981 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 36/134 2023-09-12T15:53:05.157 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 37/134 2023-09-12T15:53:05.164 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-12T15:53:05.298 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-12T15:53:05.333 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-12T15:53:05.477 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-12T15:53:05.511 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-12T15:53:05.666 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-12T15:53:05.723 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-12T15:53:05.726 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-12T15:53:05.805 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-12T15:53:05.962 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-12T15:53:06.085 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-12T15:53:06.121 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-12T15:53:06.195 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-12T15:53:06.367 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-12T15:53:06.375 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-12T15:53:06.381 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-12T15:53:06.452 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-12T15:53:06.579 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-12T15:53:06.657 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-12T15:53:06.683 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-12T15:53:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-12T15:53:06.838 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-12T15:53:06.954 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-12T15:53:06.991 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-12T15:53:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-12T15:53:07.159 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-12T15:53:07.177 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-12T15:53:07.344 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-12T15:53:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-12T15:53:07.479 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-12T15:53:07.610 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-12T15:53:07.716 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-12T15:53:07.795 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-12T15:53:07.907 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-12T15:53:07.962 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-12T15:53:08.015 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-12T15:53:08.117 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-12T15:53:08.173 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-12T15:53:08.188 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-12T15:53:08.301 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-12T15:53:08.343 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 93/134 2023-09-12T15:53:08.381 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-12T15:53:08.626 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-12T15:53:08.694 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-12T15:53:08.886 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-12T15:53:08.951 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-12T15:53:09.096 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-12T15:53:09.208 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-12T15:53:09.346 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-12T15:53:09.450 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-12T15:53:09.634 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-12T15:53:09.649 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-12T15:53:09.770 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-12T15:53:09.832 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-12T15:53:09.952 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-12T15:53:10.017 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-12T15:53:10.052 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-12T15:53:10.263 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-12T15:53:10.271 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-12T15:53:10.464 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-12T15:53:10.478 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-12T15:53:10.610 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-12T15:53:10.644 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-12T15:53:10.687 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-12T15:53:10.901 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-12T15:53:10.969 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-12T15:53:11.087 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-12T15:53:11.223 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-12T15:53:11.253 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-12T15:53:11.425 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-12T15:53:11.462 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-12T15:53:11.694 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-12T15:53:11.717 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-12T15:53:11.898 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-12T15:53:11.918 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-12T15:53:12.053 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-12T15:53:12.057 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-12T15:53:12.110 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-12T15:53:12.189 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-12T15:53:12.300 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-12T15:53:12.367 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-12T15:53:12.413 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-12T15:53:12.607 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-12T15:53:12.647 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-12T15:53:12.778 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-12T15:53:12.781 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-12T15:53:12.894 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-12T15:53:12.959 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-12T15:53:12.963 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-12T15:53:13.091 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-12T15:53:13.140 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-12T15:53:13.317 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-12T15:53:13.368 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-12T15:53:13.548 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-12T15:53:13.549 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-12T15:53:13.743 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-12T15:53:13.749 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-12T15:53:13.830 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-12T15:53:13.878 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-12T15:53:13.920 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-12T15:53:13.975 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-12T15:53:14.049 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-12T15:53:14.098 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-12T15:53:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-12T15:53:14.318 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-12T15:53:14.375 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-12T15:53:14.521 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-12T15:53:14.576 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-12T15:53:14.754 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-12T15:53:14.981 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-12T15:53:15.140 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-12T15:53:15.365 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-12T15:53:15.700 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-12T15:53:15.963 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-12T15:53:16.193 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-12T15:53:16.464 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-12T15:53:16.529 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-12T15:53:16.744 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-12T15:53:16.814 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-12T15:53:17.014 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-12T15:53:17.040 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-12T15:53:17.307 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-12T15:53:17.564 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-12T15:53:17.782 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-12T15:53:17.832 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-12T15:53:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-12T15:53:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-12T15:53:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 93/134 2023-09-12T15:53:18.553 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-12T15:53:18.714 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-12T15:53:18.773 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-12T15:53:18.957 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 93/134 2023-09-12T15:53:21.987 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-12T15:53:22.251 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-12T15:53:22.764 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-12T15:53:22.846 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-12T15:53:23.006 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-12T15:53:23.107 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-12T15:53:23.174 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-12T15:53:23.300 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-12T15:53:23.617 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-12T15:53:23.895 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-12T15:53:23.974 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-12T15:53:24.105 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-12T15:53:32.825 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-12T15:53:32.825 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /sys 2023-09-12T15:53:32.826 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /proc 2023-09-12T15:53:32.826 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /dev 2023-09-12T15:53:32.826 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /run 2023-09-12T15:53:32.826 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /mnt 2023-09-12T15:53:32.827 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /var/tmp 2023-09-12T15:53:32.827 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /home 2023-09-12T15:53:32.827 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /tmp 2023-09-12T15:53:32.827 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /dev 2023-09-12T15:53:32.827 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:53:33.222 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-12T15:53:34.788 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-12T15:53:34.951 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-12T15:53:35.174 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-12T15:53:35.338 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-12T15:53:35.792 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-12T15:53:35.922 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-12T15:53:35.939 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-12T15:53:37.568 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-12T15:53:37.737 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-12T15:53:37.852 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 102/134 2023-09-12T15:53:38.028 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 103/134 2023-09-12T15:53:38.246 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-12T15:53:38.467 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-12T15:53:38.763 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-12T15:53:39.723 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-12T15:53:39.887 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-12T15:53:39.978 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-12T15:53:40.317 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 107/134 2023-09-12T15:53:40.561 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-12T15:53:40.597 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-12T15:53:40.753 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-12T15:53:40.765 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-12T15:53:41.818 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-12T15:53:41.819 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2023-09-12T15:53:41.819 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2023-09-12T15:53:41.819 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2023-09-12T15:53:41.819 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /run 2023-09-12T15:53:41.820 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2023-09-12T15:53:41.820 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2023-09-12T15:53:41.820 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2023-09-12T15:53:41.820 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2023-09-12T15:53:41.820 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2023-09-12T15:53:41.821 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:53:42.065 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 110/134 2023-09-12T15:53:42.139 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-12T15:53:42.236 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-12T15:53:42.270 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-12T15:53:42.412 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-12T15:53:42.463 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-12T15:53:42.637 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-12T15:53:42.995 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-12T15:53:43.051 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-12T15:53:43.051 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2023-09-12T15:53:43.051 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2023-09-12T15:53:43.051 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /dev 2023-09-12T15:53:43.052 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /run 2023-09-12T15:53:43.052 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2023-09-12T15:53:43.052 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2023-09-12T15:53:43.052 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2023-09-12T15:53:43.052 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2023-09-12T15:53:43.053 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /dev 2023-09-12T15:53:43.053 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:53:43.137 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-12T15:53:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-12T15:53:43.708 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-12T15:53:43.879 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-12T15:53:44.101 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-12T15:53:44.281 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-12T15:53:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-12T15:53:44.914 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-12T15:53:44.931 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-12T15:53:45.002 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-12T15:53:45.182 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-12T15:53:45.404 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-12T15:53:45.577 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-12T15:53:46.033 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-12T15:53:46.219 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-12T15:53:46.236 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-12T15:53:46.662 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-12T15:53:46.873 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-12T15:53:47.011 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 102/134 2023-09-12T15:53:47.196 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 103/134 2023-09-12T15:53:47.413 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-12T15:53:47.633 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-12T15:53:47.762 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-12T15:53:47.862 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-12T15:53:47.951 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-12T15:53:48.176 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-12T15:53:48.308 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 102/134 2023-09-12T15:53:48.510 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 103/134 2023-09-12T15:53:48.648 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 115/134 2023-09-12T15:53:48.743 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-12T15:53:48.830 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-12T15:53:48.955 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-12T15:53:48.962 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-12T15:53:49.004 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-12T15:53:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-12T15:53:49.180 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 117/134 2023-09-12T15:53:49.214 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-12T15:53:49.413 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-12T15:53:49.462 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 107/134 2023-09-12T15:53:49.562 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-12T15:53:49.681 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-12T15:53:49.715 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-12T15:53:49.772 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-12T15:53:49.863 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-12T15:53:49.874 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-12T15:53:49.990 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-12T15:53:50.165 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 121/134 2023-09-12T15:53:50.177 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-12T15:53:50.353 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-12T15:53:50.393 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-12T15:53:50.484 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-12T15:53:50.839 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 107/134 2023-09-12T15:53:51.060 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-12T15:53:51.095 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-12T15:53:51.205 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 110/134 2023-09-12T15:53:51.226 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-12T15:53:51.237 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-12T15:53:51.414 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-12T15:53:51.447 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-12T15:53:51.588 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-12T15:53:51.639 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-12T15:53:51.804 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-12T15:53:52.163 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-12T15:53:52.329 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-12T15:53:52.546 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-12T15:53:52.552 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 110/134 2023-09-12T15:53:52.749 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-12T15:53:52.783 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-12T15:53:52.802 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-12T15:53:52.819 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-12T15:53:52.926 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-12T15:53:52.972 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-12T15:53:52.978 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 124/134 2023-09-12T15:53:53.159 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-12T15:53:53.260 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 125/134 2023-09-12T15:53:53.518 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-12T15:53:53.560 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 126/134 2023-09-12T15:53:53.667 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-12T15:53:53.893 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-12T15:53:54.101 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-12T15:53:54.183 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-12T15:53:54.376 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-12T15:53:54.389 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-12T15:53:54.535 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-12T15:53:54.543 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-12T15:53:54.687 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-12T15:53:54.687 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-12T15:53:54.897 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-12T15:53:54.897 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-12T15:53:55.126 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-12T15:53:55.350 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-12T15:53:55.878 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-12T15:53:55.879 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 1/134 2023-09-12T15:53:55.879 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2/134 2023-09-12T15:53:55.879 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 3/134 2023-09-12T15:53:55.879 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 4/134 2023-09-12T15:53:55.880 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 5/134 2023-09-12T15:53:55.880 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-12T15:53:55.880 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-12T15:53:55.880 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-12T15:53:55.880 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 9/134 2023-09-12T15:53:55.881 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 10/134 2023-09-12T15:53:55.881 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 11/134 2023-09-12T15:53:55.881 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 12/134 2023-09-12T15:53:55.881 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 13/134 2023-09-12T15:53:55.882 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 14/134 2023-09-12T15:53:55.882 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 15/134 2023-09-12T15:53:55.882 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 16/134 2023-09-12T15:53:55.882 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 17/134 2023-09-12T15:53:55.882 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 18/134 2023-09-12T15:53:55.883 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 19/134 2023-09-12T15:53:55.883 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 20/134 2023-09-12T15:53:55.883 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 21/134 2023-09-12T15:53:55.883 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 22/134 2023-09-12T15:53:55.883 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 23/134 2023-09-12T15:53:55.884 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 24/134 2023-09-12T15:53:55.884 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 25/134 2023-09-12T15:53:55.884 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 26/134 2023-09-12T15:53:55.884 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 27/134 2023-09-12T15:53:55.895 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 28/134 2023-09-12T15:53:55.895 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 29/134 2023-09-12T15:53:55.895 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 30/134 2023-09-12T15:53:55.896 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 31/134 2023-09-12T15:53:55.896 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 32/134 2023-09-12T15:53:55.896 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 33/134 2023-09-12T15:53:55.896 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 34/134 2023-09-12T15:53:55.896 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 35/134 2023-09-12T15:53:55.897 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 36/134 2023-09-12T15:53:55.897 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 37/134 2023-09-12T15:53:55.897 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 38/134 2023-09-12T15:53:55.897 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-12T15:53:55.898 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-12T15:53:55.898 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-12T15:53:55.898 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-12T15:53:55.898 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-12T15:53:55.898 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-12T15:53:55.899 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-12T15:53:55.899 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-12T15:53:55.899 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-12T15:53:55.899 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-12T15:53:55.900 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-12T15:53:55.900 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-12T15:53:55.900 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-12T15:53:55.900 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-12T15:53:55.900 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-12T15:53:55.901 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-12T15:53:55.901 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-12T15:53:55.901 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-12T15:53:55.901 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-12T15:53:55.901 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-12T15:53:55.902 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-12T15:53:55.902 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-12T15:53:55.902 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-12T15:53:55.902 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-12T15:53:55.902 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-12T15:53:55.903 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-12T15:53:55.903 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-12T15:53:55.903 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-12T15:53:55.903 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-12T15:53:55.903 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-12T15:53:55.904 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-12T15:53:55.904 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-12T15:53:55.904 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-12T15:53:55.904 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-12T15:53:55.904 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-12T15:53:55.905 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-12T15:53:55.905 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-12T15:53:55.905 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-12T15:53:55.905 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-12T15:53:55.905 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-12T15:53:55.906 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-12T15:53:55.906 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-12T15:53:55.906 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-12T15:53:55.906 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-12T15:53:55.906 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-12T15:53:55.907 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-12T15:53:55.907 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-12T15:53:55.907 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-12T15:53:55.907 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-12T15:53:55.908 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-12T15:53:55.908 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-12T15:53:55.908 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-12T15:53:55.908 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-12T15:53:55.908 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-12T15:53:55.909 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-12T15:53:55.909 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-12T15:53:55.909 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-12T15:53:55.909 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-12T15:53:55.910 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-12T15:53:55.910 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-12T15:53:55.910 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-12T15:53:55.910 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-12T15:53:55.910 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-12T15:53:55.911 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-12T15:53:55.911 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-12T15:53:55.911 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-12T15:53:55.911 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-12T15:53:55.912 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-12T15:53:55.912 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-12T15:53:55.912 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-12T15:53:55.912 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-12T15:53:55.912 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-12T15:53:55.913 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-12T15:53:55.913 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-12T15:53:55.913 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-12T15:53:55.913 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-12T15:53:55.914 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-12T15:53:55.914 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-12T15:53:55.914 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-12T15:53:55.914 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-12T15:53:55.914 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-12T15:53:55.915 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-12T15:53:55.915 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-12T15:53:55.915 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-12T15:53:55.915 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-12T15:53:55.915 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-12T15:53:55.916 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-12T15:53:55.916 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-12T15:53:55.916 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-12T15:53:55.916 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-12T15:53:55.916 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 129/134 2023-09-12T15:53:55.917 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-12T15:53:55.917 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 131/134 2023-09-12T15:53:55.917 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-12T15:53:55.917 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-12T15:53:56.971 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-12T15:53:57.273 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 115/134 2023-09-12T15:53:57.420 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-12T15:53:57.612 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-12T15:53:58.315 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-12T15:53:58.449 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-12T15:53:58.449 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:49 ago on Tue 12 Sep 2023 03:52:09 PM UTC. 2023-09-12T15:53:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-12T15:53:58.864 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-12T15:53:59.031 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 119/134 2023-09-12T15:53:59.182 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 115/134 2023-09-12T15:53:59.224 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-12T15:53:59.382 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 121/134 2023-09-12T15:53:59.494 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-12T15:53:59.529 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-12T15:53:59.661 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 117/134 2023-09-12T15:53:59.887 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-12T15:54:00.049 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-12T15:54:00.241 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-12T15:54:00.504 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-12T15:54:00.712 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 121/134 2023-09-12T15:54:00.850 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-12T15:54:01.042 INFO:teuthology.orchestra.run.smithi130.stdout:Installed products updated. 2023-09-12T15:54:01.260 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:54:01.261 INFO:teuthology.orchestra.run.smithi130.stdout:Upgraded: 2023-09-12T15:54:01.261 INFO:teuthology.orchestra.run.smithi130.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-12T15:54:01.261 INFO:teuthology.orchestra.run.smithi130.stdout: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.261 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.262 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2023-09-12T15:54:01.262 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-12T15:54:01.262 INFO:teuthology.orchestra.run.smithi130.stdout: automake-1.16.1-7.el8.noarch 2023-09-12T15:54:01.262 INFO:teuthology.orchestra.run.smithi130.stdout: bison-3.0.4-10.el8.x86_64 2023-09-12T15:54:01.262 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.263 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.263 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.263 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.263 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.263 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.264 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.264 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.264 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.264 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.264 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.264 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.265 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.265 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.265 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.265 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.265 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.266 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.266 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.266 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.266 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.266 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.267 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:01.267 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-12T15:54:01.267 INFO:teuthology.orchestra.run.smithi130.stdout: flex-2.6.1-9.el8.x86_64 2023-09-12T15:54:01.267 INFO:teuthology.orchestra.run.smithi130.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-12T15:54:01.267 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-12T15:54:01.267 INFO:teuthology.orchestra.run.smithi130.stdout: jq-1.6-3.el8.x86_64 2023-09-12T15:54:01.268 INFO:teuthology.orchestra.run.smithi130.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-12T15:54:01.268 INFO:teuthology.orchestra.run.smithi130.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-12T15:54:01.268 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-12T15:54:01.268 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.268 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.269 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.269 INFO:teuthology.orchestra.run.smithi130.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-12T15:54:01.269 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-12T15:54:01.269 INFO:teuthology.orchestra.run.smithi130.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-12T15:54:01.269 INFO:teuthology.orchestra.run.smithi130.stdout: libpq-13.5-1.el8.x86_64 2023-09-12T15:54:01.270 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-12T15:54:01.270 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-12T15:54:01.270 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.270 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.270 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-12T15:54:01.271 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.271 INFO:teuthology.orchestra.run.smithi130.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-12T15:54:01.271 INFO:teuthology.orchestra.run.smithi130.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-12T15:54:01.271 INFO:teuthology.orchestra.run.smithi130.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-12T15:54:01.271 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-12T15:54:01.271 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-12T15:54:01.272 INFO:teuthology.orchestra.run.smithi130.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-12T15:54:01.272 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-12T15:54:01.272 INFO:teuthology.orchestra.run.smithi130.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-12T15:54:01.272 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-12T15:54:01.272 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-12T15:54:01.273 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-12T15:54:01.273 INFO:teuthology.orchestra.run.smithi130.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-12T15:54:01.273 INFO:teuthology.orchestra.run.smithi130.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-12T15:54:01.273 INFO:teuthology.orchestra.run.smithi130.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-12T15:54:01.273 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-12T15:54:01.273 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-12T15:54:01.274 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-12T15:54:01.274 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-12T15:54:01.274 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-12T15:54:01.274 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-12T15:54:01.274 INFO:teuthology.orchestra.run.smithi130.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-12T15:54:01.275 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-12T15:54:01.275 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.275 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.275 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.275 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-12T15:54:01.276 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-12T15:54:01.276 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-12T15:54:01.276 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-12T15:54:01.276 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-12T15:54:01.276 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-12T15:54:01.277 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-12T15:54:01.277 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-12T15:54:01.277 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-12T15:54:01.277 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-12T15:54:01.277 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-12T15:54:01.278 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-12T15:54:01.278 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-12T15:54:01.278 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-12T15:54:01.278 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-12T15:54:01.278 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-12T15:54:01.279 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-12T15:54:01.279 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-12T15:54:01.279 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-12T15:54:01.279 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-12T15:54:01.279 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-12T15:54:01.280 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-12T15:54:01.280 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend-2.6-1.el8.noarch 2023-09-12T15:54:01.280 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-12T15:54:01.280 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-12T15:54:01.280 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-12T15:54:01.280 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-12T15:54:01.281 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-12T15:54:01.281 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-12T15:54:01.281 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.281 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.281 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-12T15:54:01.282 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-12T15:54:01.282 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.282 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-12T15:54:01.282 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-12T15:54:01.282 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-12T15:54:01.282 INFO:teuthology.orchestra.run.smithi130.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-12T15:54:01.283 INFO:teuthology.orchestra.run.smithi130.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-12T15:54:01.283 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-12T15:54:01.283 INFO:teuthology.orchestra.run.smithi130.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-12T15:54:01.283 INFO:teuthology.orchestra.run.smithi130.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-12T15:54:01.283 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-12T15:54:01.284 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-12T15:54:01.284 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-12T15:54:01.284 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-12T15:54:01.284 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-12T15:54:01.284 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.284 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.285 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:01.285 INFO:teuthology.orchestra.run.smithi130.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-12T15:54:01.285 INFO:teuthology.orchestra.run.smithi130.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-12T15:54:01.285 INFO:teuthology.orchestra.run.smithi130.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-12T15:54:01.285 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-12T15:54:01.285 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-12T15:54:01.286 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-12T15:54:01.286 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:54:01.286 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2023-09-12T15:54:01.467 DEBUG:teuthology.parallel:result is None 2023-09-12T15:54:01.725 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-12T15:54:02.003 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-12T15:54:02.020 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-12T15:54:02.189 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 124/134 2023-09-12T15:54:02.444 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 125/134 2023-09-12T15:54:02.792 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 126/134 2023-09-12T15:54:03.071 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-12T15:54:03.115 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-12T15:54:03.227 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-12T15:54:03.235 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-12T15:54:03.341 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-12T15:54:03.360 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-12T15:54:03.418 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-12T15:54:03.541 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 124/134 2023-09-12T15:54:03.610 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-12T15:54:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-12T15:54:03.799 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 125/134 2023-09-12T15:54:03.935 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-12T15:54:03.947 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-12T15:54:03.948 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-12T15:54:04.109 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 126/134 2023-09-12T15:54:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-12T15:54:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-12T15:54:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-12T15:54:04.457 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-12T15:54:04.626 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-12T15:54:04.632 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-12T15:54:04.734 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-12T15:54:04.915 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-12T15:54:04.928 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-12T15:54:05.082 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-12T15:54:05.090 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-12T15:54:05.124 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-12T15:54:05.125 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 1/134 2023-09-12T15:54:05.125 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2/134 2023-09-12T15:54:05.125 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 3/134 2023-09-12T15:54:05.125 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 4/134 2023-09-12T15:54:05.125 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 5/134 2023-09-12T15:54:05.126 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-12T15:54:05.126 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-12T15:54:05.126 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-12T15:54:05.126 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 9/134 2023-09-12T15:54:05.126 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 10/134 2023-09-12T15:54:05.127 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 11/134 2023-09-12T15:54:05.127 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 12/134 2023-09-12T15:54:05.127 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 13/134 2023-09-12T15:54:05.127 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 14/134 2023-09-12T15:54:05.127 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 15/134 2023-09-12T15:54:05.128 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 16/134 2023-09-12T15:54:05.128 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 17/134 2023-09-12T15:54:05.128 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 18/134 2023-09-12T15:54:05.128 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 19/134 2023-09-12T15:54:05.128 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 20/134 2023-09-12T15:54:05.128 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 21/134 2023-09-12T15:54:05.129 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 22/134 2023-09-12T15:54:05.129 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 23/134 2023-09-12T15:54:05.129 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 24/134 2023-09-12T15:54:05.129 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 25/134 2023-09-12T15:54:05.129 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 26/134 2023-09-12T15:54:05.130 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 27/134 2023-09-12T15:54:05.130 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 28/134 2023-09-12T15:54:05.130 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 29/134 2023-09-12T15:54:05.139 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 30/134 2023-09-12T15:54:05.139 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 31/134 2023-09-12T15:54:05.139 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 32/134 2023-09-12T15:54:05.139 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 33/134 2023-09-12T15:54:05.140 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 34/134 2023-09-12T15:54:05.140 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 35/134 2023-09-12T15:54:05.140 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 36/134 2023-09-12T15:54:05.140 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 37/134 2023-09-12T15:54:05.140 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 38/134 2023-09-12T15:54:05.141 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-12T15:54:05.141 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-12T15:54:05.141 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-12T15:54:05.141 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-12T15:54:05.141 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-12T15:54:05.141 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-12T15:54:05.142 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-12T15:54:05.142 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-12T15:54:05.142 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-12T15:54:05.142 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-12T15:54:05.142 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-12T15:54:05.142 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-12T15:54:05.143 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-12T15:54:05.143 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-12T15:54:05.143 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-12T15:54:05.143 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-12T15:54:05.143 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-12T15:54:05.144 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-12T15:54:05.144 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-12T15:54:05.144 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-12T15:54:05.144 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-12T15:54:05.144 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-12T15:54:05.145 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-12T15:54:05.145 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-12T15:54:05.145 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-12T15:54:05.145 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-12T15:54:05.145 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-12T15:54:05.146 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-12T15:54:05.146 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-12T15:54:05.146 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-12T15:54:05.146 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-12T15:54:05.146 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-12T15:54:05.146 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-12T15:54:05.147 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-12T15:54:05.147 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-12T15:54:05.147 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-12T15:54:05.147 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-12T15:54:05.147 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-12T15:54:05.148 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-12T15:54:05.148 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-12T15:54:05.148 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-12T15:54:05.148 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-12T15:54:05.148 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-12T15:54:05.148 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-12T15:54:05.149 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-12T15:54:05.149 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-12T15:54:05.149 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-12T15:54:05.149 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-12T15:54:05.149 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-12T15:54:05.150 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-12T15:54:05.150 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-12T15:54:05.150 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-12T15:54:05.150 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-12T15:54:05.150 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-12T15:54:05.151 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-12T15:54:05.151 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-12T15:54:05.151 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-12T15:54:05.151 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-12T15:54:05.151 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-12T15:54:05.151 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-12T15:54:05.152 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-12T15:54:05.152 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-12T15:54:05.152 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-12T15:54:05.152 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-12T15:54:05.152 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-12T15:54:05.153 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-12T15:54:05.153 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-12T15:54:05.153 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-12T15:54:05.153 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-12T15:54:05.153 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-12T15:54:05.153 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-12T15:54:05.154 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-12T15:54:05.154 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-12T15:54:05.154 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-12T15:54:05.154 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-12T15:54:05.154 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-12T15:54:05.155 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-12T15:54:05.155 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-12T15:54:05.155 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-12T15:54:05.155 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-12T15:54:05.155 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-12T15:54:05.156 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-12T15:54:05.156 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-12T15:54:05.156 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-12T15:54:05.156 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-12T15:54:05.156 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-12T15:54:05.156 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-12T15:54:05.157 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-12T15:54:05.157 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-12T15:54:05.157 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-12T15:54:05.157 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 129/134 2023-09-12T15:54:05.158 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-12T15:54:05.158 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 131/134 2023-09-12T15:54:05.158 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-12T15:54:05.158 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-12T15:54:05.242 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-12T15:54:05.243 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-12T15:54:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-12T15:54:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-12T15:54:05.721 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-12T15:54:05.981 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-12T15:54:06.513 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-12T15:54:06.514 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 1/134 2023-09-12T15:54:06.514 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2/134 2023-09-12T15:54:06.514 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 3/134 2023-09-12T15:54:06.514 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 4/134 2023-09-12T15:54:06.515 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 5/134 2023-09-12T15:54:06.515 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-12T15:54:06.515 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-12T15:54:06.515 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-12T15:54:06.515 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 9/134 2023-09-12T15:54:06.516 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 10/134 2023-09-12T15:54:06.516 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 11/134 2023-09-12T15:54:06.516 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 12/134 2023-09-12T15:54:06.516 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 13/134 2023-09-12T15:54:06.516 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 14/134 2023-09-12T15:54:06.517 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 15/134 2023-09-12T15:54:06.517 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 16/134 2023-09-12T15:54:06.517 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 17/134 2023-09-12T15:54:06.517 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 18/134 2023-09-12T15:54:06.517 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 19/134 2023-09-12T15:54:06.518 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 20/134 2023-09-12T15:54:06.518 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 21/134 2023-09-12T15:54:06.518 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 22/134 2023-09-12T15:54:06.518 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 23/134 2023-09-12T15:54:06.518 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 24/134 2023-09-12T15:54:06.518 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 25/134 2023-09-12T15:54:06.519 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 26/134 2023-09-12T15:54:06.519 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 27/134 2023-09-12T15:54:06.519 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 28/134 2023-09-12T15:54:06.519 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 29/134 2023-09-12T15:54:06.519 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 30/134 2023-09-12T15:54:06.520 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 31/134 2023-09-12T15:54:06.520 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 32/134 2023-09-12T15:54:06.520 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 33/134 2023-09-12T15:54:06.520 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 34/134 2023-09-12T15:54:06.520 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 35/134 2023-09-12T15:54:06.520 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 36/134 2023-09-12T15:54:06.521 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 37/134 2023-09-12T15:54:06.521 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 38/134 2023-09-12T15:54:06.521 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-12T15:54:06.521 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-12T15:54:06.521 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-12T15:54:06.522 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-12T15:54:06.522 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-12T15:54:06.522 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-12T15:54:06.522 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-12T15:54:06.522 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-12T15:54:06.523 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-12T15:54:06.523 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-12T15:54:06.523 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-12T15:54:06.523 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-12T15:54:06.523 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-12T15:54:06.523 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-12T15:54:06.524 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-12T15:54:06.524 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-12T15:54:06.524 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-12T15:54:06.524 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-12T15:54:06.524 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-12T15:54:06.524 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-12T15:54:06.525 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-12T15:54:06.525 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-12T15:54:06.525 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-12T15:54:06.525 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-12T15:54:06.525 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-12T15:54:06.526 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-12T15:54:06.526 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-12T15:54:06.526 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-12T15:54:06.526 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-12T15:54:06.526 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-12T15:54:06.527 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-12T15:54:06.527 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-12T15:54:06.527 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-12T15:54:06.527 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-12T15:54:06.527 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-12T15:54:06.527 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-12T15:54:06.528 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-12T15:54:06.528 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-12T15:54:06.528 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-12T15:54:06.528 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-12T15:54:06.528 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-12T15:54:06.529 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-12T15:54:06.529 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-12T15:54:06.529 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-12T15:54:06.529 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-12T15:54:06.529 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-12T15:54:06.529 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-12T15:54:06.530 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-12T15:54:06.530 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-12T15:54:06.530 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-12T15:54:06.530 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-12T15:54:06.530 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-12T15:54:06.531 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-12T15:54:06.531 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-12T15:54:06.531 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-12T15:54:06.531 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-12T15:54:06.531 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-12T15:54:06.532 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-12T15:54:06.532 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-12T15:54:06.532 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-12T15:54:06.532 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-12T15:54:06.532 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-12T15:54:06.533 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-12T15:54:06.533 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-12T15:54:06.533 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-12T15:54:06.533 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-12T15:54:06.533 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-12T15:54:06.533 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-12T15:54:06.534 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-12T15:54:06.534 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-12T15:54:06.534 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-12T15:54:06.534 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-12T15:54:06.534 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-12T15:54:06.535 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-12T15:54:06.535 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-12T15:54:06.535 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-12T15:54:06.535 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-12T15:54:06.535 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-12T15:54:06.536 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-12T15:54:06.536 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-12T15:54:06.536 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-12T15:54:06.536 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-12T15:54:06.536 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-12T15:54:06.536 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-12T15:54:06.537 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-12T15:54:06.537 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-12T15:54:06.537 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-12T15:54:06.537 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-12T15:54:06.537 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-12T15:54:06.537 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-12T15:54:06.538 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 129/134 2023-09-12T15:54:06.538 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-12T15:54:06.538 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 131/134 2023-09-12T15:54:06.538 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-12T15:54:06.538 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-12T15:54:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-12T15:54:07.556 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:58 ago on Tue 12 Sep 2023 03:52:09 PM UTC. 2023-09-12T15:54:09.056 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-12T15:54:09.056 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:02:00 ago on Tue 12 Sep 2023 03:52:09 PM UTC. 2023-09-12T15:54:10.169 INFO:teuthology.orchestra.run.smithi159.stdout:Installed products updated. 2023-09-12T15:54:10.388 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:54:10.389 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2023-09-12T15:54:10.389 INFO:teuthology.orchestra.run.smithi159.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-12T15:54:10.389 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.389 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.389 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-09-12T15:54:10.390 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-12T15:54:10.390 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.1-7.el8.noarch 2023-09-12T15:54:10.390 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.0.4-10.el8.x86_64 2023-09-12T15:54:10.390 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.390 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.391 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.391 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.391 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.391 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.391 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.392 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.392 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.392 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.392 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.392 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.393 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.393 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.393 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.393 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.393 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.394 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.394 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.394 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.394 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.394 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.394 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:10.395 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-12T15:54:10.395 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.1-9.el8.x86_64 2023-09-12T15:54:10.395 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-12T15:54:10.395 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-12T15:54:10.395 INFO:teuthology.orchestra.run.smithi159.stdout: jq-1.6-3.el8.x86_64 2023-09-12T15:54:10.396 INFO:teuthology.orchestra.run.smithi159.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-12T15:54:10.396 INFO:teuthology.orchestra.run.smithi159.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-12T15:54:10.396 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-12T15:54:10.396 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.396 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.397 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.397 INFO:teuthology.orchestra.run.smithi159.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-12T15:54:10.397 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-12T15:54:10.397 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-12T15:54:10.397 INFO:teuthology.orchestra.run.smithi159.stdout: libpq-13.5-1.el8.x86_64 2023-09-12T15:54:10.397 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-12T15:54:10.398 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-12T15:54:10.398 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.398 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.398 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-12T15:54:10.398 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.399 INFO:teuthology.orchestra.run.smithi159.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-12T15:54:10.399 INFO:teuthology.orchestra.run.smithi159.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-12T15:54:10.399 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-12T15:54:10.399 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-12T15:54:10.399 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-12T15:54:10.399 INFO:teuthology.orchestra.run.smithi159.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-12T15:54:10.400 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-12T15:54:10.400 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-12T15:54:10.400 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-12T15:54:10.400 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-12T15:54:10.400 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-12T15:54:10.401 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-12T15:54:10.401 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-12T15:54:10.401 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-12T15:54:10.401 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-12T15:54:10.401 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-12T15:54:10.401 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-12T15:54:10.402 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-12T15:54:10.402 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-12T15:54:10.402 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-12T15:54:10.402 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-12T15:54:10.402 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-12T15:54:10.403 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.403 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.403 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.403 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-12T15:54:10.403 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-12T15:54:10.403 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-12T15:54:10.404 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-12T15:54:10.404 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-12T15:54:10.404 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-12T15:54:10.404 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-12T15:54:10.404 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-12T15:54:10.405 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-12T15:54:10.405 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-12T15:54:10.405 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-12T15:54:10.405 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-12T15:54:10.405 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-12T15:54:10.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-12T15:54:10.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-12T15:54:10.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-12T15:54:10.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-12T15:54:10.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-12T15:54:10.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-12T15:54:10.407 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-12T15:54:10.407 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-12T15:54:10.407 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-12T15:54:10.407 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-2.6-1.el8.noarch 2023-09-12T15:54:10.407 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-12T15:54:10.408 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-12T15:54:10.408 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-12T15:54:10.408 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-12T15:54:10.408 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-12T15:54:10.408 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-12T15:54:10.408 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.409 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.409 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-12T15:54:10.409 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-12T15:54:10.409 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.409 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-12T15:54:10.410 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-12T15:54:10.410 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-12T15:54:10.410 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-12T15:54:10.410 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-12T15:54:10.410 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-12T15:54:10.410 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-12T15:54:10.411 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-12T15:54:10.411 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-12T15:54:10.411 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-12T15:54:10.411 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-12T15:54:10.411 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-12T15:54:10.412 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-12T15:54:10.412 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.412 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.412 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:10.412 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-12T15:54:10.412 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-12T15:54:10.413 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-12T15:54:10.413 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-12T15:54:10.413 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-12T15:54:10.414 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-12T15:54:10.414 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-09-12T15:54:10.414 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-09-12T15:54:10.599 DEBUG:teuthology.parallel:result is None 2023-09-12T15:54:11.705 INFO:teuthology.orchestra.run.smithi203.stdout:Installed products updated. 2023-09-12T15:54:11.919 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:54:11.919 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2023-09-12T15:54:11.919 INFO:teuthology.orchestra.run.smithi203.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-12T15:54:11.920 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.920 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.920 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2023-09-12T15:54:11.920 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-12T15:54:11.921 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.1-7.el8.noarch 2023-09-12T15:54:11.921 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.0.4-10.el8.x86_64 2023-09-12T15:54:11.921 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.921 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.921 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.922 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.922 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.922 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.922 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.922 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.923 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.923 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.923 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.923 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.923 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.924 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.924 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.924 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.924 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.924 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.925 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.925 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.925 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.925 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.925 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-12T15:54:11.926 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-12T15:54:11.926 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.1-9.el8.x86_64 2023-09-12T15:54:11.926 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-12T15:54:11.926 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-12T15:54:11.927 INFO:teuthology.orchestra.run.smithi203.stdout: jq-1.6-3.el8.x86_64 2023-09-12T15:54:11.927 INFO:teuthology.orchestra.run.smithi203.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-12T15:54:11.927 INFO:teuthology.orchestra.run.smithi203.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-12T15:54:11.927 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-12T15:54:11.927 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.928 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.928 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.928 INFO:teuthology.orchestra.run.smithi203.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-12T15:54:11.928 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-12T15:54:11.928 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-12T15:54:11.929 INFO:teuthology.orchestra.run.smithi203.stdout: libpq-13.5-1.el8.x86_64 2023-09-12T15:54:11.929 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-12T15:54:11.929 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-12T15:54:11.929 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.929 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.930 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-12T15:54:11.930 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.930 INFO:teuthology.orchestra.run.smithi203.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-12T15:54:11.930 INFO:teuthology.orchestra.run.smithi203.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-12T15:54:11.930 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-12T15:54:11.931 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-12T15:54:11.931 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-12T15:54:11.931 INFO:teuthology.orchestra.run.smithi203.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-12T15:54:11.931 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-12T15:54:11.931 INFO:teuthology.orchestra.run.smithi203.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-12T15:54:11.932 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-12T15:54:11.932 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-12T15:54:11.932 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-12T15:54:11.932 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-12T15:54:11.932 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-12T15:54:11.933 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-12T15:54:11.933 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-12T15:54:11.933 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-12T15:54:11.933 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-12T15:54:11.933 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-12T15:54:11.934 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-12T15:54:11.934 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-12T15:54:11.934 INFO:teuthology.orchestra.run.smithi203.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-12T15:54:11.934 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-12T15:54:11.934 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.935 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.935 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.935 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-12T15:54:11.935 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-12T15:54:11.935 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-12T15:54:11.936 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-12T15:54:11.936 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-12T15:54:11.936 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-12T15:54:11.936 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-12T15:54:11.936 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-12T15:54:11.937 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-12T15:54:11.937 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-12T15:54:11.937 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-12T15:54:11.937 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-12T15:54:11.937 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-12T15:54:11.938 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-12T15:54:11.938 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-12T15:54:11.938 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-12T15:54:11.938 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-12T15:54:11.938 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-12T15:54:11.939 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-12T15:54:11.939 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-12T15:54:11.939 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-12T15:54:11.939 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-12T15:54:11.939 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-2.6-1.el8.noarch 2023-09-12T15:54:11.940 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-12T15:54:11.940 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-12T15:54:11.940 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-12T15:54:11.940 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-12T15:54:11.940 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-12T15:54:11.941 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-12T15:54:11.941 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.941 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.941 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-12T15:54:11.942 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-12T15:54:11.942 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.942 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-12T15:54:11.942 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-12T15:54:11.942 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-12T15:54:11.943 INFO:teuthology.orchestra.run.smithi203.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-12T15:54:11.943 INFO:teuthology.orchestra.run.smithi203.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-12T15:54:11.943 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-12T15:54:11.943 INFO:teuthology.orchestra.run.smithi203.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-12T15:54:11.943 INFO:teuthology.orchestra.run.smithi203.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-12T15:54:11.944 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-12T15:54:11.944 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-12T15:54:11.944 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-12T15:54:11.944 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-12T15:54:11.944 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-12T15:54:11.945 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.945 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.945 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-12T15:54:11.945 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-12T15:54:11.945 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-12T15:54:11.946 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-12T15:54:11.946 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-12T15:54:11.946 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-12T15:54:11.947 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-12T15:54:11.947 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:54:11.947 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-09-12T15:54:12.118 DEBUG:teuthology.parallel:result is None 2023-09-12T15:54:12.119 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:54:12.346 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-12T15:54:12.375 INFO:teuthology.orchestra.run.smithi130.stdout:18.0.0-6088.g2110e007.el8 2023-09-12T15:54:12.376 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6088.g2110e007.el8 2023-09-12T15:54:12.376 INFO:teuthology.task.install:The correct ceph version 18.0.0-6088.g2110e007 is installed. 2023-09-12T15:54:12.377 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:54:12.521 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-12T15:54:12.549 INFO:teuthology.orchestra.run.smithi159.stdout:18.0.0-6088.g2110e007.el8 2023-09-12T15:54:12.550 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6088.g2110e007.el8 2023-09-12T15:54:12.550 INFO:teuthology.task.install:The correct ceph version 18.0.0-6088.g2110e007 is installed. 2023-09-12T15:54:12.551 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:54:12.767 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-12T15:54:12.794 INFO:teuthology.orchestra.run.smithi203.stdout:18.0.0-6088.g2110e007.el8 2023-09-12T15:54:12.794 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6088.g2110e007.el8 2023-09-12T15:54:12.794 INFO:teuthology.task.install:The correct ceph version 18.0.0-6088.g2110e007 is installed. 2023-09-12T15:54:12.795 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-12T15:54:12.795 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:54:12.796 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-12T15:54:12.844 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:54:12.844 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-12T15:54:12.892 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:54:12.892 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-12T15:54:12.940 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-12T15:54:12.940 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:54:12.941 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2023-09-12T15:54:12.971 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-12T15:54:13.041 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:54:13.041 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2023-09-12T15:54:13.073 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-12T15:54:13.143 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:54:13.143 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2023-09-12T15:54:13.176 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-12T15:54:13.248 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-12T15:54:13.249 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:54:13.249 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-12T15:54:13.278 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-12T15:54:13.347 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:54:13.347 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-12T15:54:13.376 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-12T15:54:13.445 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:54:13.445 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-12T15:54:13.476 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-12T15:54:13.546 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-12T15:54:13.547 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:54:13.547 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/stdin-killer 2023-09-12T15:54:13.575 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-12T15:54:13.642 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:54:13.643 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2023-09-12T15:54:13.672 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-12T15:54:13.740 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:54:13.741 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2023-09-12T15:54:13.771 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-12T15:54:13.841 INFO:teuthology.run_tasks:Running task cephadm... 2023-09-12T15:54:13.925 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-09-12T15:54:13.926 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:54:13.927 INFO:tasks.cephadm:Cluster fsid is 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:54:13.927 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-09-12T15:54:13.927 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.130', 'mon.b': '172.21.15.159', 'mon.c': '172.21.15.203'} 2023-09-12T15:54:13.928 INFO:tasks.cephadm:First mon is mon.a on smithi130 2023-09-12T15:54:13.928 INFO:tasks.cephadm:First mgr is x 2023-09-12T15:54:13.929 INFO:tasks.cephadm:Normalizing hostnames... 2023-09-12T15:54:13.929 DEBUG:teuthology.orchestra.run.smithi130:> sudo hostname $(hostname -s) 2023-09-12T15:54:13.962 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2023-09-12T15:54:13.991 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2023-09-12T15:54:14.023 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-09-12T15:54:14.024 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:54:14.240 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72559/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6088-g2110e007', 'node_name': '172.21.5.32+adami02', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6088.g2110e007'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-12 14:00:53.539150', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20230912.122437', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-09-12T15:54:14.453 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20230912.122437, sha1 2110e00747b31a0d73768c9e5229da18b26b8aa0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T15:54:14.453 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/x86_64/flavors/default/cephadm 2023-09-12T15:54:14.454 DEBUG:teuthology.orchestra.run.smithi130:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-12T15:54:15.368 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-rw-r--. 1 ubuntu ubuntu 728366 Sep 12 15:54 /home/ubuntu/cephtest/cephadm 2023-09-12T15:54:15.369 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-12T15:54:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-rw-r--. 1 ubuntu ubuntu 728366 Sep 12 15:54 /home/ubuntu/cephtest/cephadm 2023-09-12T15:54:16.210 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-12T15:54:17.081 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-rw-r--. 1 ubuntu ubuntu 728366 Sep 12 15:54 /home/ubuntu/cephtest/cephadm 2023-09-12T15:54:17.081 DEBUG:teuthology.orchestra.run.smithi130:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-12T15:54:17.103 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-12T15:54:17.125 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-12T15:54:17.161 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 on all hosts... 2023-09-12T15:54:17.161 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 pull 2023-09-12T15:54:17.164 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 pull 2023-09-12T15:54:17.167 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 pull 2023-09-12T15:54:20.271 INFO:teuthology.orchestra.run.smithi130.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-12T15:54:22.436 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-12T15:54:23.487 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-12T15:54:45.750 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T15:54:45.751 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T15:54:45.751 INFO:teuthology.orchestra.run.smithi130.stdout: "image_id": "bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed", 2023-09-12T15:54:45.751 INFO:teuthology.orchestra.run.smithi130.stdout: "repo_digests": [ 2023-09-12T15:54:45.751 INFO:teuthology.orchestra.run.smithi130.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:245225109b90f7de349b7e93f44579ca2c2de61e498376ad1f95f5caf5575b65" 2023-09-12T15:54:45.752 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T15:54:45.752 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T15:54:51.026 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-09-12T15:54:51.026 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T15:54:51.027 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed", 2023-09-12T15:54:51.027 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2023-09-12T15:54:51.027 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:245225109b90f7de349b7e93f44579ca2c2de61e498376ad1f95f5caf5575b65" 2023-09-12T15:54:51.027 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2023-09-12T15:54:51.027 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-09-12T15:54:52.485 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-09-12T15:54:52.486 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T15:54:52.486 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed", 2023-09-12T15:54:52.486 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2023-09-12T15:54:52.486 INFO:teuthology.orchestra.run.smithi203.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:245225109b90f7de349b7e93f44579ca2c2de61e498376ad1f95f5caf5575b65" 2023-09-12T15:54:52.486 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2023-09-12T15:54:52.487 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-09-12T15:54:52.505 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph 2023-09-12T15:54:52.542 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2023-09-12T15:54:52.581 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2023-09-12T15:54:52.612 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /etc/ceph 2023-09-12T15:54:52.644 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2023-09-12T15:54:52.675 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2023-09-12T15:54:52.702 INFO:tasks.cephadm:Writing seed config... 2023-09-12T15:54:52.704 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-09-12T15:54:52.704 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-09-12T15:54:52.704 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-09-12T15:54:52.704 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-09-12T15:54:52.705 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-09-12T15:54:52.705 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-09-12T15:54:52.705 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-09-12T15:54:52.705 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-09-12T15:54:52.706 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-09-12T15:54:52.706 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-09-12T15:54:52.706 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-09-12T15:54:52.706 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-09-12T15:54:52.707 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-09-12T15:54:52.707 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-09-12T15:54:52.707 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-09-12T15:54:52.707 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-09-12T15:54:52.708 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-09-12T15:54:52.708 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-09-12T15:54:52.708 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-09-12T15:54:52.708 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-09-12T15:54:52.709 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-09-12T15:54:52.709 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-09-12T15:54:52.709 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-09-12T15:54:52.710 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:54:52.710 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-09-12T15:54:52.731 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2023-09-12T15:54:52.731 DEBUG:teuthology.orchestra.run.smithi130:mon.a> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.a.service 2023-09-12T15:54:52.775 DEBUG:teuthology.orchestra.run.smithi130:mgr.x> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.x.service 2023-09-12T15:54:52.805 INFO:journalctl@ceph.mon.a.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T15:54:52.806 INFO:tasks.cephadm:Bootstrapping... 2023-09-12T15:54:52.806 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 -v bootstrap --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.130 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-09-12T15:54:52.834 INFO:journalctl@ceph.mgr.x.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T15:54:52.972 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2023-09-12T15:54:52.972 INFO:teuthology.orchestra.run.smithi130.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0', '-v', 'bootstrap', '--fsid', '9ef22ea2-5184-11ee-9ab7-7b867c8bd7da', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.130', '--skip-admin-label'] 2023-09-12T15:54:53.122 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stdout 4.1.1 2023-09-12T15:54:53.123 INFO:teuthology.orchestra.run.smithi130.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-09-12T15:54:53.123 INFO:teuthology.orchestra.run.smithi130.stdout:Verifying podman|docker is present... 2023-09-12T15:54:53.266 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stdout 4.1.1 2023-09-12T15:54:53.267 INFO:teuthology.orchestra.run.smithi130.stdout:Verifying lvm2 is present... 2023-09-12T15:54:53.267 INFO:teuthology.orchestra.run.smithi130.stdout:Verifying time synchronization is in place... 2023-09-12T15:54:53.273 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-12T15:54:53.274 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-12T15:54:53.279 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-12T15:54:53.280 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout inactive 2023-09-12T15:54:53.285 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout enabled 2023-09-12T15:54:53.291 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout active 2023-09-12T15:54:53.291 INFO:teuthology.orchestra.run.smithi130.stdout:Unit chronyd.service is enabled and running 2023-09-12T15:54:53.292 INFO:teuthology.orchestra.run.smithi130.stdout:Repeating the final host check... 2023-09-12T15:54:53.434 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stdout 4.1.1 2023-09-12T15:54:53.435 INFO:teuthology.orchestra.run.smithi130.stdout:podman (/bin/podman) version 4.1.1 is present 2023-09-12T15:54:53.435 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl is present 2023-09-12T15:54:53.435 INFO:teuthology.orchestra.run.smithi130.stdout:lvcreate is present 2023-09-12T15:54:53.440 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-12T15:54:53.441 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-12T15:54:53.446 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-12T15:54:53.446 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout inactive 2023-09-12T15:54:53.452 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout enabled 2023-09-12T15:54:53.458 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout active 2023-09-12T15:54:53.458 INFO:teuthology.orchestra.run.smithi130.stdout:Unit chronyd.service is enabled and running 2023-09-12T15:54:53.458 INFO:teuthology.orchestra.run.smithi130.stdout:Host looks OK 2023-09-12T15:54:53.458 INFO:teuthology.orchestra.run.smithi130.stdout:Cluster fsid: 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:54:53.459 INFO:teuthology.orchestra.run.smithi130.stdout:Acquiring lock 140237696373760 on /run/cephadm/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.lock 2023-09-12T15:54:53.459 INFO:teuthology.orchestra.run.smithi130.stdout:Lock 140237696373760 acquired on /run/cephadm/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.lock 2023-09-12T15:54:53.459 INFO:teuthology.orchestra.run.smithi130.stdout:Verifying IP 172.21.15.130 port 3300 ... 2023-09-12T15:54:53.460 INFO:teuthology.orchestra.run.smithi130.stdout:Verifying IP 172.21.15.130 port 6789 ... 2023-09-12T15:54:53.460 INFO:teuthology.orchestra.run.smithi130.stdout:Base mon IP(s) is [172.21.15.130:3300, 172.21.15.130:6789], mon addrv is [v2:172.21.15.130:3300,v1:172.21.15.130:6789] 2023-09-12T15:54:53.462 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-09-12T15:54:53.462 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.130 metric 100 2023-09-12T15:54:53.465 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-09-12T15:54:53.465 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-09-12T15:54:53.465 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-09-12T15:54:53.467 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-09-12T15:54:53.468 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-09-12T15:54:53.468 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-12T15:54:53.468 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-09-12T15:54:53.468 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout inet6 fe80::13c4:484c:d749:6616/64 scope link noprefixroute 2023-09-12T15:54:53.468 INFO:teuthology.orchestra.run.smithi130.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-12T15:54:53.469 INFO:teuthology.orchestra.run.smithi130.stdout:Mon IP `172.21.15.130` is in CIDR network `172.21.0.0/20` 2023-09-12T15:54:53.469 INFO:teuthology.orchestra.run.smithi130.stdout:Mon IP `172.21.15.130` is in CIDR network `172.21.0.0/20` 2023-09-12T15:54:53.470 INFO:teuthology.orchestra.run.smithi130.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-09-12T15:54:53.470 INFO:teuthology.orchestra.run.smithi130.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-09-12T15:54:53.470 INFO:teuthology.orchestra.run.smithi130.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-12T15:54:53.813 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stdout bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed 2023-09-12T15:54:53.813 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-12T15:54:53.814 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stderr Getting image source signatures 2023-09-12T15:54:53.814 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stderr Copying blob sha256:d0e1bfbb8440979edf72f0bc146ed182f1efda6b57c3af4c34dad6d07bc7592b 2023-09-12T15:54:53.814 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stderr Copying blob sha256:aba40bcce7ff9e06f5f6d056b94d79b3c5f9fa5aba1a4d6d95b68b2cadf32b62 2023-09-12T15:54:53.814 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stderr Copying config sha256:bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed 2023-09-12T15:54:53.814 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stderr Writing manifest to image destination 2023-09-12T15:54:53.815 INFO:teuthology.orchestra.run.smithi130.stdout:/bin/podman: stderr Storing signatures 2023-09-12T15:54:54.661 INFO:teuthology.orchestra.run.smithi130.stdout:ceph: stdout ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev) 2023-09-12T15:54:54.661 INFO:teuthology.orchestra.run.smithi130.stdout:Ceph version: ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev) 2023-09-12T15:54:54.661 INFO:teuthology.orchestra.run.smithi130.stdout:Extracting ceph user uid/gid from container image... 2023-09-12T15:54:55.414 INFO:teuthology.orchestra.run.smithi130.stdout:stat: stdout 167 167 2023-09-12T15:54:55.415 INFO:teuthology.orchestra.run.smithi130.stdout:Creating initial keys... 2023-09-12T15:54:56.165 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph-authtool: stdout AQDPiQBl2CKPLhAAmbOZn4agBQaB2l6Km+2i3Q== 2023-09-12T15:54:57.923 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph-authtool: stdout AQDQiQBlcKDWIBAAvOmRC7o16jplTiuBE+xyAw== 2023-09-12T15:54:59.684 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph-authtool: stdout AQDTiQBlhTn+FBAAtDIyx70PK/EerVR/I+Vk6Q== 2023-09-12T15:54:59.685 INFO:teuthology.orchestra.run.smithi130.stdout:Creating initial monmap... 2023-09-12T15:55:00.439 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-12T15:55:00.439 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-09-12T15:55:00.439 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:00.440 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-12T15:55:00.440 INFO:teuthology.orchestra.run.smithi130.stdout:monmaptool for a [v2:172.21.15.130:3300,v1:172.21.15.130:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-12T15:55:00.440 INFO:teuthology.orchestra.run.smithi130.stdout:setting min_mon_release = pacific 2023-09-12T15:55:00.440 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/monmaptool: set fsid to 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:00.440 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-12T15:55:00.441 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:00.441 INFO:teuthology.orchestra.run.smithi130.stdout:Creating mon... 2023-09-12T15:55:01.190 INFO:teuthology.orchestra.run.smithi130.stdout:create mon.a on 2023-09-12T15:55:01.314 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-09-12T15:55:01.399 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-09-12T15:55:01.515 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.target → /etc/systemd/system/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.target. 2023-09-12T15:55:01.515 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.target → /etc/systemd/system/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.target. 2023-09-12T15:55:01.702 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.a 2023-09-12T15:55:01.702 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.a.service: Unit ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.a.service not loaded. 2023-09-12T15:55:01.787 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.target.wants/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.a.service → /etc/systemd/system/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@.service. 2023-09-12T15:55:02.060 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:01 smithi130 systemd[1]: Starting Ceph mon.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T15:55:02.290 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-12T15:55:02.291 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout disabled 2023-09-12T15:55:02.298 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-12T15:55:02.298 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout inactive 2023-09-12T15:55:02.298 INFO:teuthology.orchestra.run.smithi130.stdout:firewalld.service is not enabled 2023-09-12T15:55:02.299 INFO:teuthology.orchestra.run.smithi130.stdout:Not possible to enable service . firewalld.service is not available 2023-09-12T15:55:02.299 INFO:teuthology.orchestra.run.smithi130.stdout:Waiting for mon to start... 2023-09-12T15:55:02.299 INFO:teuthology.orchestra.run.smithi130.stdout:Waiting for mon... 2023-09-12T15:55:02.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:02 smithi130 podman[98136]: 2023-09-12T15:55:02.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:02 smithi130 bash[98136]: 21a7d5d7e20d6778a3b4e61322177fb09ef777ef48153dfe8e0d1c62d11e4632 2023-09-12T15:55:02.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:02 smithi130 systemd[1]: Started Ceph mon.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T15:55:02.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:02 smithi130 ceph-mon[98176]: mkfs 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:02.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:02 smithi130 ceph-mon[98176]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-12T15:55:03.146 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout cluster: 2023-09-12T15:55:03.147 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout id: 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:03.147 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-09-12T15:55:03.147 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:03.148 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout services: 2023-09-12T15:55:03.148 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.552175s) 2023-09-12T15:55:03.148 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-09-12T15:55:03.148 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-09-12T15:55:03.148 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:03.149 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout data: 2023-09-12T15:55:03.149 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-09-12T15:55:03.149 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-09-12T15:55:03.149 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-09-12T15:55:03.149 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout pgs: 2023-09-12T15:55:03.149 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:03.150 INFO:teuthology.orchestra.run.smithi130.stdout:mon is available 2023-09-12T15:55:03.150 INFO:teuthology.orchestra.run.smithi130.stdout:Assimilating anything we can from ceph.conf... 2023-09-12T15:55:03.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:03 smithi130 ceph-mon[98176]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-12T15:55:03.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:03 smithi130 ceph-mon[98176]: monmap e1: 1 mons at {a=[v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0]} removed_ranks: {} 2023-09-12T15:55:03.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:03 smithi130 ceph-mon[98176]: fsmap 2023-09-12T15:55:03.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:03 smithi130 ceph-mon[98176]: osdmap e1: 0 total, 0 up, 0 in 2023-09-12T15:55:03.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:03 smithi130 ceph-mon[98176]: mgrmap e1: no daemons active 2023-09-12T15:55:03.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:03 smithi130 ceph-mon[98176]: from='client.? 172.21.15.130:0/464206966' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-09-12T15:55:04.088 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:04.088 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout [global] 2023-09-12T15:55:04.088 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout fsid = 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:04.088 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.130:3300,v1:172.21.15.130:6789] 2023-09-12T15:55:04.089 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-12T15:55:04.089 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-12T15:55:04.089 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-12T15:55:04.089 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-12T15:55:04.089 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:04.090 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-12T15:55:04.090 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-12T15:55:04.090 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:04.090 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout [osd] 2023-09-12T15:55:04.090 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-12T15:55:04.090 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-12T15:55:04.091 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-12T15:55:04.091 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-12T15:55:04.091 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-12T15:55:04.091 INFO:teuthology.orchestra.run.smithi130.stdout:Generating new minimal ceph.conf... 2023-09-12T15:55:04.645 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:04 smithi130 ceph-mon[98176]: from='client.? 172.21.15.130:0/4213888760' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-12T15:55:04.645 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:04 smithi130 ceph-mon[98176]: from='client.? 172.21.15.130:0/4213888760' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-09-12T15:55:04.929 INFO:teuthology.orchestra.run.smithi130.stdout:Restarting the monitor... 2023-09-12T15:55:05.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:04 smithi130 systemd[1]: Stopping Ceph mon.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T15:55:05.555 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:05 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-a[98152]: 2023-09-12T15:55:05.259+0000 7fd596dc9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T15:55:05.555 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:05 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-a[98152]: 2023-09-12T15:55:05.259+0000 7fd596dc9700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-12T15:55:05.809 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:05 smithi130 bash[98510]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-a 2023-09-12T15:55:06.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 systemd[1]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.a.service: Succeeded. 2023-09-12T15:55:06.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 systemd[1]: Stopped Ceph mon.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T15:55:06.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 systemd[1]: Starting Ceph mon.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T15:55:06.495 INFO:teuthology.orchestra.run.smithi130.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-09-12T15:55:06.569 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 podman[98630]: 2023-09-12T15:55:06.569 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 bash[98630]: 2e27805169023e1ad81b310b56982711bf687be603c9ea153f65ae716f8b3736 2023-09-12T15:55:06.570 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: set uid:gid to 167:167 (ceph:ceph) 2023-09-12T15:55:06.570 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev), process ceph-mon, pid 7 2023-09-12T15:55:06.570 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: pidfile_write: ignore empty --pid-file 2023-09-12T15:55:06.570 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: load: jerasure load: lrc 2023-09-12T15:55:06.570 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 systemd[1]: Started Ceph mon.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T15:55:06.571 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: RocksDB version: 7.9.2 2023-09-12T15:55:06.571 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Git sha 0 2023-09-12T15:55:06.571 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Compile date 2023-09-12 13:00:11 2023-09-12T15:55:06.571 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: DB SUMMARY 2023-09-12T15:55:06.571 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: DB Session ID: D8PE9X171TBY49XN57OV 2023-09-12T15:55:06.572 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: CURRENT file: CURRENT 2023-09-12T15:55:06.572 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: IDENTITY file: IDENTITY 2023-09-12T15:55:06.572 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-09-12T15:55:06.572 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-09-12T15:55:06.572 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 87882 ; 2023-09-12T15:55:06.572 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.error_if_exists: 0 2023-09-12T15:55:06.573 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.create_if_missing: 0 2023-09-12T15:55:06.573 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.paranoid_checks: 1 2023-09-12T15:55:06.573 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-12T15:55:06.573 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-12T15:55:06.573 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-12T15:55:06.573 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.env: 0x5638767d2da0 2023-09-12T15:55:06.574 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.fs: PosixFileSystem 2023-09-12T15:55:06.574 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.info_log: 0x56387756b4e0 2023-09-12T15:55:06.574 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_file_opening_threads: 16 2023-09-12T15:55:06.574 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.statistics: (nil) 2023-09-12T15:55:06.574 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.use_fsync: 0 2023-09-12T15:55:06.575 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_log_file_size: 0 2023-09-12T15:55:06.575 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-12T15:55:06.575 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-12T15:55:06.575 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.keep_log_file_num: 1000 2023-09-12T15:55:06.575 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.recycle_log_file_num: 0 2023-09-12T15:55:06.575 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.allow_fallocate: 1 2023-09-12T15:55:06.576 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.allow_mmap_reads: 0 2023-09-12T15:55:06.576 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.allow_mmap_writes: 0 2023-09-12T15:55:06.576 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.use_direct_reads: 0 2023-09-12T15:55:06.576 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-12T15:55:06.576 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.create_missing_column_families: 0 2023-09-12T15:55:06.577 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.db_log_dir: 2023-09-12T15:55:06.577 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.wal_dir: 2023-09-12T15:55:06.577 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-12T15:55:06.577 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-12T15:55:06.577 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-12T15:55:06.577 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-12T15:55:06.578 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-12T15:55:06.578 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-12T15:55:06.578 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.advise_random_on_open: 1 2023-09-12T15:55:06.578 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.db_write_buffer_size: 0 2023-09-12T15:55:06.578 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.write_buffer_manager: 0x5638775254a0 2023-09-12T15:55:06.579 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-12T15:55:06.579 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-12T15:55:06.579 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-12T15:55:06.579 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.rate_limiter: (nil) 2023-09-12T15:55:06.579 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-12T15:55:06.579 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.wal_recovery_mode: 2 2023-09-12T15:55:06.580 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.enable_thread_tracking: 0 2023-09-12T15:55:06.580 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.enable_pipelined_write: 0 2023-09-12T15:55:06.580 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.unordered_write: 0 2023-09-12T15:55:06.580 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-12T15:55:06.580 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-12T15:55:06.581 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-12T15:55:06.581 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-12T15:55:06.581 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.row_cache: None 2023-09-12T15:55:06.581 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.wal_filter: None 2023-09-12T15:55:06.581 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-12T15:55:06.582 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.allow_ingest_behind: 0 2023-09-12T15:55:06.582 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.two_write_queues: 0 2023-09-12T15:55:06.582 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.manual_wal_flush: 0 2023-09-12T15:55:06.582 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.wal_compression: 0 2023-09-12T15:55:06.582 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.atomic_flush: 0 2023-09-12T15:55:06.582 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-12T15:55:06.583 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-12T15:55:06.583 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-12T15:55:06.583 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.log_readahead_size: 0 2023-09-12T15:55:06.583 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-12T15:55:06.583 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.best_efforts_recovery: 0 2023-09-12T15:55:06.584 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-12T15:55:06.584 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-12T15:55:06.585 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.allow_data_in_errors: 0 2023-09-12T15:55:06.585 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.db_host_id: __hostname__ 2023-09-12T15:55:06.586 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-12T15:55:06.586 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_background_jobs: 2 2023-09-12T15:55:06.586 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_background_compactions: -1 2023-09-12T15:55:06.586 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_subcompactions: 1 2023-09-12T15:55:06.586 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-12T15:55:06.587 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-12T15:55:06.587 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-12T15:55:06.587 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_total_wal_size: 0 2023-09-12T15:55:06.587 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-12T15:55:06.587 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-12T15:55:06.588 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-12T15:55:06.588 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-12T15:55:06.588 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_open_files: -1 2023-09-12T15:55:06.588 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bytes_per_sync: 0 2023-09-12T15:55:06.588 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-12T15:55:06.589 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-12T15:55:06.589 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_readahead_size: 0 2023-09-12T15:55:06.589 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_background_flushes: -1 2023-09-12T15:55:06.589 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Compression algorithms supported: 2023-09-12T15:55:06.589 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-12T15:55:06.590 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: kZSTD supported: 0 2023-09-12T15:55:06.590 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: kXpressCompression supported: 0 2023-09-12T15:55:06.590 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: kLZ4HCCompression supported: 1 2023-09-12T15:55:06.590 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: kZlibCompression supported: 1 2023-09-12T15:55:06.590 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: kSnappyCompression supported: 1 2023-09-12T15:55:06.591 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: kLZ4Compression supported: 1 2023-09-12T15:55:06.591 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: kBZip2Compression supported: 0 2023-09-12T15:55:06.591 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-12T15:55:06.591 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-12T15:55:06.591 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-09-12T15:55:06.591 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-12T15:55:06.592 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-12T15:55:06.592 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.merge_operator: 2023-09-12T15:55:06.592 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_filter: None 2023-09-12T15:55:06.592 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_filter_factory: None 2023-09-12T15:55:06.592 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.sst_partitioner_factory: None 2023-09-12T15:55:06.593 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-12T15:55:06.593 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-12T15:55:06.593 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56387756b5e0) 2023-09-12T15:55:06.593 INFO:journalctl@ceph.mon.a.smithi130.stdout: cache_index_and_filter_blocks: 1 2023-09-12T15:55:06.593 INFO:journalctl@ceph.mon.a.smithi130.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-12T15:55:06.593 INFO:journalctl@ceph.mon.a.smithi130.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-12T15:55:06.594 INFO:journalctl@ceph.mon.a.smithi130.stdout: pin_top_level_index_and_filter: 1 2023-09-12T15:55:06.594 INFO:journalctl@ceph.mon.a.smithi130.stdout: index_type: 0 2023-09-12T15:55:06.594 INFO:journalctl@ceph.mon.a.smithi130.stdout: data_block_index_type: 0 2023-09-12T15:55:06.594 INFO:journalctl@ceph.mon.a.smithi130.stdout: index_shortening: 1 2023-09-12T15:55:06.594 INFO:journalctl@ceph.mon.a.smithi130.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-12T15:55:06.595 INFO:journalctl@ceph.mon.a.smithi130.stdout: checksum: 4 2023-09-12T15:55:06.595 INFO:journalctl@ceph.mon.a.smithi130.stdout: no_block_cache: 0 2023-09-12T15:55:06.595 INFO:journalctl@ceph.mon.a.smithi130.stdout: block_cache: 0x5638775c31f0 2023-09-12T15:55:06.595 INFO:journalctl@ceph.mon.a.smithi130.stdout: block_cache_name: BinnedLRUCache 2023-09-12T15:55:06.595 INFO:journalctl@ceph.mon.a.smithi130.stdout: block_cache_options: 2023-09-12T15:55:06.595 INFO:journalctl@ceph.mon.a.smithi130.stdout: capacity : 536870912 2023-09-12T15:55:06.596 INFO:journalctl@ceph.mon.a.smithi130.stdout: num_shard_bits : 4 2023-09-12T15:55:06.596 INFO:journalctl@ceph.mon.a.smithi130.stdout: strict_capacity_limit : 0 2023-09-12T15:55:06.596 INFO:journalctl@ceph.mon.a.smithi130.stdout: high_pri_pool_ratio: 0.000 2023-09-12T15:55:06.596 INFO:journalctl@ceph.mon.a.smithi130.stdout: block_cache_compressed: (nil) 2023-09-12T15:55:06.596 INFO:journalctl@ceph.mon.a.smithi130.stdout: persistent_cache: (nil) 2023-09-12T15:55:06.596 INFO:journalctl@ceph.mon.a.smithi130.stdout: block_size: 4096 2023-09-12T15:55:06.597 INFO:journalctl@ceph.mon.a.smithi130.stdout: block_size_deviation: 10 2023-09-12T15:55:06.597 INFO:journalctl@ceph.mon.a.smithi130.stdout: block_restart_interval: 16 2023-09-12T15:55:06.597 INFO:journalctl@ceph.mon.a.smithi130.stdout: index_block_restart_interval: 1 2023-09-12T15:55:06.597 INFO:journalctl@ceph.mon.a.smithi130.stdout: metadata_block_size: 4096 2023-09-12T15:55:06.597 INFO:journalctl@ceph.mon.a.smithi130.stdout: partition_filters: 0 2023-09-12T15:55:06.597 INFO:journalctl@ceph.mon.a.smithi130.stdout: use_delta_encoding: 1 2023-09-12T15:55:06.598 INFO:journalctl@ceph.mon.a.smithi130.stdout: filter_policy: bloomfilter 2023-09-12T15:55:06.598 INFO:journalctl@ceph.mon.a.smithi130.stdout: whole_key_filtering: 1 2023-09-12T15:55:06.598 INFO:journalctl@ceph.mon.a.smithi130.stdout: verify_compression: 0 2023-09-12T15:55:06.598 INFO:journalctl@ceph.mon.a.smithi130.stdout: read_amp_bytes_per_bit: 0 2023-09-12T15:55:06.598 INFO:journalctl@ceph.mon.a.smithi130.stdout: format_version: 5 2023-09-12T15:55:06.599 INFO:journalctl@ceph.mon.a.smithi130.stdout: enable_index_compression: 1 2023-09-12T15:55:06.599 INFO:journalctl@ceph.mon.a.smithi130.stdout: block_align: 0 2023-09-12T15:55:06.599 INFO:journalctl@ceph.mon.a.smithi130.stdout: max_auto_readahead_size: 262144 2023-09-12T15:55:06.599 INFO:journalctl@ceph.mon.a.smithi130.stdout: prepopulate_block_cache: 0 2023-09-12T15:55:06.599 INFO:journalctl@ceph.mon.a.smithi130.stdout: initial_auto_readahead_size: 8192 2023-09-12T15:55:06.599 INFO:journalctl@ceph.mon.a.smithi130.stdout: num_file_reads_for_auto_readahead: 2 2023-09-12T15:55:06.600 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.write_buffer_size: 33554432 2023-09-12T15:55:06.600 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_write_buffer_number: 2 2023-09-12T15:55:06.600 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression: NoCompression 2023-09-12T15:55:06.600 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression: Disabled 2023-09-12T15:55:06.600 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.prefix_extractor: nullptr 2023-09-12T15:55:06.600 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-12T15:55:06.601 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.num_levels: 7 2023-09-12T15:55:06.601 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-12T15:55:06.601 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-12T15:55:06.601 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-12T15:55:06.601 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-12T15:55:06.602 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-12T15:55:06.602 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-12T15:55:06.602 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-12T15:55:06.602 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-12T15:55:06.602 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-12T15:55:06.603 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-12T15:55:06.603 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-12T15:55:06.603 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-12T15:55:06.603 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-12T15:55:06.603 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression_opts.level: 32767 2023-09-12T15:55:06.603 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression_opts.strategy: 0 2023-09-12T15:55:06.603 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-12T15:55:06.604 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-12T15:55:06.604 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-12T15:55:06.604 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-12T15:55:06.604 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression_opts.enabled: false 2023-09-12T15:55:06.604 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-12T15:55:06.604 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-12T15:55:06.605 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-12T15:55:06.605 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-12T15:55:06.605 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.target_file_size_base: 67108864 2023-09-12T15:55:06.605 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-12T15:55:06.605 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-12T15:55:06.605 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-12T15:55:06.606 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-12T15:55:06.606 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-12T15:55:06.606 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-12T15:55:06.606 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-12T15:55:06.606 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-12T15:55:06.606 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-12T15:55:06.607 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-12T15:55:06.607 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-12T15:55:06.607 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-12T15:55:06.607 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-12T15:55:06.607 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-12T15:55:06.607 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.arena_block_size: 1048576 2023-09-12T15:55:06.608 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-12T15:55:06.608 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-12T15:55:06.608 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.disable_auto_compactions: 0 2023-09-12T15:55:06.608 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-12T15:55:06.608 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-12T15:55:06.608 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-12T15:55:06.609 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-12T15:55:06.609 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-12T15:55:06.609 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-12T15:55:06.609 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-12T15:55:06.609 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-12T15:55:06.609 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-12T15:55:06.610 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-12T15:55:06.610 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.table_properties_collectors: 2023-09-12T15:55:06.610 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.inplace_update_support: 0 2023-09-12T15:55:06.610 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-12T15:55:06.610 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-12T15:55:06.611 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-12T15:55:06.611 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-12T15:55:06.611 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.bloom_locality: 0 2023-09-12T15:55:06.611 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.max_successive_merges: 0 2023-09-12T15:55:06.611 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-12T15:55:06.611 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.paranoid_file_checks: 0 2023-09-12T15:55:06.612 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.force_consistency_checks: 1 2023-09-12T15:55:06.612 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.report_bg_io_stats: 0 2023-09-12T15:55:06.612 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.ttl: 2592000 2023-09-12T15:55:06.612 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-12T15:55:06.612 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-12T15:55:06.612 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-12T15:55:06.613 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.enable_blob_files: false 2023-09-12T15:55:06.613 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.min_blob_size: 0 2023-09-12T15:55:06.613 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.blob_file_size: 268435456 2023-09-12T15:55:06.613 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-12T15:55:06.613 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-12T15:55:06.613 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-12T15:55:06.614 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-12T15:55:06.614 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-12T15:55:06.614 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.blob_file_starting_level: 0 2023-09-12T15:55:06.614 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-12T15:55:06.614 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-09-12T15:55:06.615 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-09-12T15:55:06.615 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6b493530-2d6b-4fa9-b3bf-3a1027036d57 2023-09-12T15:55:06.615 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694534106494630, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-09-12T15:55:06.615 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-09-12T15:55:06.615 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694534106495496, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 84200, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 265, "table_properties": {"data_size": 82367, "index_size": 243, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 11175, "raw_average_key_size": 49, "raw_value_size": 76095, "raw_average_value_size": 333, "num_data_blocks": 10, "num_entries": 228, "num_filter_entries": 228, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694534106, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6b493530-2d6b-4fa9-b3bf-3a1027036d57", "db_session_id": "D8PE9X171TBY49XN57OV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-09-12T15:55:06.615 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694534106495570, "job": 1, "event": "recovery_finished"} 2023-09-12T15:55:06.616 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-09-12T15:55:06.616 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-12T15:55:06.616 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56387765e000 2023-09-12T15:55:06.616 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: DB pointer 0x56387764a000 2023-09-12T15:55:06.616 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-12T15:55:06.616 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-12T15:55:06.617 INFO:journalctl@ceph.mon.a.smithi130.stdout: ** DB Stats ** 2023-09-12T15:55:06.617 INFO:journalctl@ceph.mon.a.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-12T15:55:06.617 INFO:journalctl@ceph.mon.a.smithi130.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-12T15:55:06.617 INFO:journalctl@ceph.mon.a.smithi130.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-12T15:55:06.617 INFO:journalctl@ceph.mon.a.smithi130.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-12T15:55:06.617 INFO:journalctl@ceph.mon.a.smithi130.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-12T15:55:06.618 INFO:journalctl@ceph.mon.a.smithi130.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-12T15:55:06.618 INFO:journalctl@ceph.mon.a.smithi130.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-12T15:55:06.618 INFO:journalctl@ceph.mon.a.smithi130.stdout: 2023-09-12T15:55:06.618 INFO:journalctl@ceph.mon.a.smithi130.stdout: ** Compaction Stats [default] ** 2023-09-12T15:55:06.618 INFO:journalctl@ceph.mon.a.smithi130.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-12T15:55:06.619 INFO:journalctl@ceph.mon.a.smithi130.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-12T15:55:06.619 INFO:journalctl@ceph.mon.a.smithi130.stdout: L0 2/0 84.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-12T15:55:06.619 INFO:journalctl@ceph.mon.a.smithi130.stdout: Sum 2/0 84.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-12T15:55:06.619 INFO:journalctl@ceph.mon.a.smithi130.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-12T15:55:06.619 INFO:journalctl@ceph.mon.a.smithi130.stdout: 2023-09-12T15:55:06.619 INFO:journalctl@ceph.mon.a.smithi130.stdout: ** Compaction Stats [default] ** 2023-09-12T15:55:06.620 INFO:journalctl@ceph.mon.a.smithi130.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-12T15:55:06.620 INFO:journalctl@ceph.mon.a.smithi130.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-12T15:55:06.620 INFO:journalctl@ceph.mon.a.smithi130.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-12T15:55:06.620 INFO:journalctl@ceph.mon.a.smithi130.stdout: 2023-09-12T15:55:06.620 INFO:journalctl@ceph.mon.a.smithi130.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-12T15:55:06.621 INFO:journalctl@ceph.mon.a.smithi130.stdout: 2023-09-12T15:55:06.621 INFO:journalctl@ceph.mon.a.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-12T15:55:06.621 INFO:journalctl@ceph.mon.a.smithi130.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-12T15:55:06.621 INFO:journalctl@ceph.mon.a.smithi130.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-12T15:55:06.621 INFO:journalctl@ceph.mon.a.smithi130.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-12T15:55:06.621 INFO:journalctl@ceph.mon.a.smithi130.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-12T15:55:06.622 INFO:journalctl@ceph.mon.a.smithi130.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-12T15:55:06.622 INFO:journalctl@ceph.mon.a.smithi130.stdout: Cumulative compaction: 0.00 GB write, 28.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-12T15:55:06.622 INFO:journalctl@ceph.mon.a.smithi130.stdout: Interval compaction: 0.00 GB write, 28.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-12T15:55:06.622 INFO:journalctl@ceph.mon.a.smithi130.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-12T15:55:06.622 INFO:journalctl@ceph.mon.a.smithi130.stdout: Block cache BinnedLRUCache@0x5638775c31f0#7 capacity: 512.00 MB usage: 1.20 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-12T15:55:06.623 INFO:journalctl@ceph.mon.a.smithi130.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2023-09-12T15:55:06.623 INFO:journalctl@ceph.mon.a.smithi130.stdout: 2023-09-12T15:55:06.623 INFO:journalctl@ceph.mon.a.smithi130.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-12T15:55:06.623 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: starting mon.a rank 0 at public addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] at bind addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:06.623 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mon.a@-1(???) e1 preinit fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:06.623 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mon.a@-1(???).mds e1 new map 2023-09-12T15:55:06.624 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mon.a@-1(???).mds e1 print_map 2023-09-12T15:55:06.624 INFO:journalctl@ceph.mon.a.smithi130.stdout: e1 2023-09-12T15:55:06.624 INFO:journalctl@ceph.mon.a.smithi130.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-12T15:55:06.624 INFO:journalctl@ceph.mon.a.smithi130.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-12T15:55:06.624 INFO:journalctl@ceph.mon.a.smithi130.stdout: legacy client fscid: -1 2023-09-12T15:55:06.625 INFO:journalctl@ceph.mon.a.smithi130.stdout: 2023-09-12T15:55:06.625 INFO:journalctl@ceph.mon.a.smithi130.stdout: No filesystems configured 2023-09-12T15:55:06.625 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-12T15:55:06.625 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-12T15:55:06.625 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-12T15:55:06.625 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-12T15:55:06.626 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-09-12T15:55:06.626 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta expand map: {default=false} 2023-09-12T15:55:06.626 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta from 'false' to 'false' 2023-09-12T15:55:06.626 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta expanded map: {default=false} 2023-09-12T15:55:06.626 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta expand map: {default=info} 2023-09-12T15:55:06.626 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta from 'info' to 'info' 2023-09-12T15:55:06.627 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta expanded map: {default=info} 2023-09-12T15:55:06.627 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta expand map: {default=daemon} 2023-09-12T15:55:06.627 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-12T15:55:06.627 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta expanded map: {default=daemon} 2023-09-12T15:55:06.627 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta expand map: {default=debug} 2023-09-12T15:55:06.627 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta from 'debug' to 'debug' 2023-09-12T15:55:06.627 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: expand_channel_meta expanded map: {default=debug} 2023-09-12T15:55:06.628 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-12T15:55:06.628 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: monmap e1: 1 mons at {a=[v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0]} removed_ranks: {} 2023-09-12T15:55:06.628 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: fsmap 2023-09-12T15:55:06.628 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: osdmap e1: 0 total, 0 up, 0 in 2023-09-12T15:55:06.628 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:06 smithi130 ceph-mon[98670]: mgrmap e1: no daemons active 2023-09-12T15:55:07.443 INFO:teuthology.orchestra.run.smithi130.stdout:Wrote config to /etc/ceph/ceph.conf 2023-09-12T15:55:07.489 INFO:teuthology.orchestra.run.smithi130.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-09-12T15:55:07.489 INFO:teuthology.orchestra.run.smithi130.stdout:Creating mgr... 2023-09-12T15:55:07.490 INFO:teuthology.orchestra.run.smithi130.stdout:Verifying port 0.0.0.0:9283 ... 2023-09-12T15:55:07.490 INFO:teuthology.orchestra.run.smithi130.stdout:Verifying port 0.0.0.0:8765 ... 2023-09-12T15:55:07.687 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.x 2023-09-12T15:55:07.687 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.x.service: Unit ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.x.service not loaded. 2023-09-12T15:55:07.773 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.target.wants/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.x.service → /etc/systemd/system/ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@.service. 2023-09-12T15:55:07.974 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:07 smithi130 systemd[1]: Starting Ceph mgr.x for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T15:55:08.364 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:08 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3845023406' entity='client.admin' 2023-09-12T15:55:08.365 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:08 smithi130 podman[98897]: 2023-09-12T15:55:08.365 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:08 smithi130 bash[98897]: 7d7d2256e25cd58f572bd3fca84c85c58d0de3c3ac2696f5af9cf1d6cdfc1271 2023-09-12T15:55:08.376 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-12T15:55:08.377 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout disabled 2023-09-12T15:55:08.384 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-12T15:55:08.384 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout inactive 2023-09-12T15:55:08.384 INFO:teuthology.orchestra.run.smithi130.stdout:firewalld.service is not enabled 2023-09-12T15:55:08.385 INFO:teuthology.orchestra.run.smithi130.stdout:Not possible to enable service . firewalld.service is not available 2023-09-12T15:55:08.393 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-12T15:55:08.393 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout disabled 2023-09-12T15:55:08.400 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-12T15:55:08.400 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout inactive 2023-09-12T15:55:08.400 INFO:teuthology.orchestra.run.smithi130.stdout:firewalld.service is not enabled 2023-09-12T15:55:08.401 INFO:teuthology.orchestra.run.smithi130.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-09-12T15:55:08.401 INFO:teuthology.orchestra.run.smithi130.stdout:Waiting for mgr to start... 2023-09-12T15:55:08.401 INFO:teuthology.orchestra.run.smithi130.stdout:Waiting for mgr... 2023-09-12T15:55:08.624 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:08 smithi130 systemd[1]: Started Ceph mgr.x for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T15:55:08.625 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:08.384+0000 7f826f37f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-12T15:55:08.625 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:08.473+0000 7f826f37f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-12T15:55:08.625 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:08.567+0000 7f826f37f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-12T15:55:08.972 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:08.668+0000 7f826f37f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-12T15:55:08.972 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:08.845+0000 7f826f37f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-12T15:55:09.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:09 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2257464252' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-12T15:55:09.249 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:09.252 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout { 2023-09-12T15:55:09.252 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T15:55:09.252 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "health": { 2023-09-12T15:55:09.252 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-12T15:55:09.253 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-12T15:55:09.253 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-12T15:55:09.253 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:09.253 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-12T15:55:09.253 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-12T15:55:09.253 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 0 2023-09-12T15:55:09.254 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ], 2023-09-12T15:55:09.254 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-12T15:55:09.254 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "a" 2023-09-12T15:55:09.254 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ], 2023-09-12T15:55:09.254 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-09-12T15:55:09.255 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-12T15:55:09.255 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:09.255 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-12T15:55:09.255 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-12T15:55:09.255 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:09.255 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-12T15:55:09.256 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:09.256 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-12T15:55:09.256 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-12T15:55:09.256 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-12T15:55:09.256 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-12T15:55:09.260 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-12T15:55:09.260 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-12T15:55:09.260 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:09.260 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-12T15:55:09.260 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-12T15:55:09.261 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-12T15:55:09.261 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-12T15:55:09.261 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-12T15:55:09.261 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-12T15:55:09.261 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-12T15:55:09.262 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-12T15:55:09.262 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-12T15:55:09.262 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:09.262 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-12T15:55:09.262 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:09.263 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-12T15:55:09.263 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-12T15:55:09.263 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:09.263 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-12T15:55:09.263 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-12T15:55:09.263 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-12T15:55:09.264 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-12T15:55:09.264 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-12T15:55:09.264 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-12T15:55:09.264 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "restful" 2023-09-12T15:55:09.264 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ], 2023-09-12T15:55:09.264 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-12T15:55:09.265 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:09.265 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-12T15:55:09.265 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:09.265 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "modified": "2023-09-12T15:55:02.305756+0000", 2023-09-12T15:55:09.265 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-12T15:55:09.265 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:09.266 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-12T15:55:09.266 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout } 2023-09-12T15:55:09.266 INFO:teuthology.orchestra.run.smithi130.stdout:mgr not available, waiting (1/15)... 2023-09-12T15:55:09.549 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:09.231+0000 7f826f37f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-12T15:55:09.550 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:09.365+0000 7f826f37f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-12T15:55:09.550 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:09.452+0000 7f826f37f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-12T15:55:09.971 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:09.549+0000 7f826f37f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-12T15:55:10.974 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:09.971+0000 7f826f37f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-12T15:55:10.975 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:10 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:10.643+0000 7f826f37f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-12T15:55:10.975 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:10 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:10.725+0000 7f826f37f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-12T15:55:11.777 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:11 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:11.509+0000 7f826f37f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:55:11.777 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:11 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:11.600+0000 7f826f37f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-12T15:55:12.061 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:11 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:11.776+0000 7f826f37f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:55:12.344 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:12 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:12.061+0000 7f826f37f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-12T15:55:12.344 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:12 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:12.292+0000 7f826f37f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-12T15:55:12.719 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:12 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:12.451+0000 7f826f37f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-12T15:55:12.974 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:12 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:12.718+0000 7f826f37f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-12T15:55:13.724 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:13 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2238192562' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-12T15:55:13.725 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:13 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:13.436+0000 7f826f37f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-12T15:55:13.725 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:13 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:13.566+0000 7f826f37f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-12T15:55:14.103 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:13 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:13.771+0000 7f826f37f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-12T15:55:14.103 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:13 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:13.884+0000 7f826f37f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-12T15:55:14.474 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:14 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:14.104+0000 7f826f37f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-12T15:55:15.225 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:14 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:14.972+0000 7f826f37f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-12T15:55:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: Activating manager daemon x 2023-09-12T15:55:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: mgrmap e2: x(active, starting, since 0.00302791s) 2023-09-12T15:55:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-12T15:55:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-12T15:55:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-12T15:55:15.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:55:15.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-12T15:55:15.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: Manager daemon x is now available 2023-09-12T15:55:15.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-12T15:55:15.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-12T15:55:15.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' 2023-09-12T15:55:15.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' 2023-09-12T15:55:15.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:15 smithi130 ceph-mon[98670]: from='mgr.14100 172.21.15.130:0/321538392' entity='mgr.x' 2023-09-12T15:55:15.708 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:15.714 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout { 2023-09-12T15:55:15.714 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T15:55:15.714 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "health": { 2023-09-12T15:55:15.714 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-12T15:55:15.714 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-12T15:55:15.715 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-12T15:55:15.715 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:15.715 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-12T15:55:15.715 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-12T15:55:15.715 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 0 2023-09-12T15:55:15.716 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ], 2023-09-12T15:55:15.716 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-12T15:55:15.716 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "a" 2023-09-12T15:55:15.716 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ], 2023-09-12T15:55:15.716 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-09-12T15:55:15.716 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-12T15:55:15.717 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:15.717 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-12T15:55:15.717 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-12T15:55:15.717 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:15.717 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-12T15:55:15.718 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:15.718 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-12T15:55:15.718 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-12T15:55:15.718 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-12T15:55:15.718 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-12T15:55:15.718 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-12T15:55:15.719 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-12T15:55:15.719 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:15.719 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-12T15:55:15.719 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-12T15:55:15.719 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-12T15:55:15.719 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-12T15:55:15.720 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-12T15:55:15.720 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-12T15:55:15.720 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-12T15:55:15.720 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-12T15:55:15.720 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-12T15:55:15.721 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:15.721 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-12T15:55:15.721 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:15.721 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-12T15:55:15.721 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-12T15:55:15.722 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:15.722 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-12T15:55:15.722 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-12T15:55:15.722 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-12T15:55:15.722 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-12T15:55:15.722 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-12T15:55:15.723 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-12T15:55:15.723 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "restful" 2023-09-12T15:55:15.723 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ], 2023-09-12T15:55:15.723 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-12T15:55:15.723 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:15.724 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-12T15:55:15.724 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:15.724 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "modified": "2023-09-12T15:55:02.305756+0000", 2023-09-12T15:55:15.724 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-12T15:55:15.724 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:15.724 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-12T15:55:15.725 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout } 2023-09-12T15:55:15.725 INFO:teuthology.orchestra.run.smithi130.stdout:mgr not available, waiting (2/15)... 2023-09-12T15:55:17.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:16 smithi130 ceph-mon[98670]: mgrmap e3: x(active, since 1.00702s) 2023-09-12T15:55:19.427 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:18 smithi130 ceph-mon[98670]: mgrmap e4: x(active, since 3s) 2023-09-12T15:55:19.469 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:19.469 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout { 2023-09-12T15:55:19.470 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T15:55:19.470 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "health": { 2023-09-12T15:55:19.470 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-12T15:55:19.470 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-12T15:55:19.470 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-12T15:55:19.471 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:19.471 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-12T15:55:19.471 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-12T15:55:19.471 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 0 2023-09-12T15:55:19.472 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ], 2023-09-12T15:55:19.472 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-12T15:55:19.472 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "a" 2023-09-12T15:55:19.472 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ], 2023-09-12T15:55:19.472 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-09-12T15:55:19.473 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-12T15:55:19.473 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:19.473 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-12T15:55:19.473 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-12T15:55:19.473 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:19.474 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-12T15:55:19.474 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:19.474 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-12T15:55:19.474 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-12T15:55:19.474 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-12T15:55:19.475 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-12T15:55:19.475 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-12T15:55:19.475 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-12T15:55:19.475 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:19.475 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-12T15:55:19.476 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-12T15:55:19.476 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-12T15:55:19.476 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-12T15:55:19.476 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-12T15:55:19.476 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-12T15:55:19.477 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-12T15:55:19.477 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-12T15:55:19.477 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-12T15:55:19.477 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:19.477 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-12T15:55:19.477 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:19.478 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-12T15:55:19.478 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-12T15:55:19.478 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:19.478 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-12T15:55:19.479 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-12T15:55:19.479 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-12T15:55:19.479 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-12T15:55:19.479 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-12T15:55:19.479 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-12T15:55:19.480 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "restful" 2023-09-12T15:55:19.480 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ], 2023-09-12T15:55:19.480 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-12T15:55:19.480 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:19.480 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-12T15:55:19.481 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-12T15:55:19.481 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "modified": "2023-09-12T15:55:02.305756+0000", 2023-09-12T15:55:19.481 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-12T15:55:19.481 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout }, 2023-09-12T15:55:19.481 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-12T15:55:19.482 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout } 2023-09-12T15:55:19.482 INFO:teuthology.orchestra.run.smithi130.stdout:mgr is available 2023-09-12T15:55:20.257 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:19 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1617898207' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-12T15:55:20.507 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:20.508 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout [global] 2023-09-12T15:55:20.508 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout fsid = 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:20.508 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-12T15:55:20.508 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-12T15:55:20.509 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-12T15:55:20.509 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-12T15:55:20.509 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:20.509 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-12T15:55:20.510 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-12T15:55:20.510 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 2023-09-12T15:55:20.510 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout [osd] 2023-09-12T15:55:20.510 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-12T15:55:20.510 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-12T15:55:20.511 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-12T15:55:20.511 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-12T15:55:20.511 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-12T15:55:20.511 INFO:teuthology.orchestra.run.smithi130.stdout:Enabling cephadm module... 2023-09-12T15:55:21.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:20 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1087942163' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-12T15:55:22.254 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:21 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/128229563' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-09-12T15:55:22.254 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: ignoring --setuser ceph since I am not root 2023-09-12T15:55:22.255 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: ignoring --setgroup ceph since I am not root 2023-09-12T15:55:22.255 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:22.173+0000 7fed9d181200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-12T15:55:22.255 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:22.254+0000 7fed9d181200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-12T15:55:22.513 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:22.346+0000 7fed9d181200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-12T15:55:22.514 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:22.449+0000 7fed9d181200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-12T15:55:22.974 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:22.629+0000 7fed9d181200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-12T15:55:23.337 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:22 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/128229563' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-09-12T15:55:23.337 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:22 smithi130 ceph-mon[98670]: mgrmap e5: x(active, since 7s) 2023-09-12T15:55:23.338 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:23.010+0000 7fed9d181200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-12T15:55:23.338 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:23.149+0000 7fed9d181200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-12T15:55:23.338 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:23.241+0000 7fed9d181200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-12T15:55:23.389 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout { 2023-09-12T15:55:23.390 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-09-12T15:55:23.390 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-12T15:55:23.390 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-12T15:55:23.390 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-12T15:55:23.390 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout } 2023-09-12T15:55:23.391 INFO:teuthology.orchestra.run.smithi130.stdout:Waiting for the mgr to restart... 2023-09-12T15:55:23.391 INFO:teuthology.orchestra.run.smithi130.stdout:Waiting for mgr epoch 5... 2023-09-12T15:55:23.594 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:23.337+0000 7fed9d181200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-12T15:55:23.974 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:23.781+0000 7fed9d181200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-12T15:55:24.441 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:23 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2103516819' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-12T15:55:24.724 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:24 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:24.441+0000 7fed9d181200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-12T15:55:24.725 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:24 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:24.524+0000 7fed9d181200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-12T15:55:25.554 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:25.291+0000 7fed9d181200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:55:25.555 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:25.382+0000 7fed9d181200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-12T15:55:25.832 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:25.555+0000 7fed9d181200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:55:26.213 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:25.832+0000 7fed9d181200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-12T15:55:26.213 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:26.066+0000 7fed9d181200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-12T15:55:26.213 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:26.213+0000 7fed9d181200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-12T15:55:26.974 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:26.476+0000 7fed9d181200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-12T15:55:27.474 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:27.176+0000 7fed9d181200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-12T15:55:27.475 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:27.306+0000 7fed9d181200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-12T15:55:27.826 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:27.498+0000 7fed9d181200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-12T15:55:27.827 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:27.610+0000 7fed9d181200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-12T15:55:27.827 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:27.826+0000 7fed9d181200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-12T15:55:28.974 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:28.715+0000 7fed9d181200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-12T15:55:28.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: Active manager daemon x restarted 2023-09-12T15:55:28.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: Activating manager daemon x 2023-09-12T15:55:28.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: osdmap e2: 0 total, 0 up, 0 in 2023-09-12T15:55:28.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: mgrmap e6: x(active, starting, since 0.00329413s) 2023-09-12T15:55:28.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:55:28.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-12T15:55:28.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-12T15:55:28.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-12T15:55:28.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-12T15:55:28.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: Manager daemon x is now available 2023-09-12T15:55:28.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:28.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:28.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:55:28.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:55:28.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:28 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:55:30.091 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:29 smithi130 ceph-mon[98670]: Found migration_current of "None". Setting to last migration. 2023-09-12T15:55:30.091 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:29 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-12T15:55:30.091 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:29 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-12T15:55:30.091 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:29 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:30.091 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:29 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:30.092 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:29 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:55:30.092 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:29 smithi130 ceph-mon[98670]: mgrmap e7: x(active, since 1.00549s) 2023-09-12T15:55:30.141 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout { 2023-09-12T15:55:30.142 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-09-12T15:55:30.142 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-12T15:55:30.142 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout } 2023-09-12T15:55:30.142 INFO:teuthology.orchestra.run.smithi130.stdout:mgr epoch 5 is available 2023-09-12T15:55:30.142 INFO:teuthology.orchestra.run.smithi130.stdout:Setting orchestrator backend to cephadm... 2023-09-12T15:55:31.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:30 smithi130 ceph-mon[98670]: [12/Sep/2023:15:55:29] ENGINE Bus STARTING 2023-09-12T15:55:31.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:30 smithi130 ceph-mon[98670]: [12/Sep/2023:15:55:29] ENGINE Serving on http://172.21.15.130:8765 2023-09-12T15:55:31.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:30 smithi130 ceph-mon[98670]: [12/Sep/2023:15:55:29] ENGINE Serving on https://172.21.15.130:7150 2023-09-12T15:55:31.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:30 smithi130 ceph-mon[98670]: [12/Sep/2023:15:55:29] ENGINE Bus STARTED 2023-09-12T15:55:31.162 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:30 smithi130 ceph-mon[98670]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-12T15:55:31.162 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:30 smithi130 ceph-mon[98670]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-12T15:55:32.119 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout value unchanged 2023-09-12T15:55:32.119 INFO:teuthology.orchestra.run.smithi130.stdout:Generating ssh key... 2023-09-12T15:55:32.214 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:31 smithi130 ceph-mon[98670]: mgrmap e8: x(active, since 2s) 2023-09-12T15:55:32.214 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:31 smithi130 ceph-mon[98670]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:55:32.215 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:31 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:32.215 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:31 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:55:33.076 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:32 smithi130 ceph-mon[98670]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:55:33.451 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: Generating public/private rsa key pair. 2023-09-12T15:55:33.452 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: Your identification has been saved in /tmp/tmphzetjo9s/key. 2023-09-12T15:55:33.452 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: Your public key has been saved in /tmp/tmphzetjo9s/key.pub. 2023-09-12T15:55:33.452 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: The key fingerprint is: 2023-09-12T15:55:33.453 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: SHA256:Zhj3mUNJonrVCAoosV26ny06GO9aZI7eQj7QSMzUjYQ ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:33.453 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: The key's randomart image is: 2023-09-12T15:55:33.453 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: +---[RSA 3072]----+ 2023-09-12T15:55:33.453 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: |.o=.+ . . . | 2023-09-12T15:55:33.453 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: |oE * o o = . | 2023-09-12T15:55:33.453 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: |* o . o + + | 2023-09-12T15:55:33.454 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: | + . . = o o | 2023-09-12T15:55:33.454 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: |.o+ . o S = | 2023-09-12T15:55:33.454 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: |+*.. + o . | 2023-09-12T15:55:33.454 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: |=+o + . | 2023-09-12T15:55:33.454 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: |o=+. . | 2023-09-12T15:55:33.455 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: |.+=o | 2023-09-12T15:55:33.455 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: +----[SHA256]-----+ 2023-09-12T15:55:34.041 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-mon[98670]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:55:34.041 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-mon[98670]: Generating ssh key... 2023-09-12T15:55:34.041 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:34.041 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:33 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:34.397 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6lLfFzPIrM9VL/P4+SAGHY+s/JnZr2O8zFpSGPxXOqVVmTG+wibIjl/jBD5S4yiUNzBy6RdVS8BlZbN5NhuNrkpTohXpnEs9kRY6bDwo98FpRPUdGLVUU1cZqvyL6AZLMNhof69JcTJPcq38unsQh7iZP2bt3n7Gi88QIt1SyTxhEOYEMUUL3RWQWbpmBEIguLEJ7UBd8AYei1DvXoVhJDgzxRX8jQVIS3MJjJnSYu90ei+RUJxtwGW10HSuhCzZXmdng4hswXZt+ECrmJ8ed2pk/8y7xB38mgqE8o0y/JqXbTlbb5JS7b9gx5OtpzoBWshEYza2d0DyCr5e2c85FiWB1YboYI59HQP2MORrG/WhSBUr6/pp4rcwBNqyjVgbyaQpLz1u4jC/m33iJPdZXz9HP56uAy9EWwexYFMkDwqXeSdTSERuMpuiP0I+m9i+X8ek8dzZaLVALcX8kuV1ntblKxhzV2rorFHmKF/CGZW2EwO6lsdFQTmwxQw+Dou8= ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:34.398 INFO:teuthology.orchestra.run.smithi130.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-09-12T15:55:34.398 INFO:teuthology.orchestra.run.smithi130.stdout:Adding key to root@localhost authorized_keys... 2023-09-12T15:55:34.399 INFO:teuthology.orchestra.run.smithi130.stdout:Adding host smithi130... 2023-09-12T15:55:34.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:34 smithi130 ceph-mon[98670]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:55:36.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:36 smithi130 ceph-mon[98670]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi130", "addr": "172.21.15.130", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:55:36.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:36 smithi130 ceph-mon[98670]: Deploying cephadm binary to smithi130 2023-09-12T15:55:37.745 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout Added host 'smithi130' with addr '172.21.15.130' 2023-09-12T15:55:37.745 INFO:teuthology.orchestra.run.smithi130.stdout:Deploying unmanaged mon service... 2023-09-12T15:55:38.636 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:38 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:38.637 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:38 smithi130 ceph-mon[98670]: Added host smithi130 2023-09-12T15:55:38.637 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:38 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:55:38.985 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-09-12T15:55:38.985 INFO:teuthology.orchestra.run.smithi130.stdout:Deploying unmanaged mgr service... 2023-09-12T15:55:39.561 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:39 smithi130 ceph-mon[98670]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:55:39.561 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:39 smithi130 ceph-mon[98670]: Saving service mon spec with placement count:5 2023-09-12T15:55:39.561 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:39 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:40.129 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-09-12T15:55:40.954 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:40 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:40.955 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:40 smithi130 ceph-mon[98670]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:55:40.955 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:40 smithi130 ceph-mon[98670]: Saving service mgr spec with placement count:2 2023-09-12T15:55:40.955 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:40 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:40.955 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:40 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:42.083 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:41 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1076218395' entity='client.admin' 2023-09-12T15:55:42.083 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:41 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:42.312 INFO:teuthology.orchestra.run.smithi130.stdout:Enabling the dashboard module... 2023-09-12T15:55:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:42 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/411416367' entity='client.admin' 2023-09-12T15:55:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:42 smithi130 ceph-mon[98670]: from='mgr.14118 172.21.15.130:0/2325604681' entity='mgr.x' 2023-09-12T15:55:44.127 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:43 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2458751086' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-09-12T15:55:44.127 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:43 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: ignoring --setuser ceph since I am not root 2023-09-12T15:55:44.127 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:43 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: ignoring --setgroup ceph since I am not root 2023-09-12T15:55:44.128 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:44 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:44.098+0000 7fcb234fc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-12T15:55:44.448 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:44 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:44.186+0000 7fcb234fc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-12T15:55:44.449 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:44 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:44.279+0000 7fcb234fc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-12T15:55:44.449 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:44 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:44.374+0000 7fcb234fc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-12T15:55:44.724 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:44 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:44.549+0000 7fcb234fc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-12T15:55:45.165 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:44 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2458751086' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-09-12T15:55:45.165 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:44 smithi130 ceph-mon[98670]: mgrmap e9: x(active, since 15s) 2023-09-12T15:55:45.166 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:44 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:44.934+0000 7fcb234fc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-12T15:55:45.166 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:45 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:45.071+0000 7fcb234fc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-12T15:55:45.166 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:45 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:45.164+0000 7fcb234fc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-12T15:55:45.453 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:45 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:45.262+0000 7fcb234fc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-12T15:55:45.703 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout { 2023-09-12T15:55:45.704 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-09-12T15:55:45.705 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-12T15:55:45.705 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-12T15:55:45.705 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-12T15:55:45.705 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout } 2023-09-12T15:55:45.705 INFO:teuthology.orchestra.run.smithi130.stdout:Waiting for the mgr to restart... 2023-09-12T15:55:45.705 INFO:teuthology.orchestra.run.smithi130.stdout:Waiting for mgr epoch 9... 2023-09-12T15:55:45.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:45 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2364165794' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-12T15:55:45.975 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:45 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:45.706+0000 7fcb234fc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-12T15:55:46.724 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:46 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:46.394+0000 7fcb234fc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-12T15:55:46.725 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:46 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:46.477+0000 7fcb234fc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-12T15:55:47.516 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:47 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:47.247+0000 7fcb234fc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:55:47.516 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:47 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:47.340+0000 7fcb234fc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-12T15:55:47.796 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:47 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:47.516+0000 7fcb234fc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:55:48.173 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:47 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:47.795+0000 7fcb234fc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-12T15:55:48.173 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:48 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:48.024+0000 7fcb234fc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-12T15:55:48.440 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:48 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:48.172+0000 7fcb234fc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-12T15:55:48.724 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:48 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:48.440+0000 7fcb234fc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-12T15:55:49.472 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:49 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:49.146+0000 7fcb234fc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-12T15:55:49.472 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:49 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:49.278+0000 7fcb234fc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-12T15:55:49.725 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:49 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:49.471+0000 7fcb234fc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-12T15:55:49.725 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:49 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:49.586+0000 7fcb234fc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-12T15:55:50.225 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:49 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:49.803+0000 7fcb234fc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-12T15:55:50.974 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T15:55:50.666+0000 7fcb234fc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-12T15:55:50.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: Active manager daemon x restarted 2023-09-12T15:55:50.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: Activating manager daemon x 2023-09-12T15:55:50.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: osdmap e3: 0 total, 0 up, 0 in 2023-09-12T15:55:50.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: mgrmap e10: x(active, starting, since 0.00344446s) 2023-09-12T15:55:50.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:55:50.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-12T15:55:50.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-12T15:55:50.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-12T15:55:50.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-12T15:55:50.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: Manager daemon x is now available 2023-09-12T15:55:50.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:55:50.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:55:51.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-12T15:55:51.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-12T15:55:51.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:51 smithi130 ceph-mon[98670]: mgrmap e11: x(active, since 1.00795s) 2023-09-12T15:55:52.370 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout { 2023-09-12T15:55:52.371 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-09-12T15:55:52.371 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-12T15:55:52.371 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout } 2023-09-12T15:55:52.371 INFO:teuthology.orchestra.run.smithi130.stdout:mgr epoch 9 is available 2023-09-12T15:55:52.371 INFO:teuthology.orchestra.run.smithi130.stdout:Generating a dashboard self-signed certificate... 2023-09-12T15:55:52.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:52 smithi130 ceph-mon[98670]: [12/Sep/2023:15:55:51] ENGINE Bus STARTING 2023-09-12T15:55:52.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:52 smithi130 ceph-mon[98670]: [12/Sep/2023:15:55:51] ENGINE Serving on https://172.21.15.130:7150 2023-09-12T15:55:52.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:52 smithi130 ceph-mon[98670]: [12/Sep/2023:15:55:51] ENGINE Serving on http://172.21.15.130:8765 2023-09-12T15:55:52.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:52 smithi130 ceph-mon[98670]: [12/Sep/2023:15:55:51] ENGINE Bus STARTED 2023-09-12T15:55:52.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:52 smithi130 ceph-mon[98670]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-12T15:55:52.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:52 smithi130 ceph-mon[98670]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-12T15:55:52.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:55:53.520 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-09-12T15:55:53.520 INFO:teuthology.orchestra.run.smithi130.stdout:Creating initial admin user... 2023-09-12T15:55:54.427 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:54 smithi130 ceph-mon[98670]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:55:54.427 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:55:54.427 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:55:54.428 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:55:54.428 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:54 smithi130 ceph-mon[98670]: mgrmap e12: x(active, since 2s) 2023-09-12T15:55:54.863 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$xr2O8W3mKn8o7gdL4U7PkuckFHoPX4ppaVSWRL7jonI4gH/UhuQba", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1694534154, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-09-12T15:55:54.864 INFO:teuthology.orchestra.run.smithi130.stdout:Fetching dashboard port number... 2023-09-12T15:55:55.702 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:55 smithi130 ceph-mon[98670]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:55:55.702 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:55:55.904 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stdout 8443 2023-09-12T15:55:55.916 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-12T15:55:55.916 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout disabled 2023-09-12T15:55:55.924 INFO:teuthology.orchestra.run.smithi130.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-12T15:55:55.924 INFO:teuthology.orchestra.run.smithi130.stdout:systemctl: stdout inactive 2023-09-12T15:55:55.924 INFO:teuthology.orchestra.run.smithi130.stdout:firewalld.service is not enabled 2023-09-12T15:55:55.924 INFO:teuthology.orchestra.run.smithi130.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-09-12T15:55:55.925 INFO:teuthology.orchestra.run.smithi130.stdout:Ceph Dashboard is now available at: 2023-09-12T15:55:55.925 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:55.926 INFO:teuthology.orchestra.run.smithi130.stdout: URL: https://smithi130.front.sepia.ceph.com:8443/ 2023-09-12T15:55:55.926 INFO:teuthology.orchestra.run.smithi130.stdout: User: admin 2023-09-12T15:55:55.926 INFO:teuthology.orchestra.run.smithi130.stdout: Password: 13qo6ofwzc 2023-09-12T15:55:55.926 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:55.927 INFO:teuthology.orchestra.run.smithi130.stdout:Saving cluster configuration to /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config directory 2023-09-12T15:55:55.927 INFO:teuthology.orchestra.run.smithi130.stdout:Enabling autotune for osd_memory_target 2023-09-12T15:55:56.724 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:56 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3055520222' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-09-12T15:55:58.106 INFO:teuthology.orchestra.run.smithi130.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-09-12T15:55:58.107 INFO:teuthology.orchestra.run.smithi130.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-09-12T15:55:58.107 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:58.107 INFO:teuthology.orchestra.run.smithi130.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-09-12T15:55:58.107 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:58.108 INFO:teuthology.orchestra.run.smithi130.stdout:Or, if you are only running a single cluster on this host: 2023-09-12T15:55:58.108 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:58.108 INFO:teuthology.orchestra.run.smithi130.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-09-12T15:55:58.108 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:58.108 INFO:teuthology.orchestra.run.smithi130.stdout:Please consider enabling telemetry to help improve Ceph: 2023-09-12T15:55:58.109 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:58.109 INFO:teuthology.orchestra.run.smithi130.stdout: ceph telemetry on 2023-09-12T15:55:58.109 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:58.109 INFO:teuthology.orchestra.run.smithi130.stdout:For more information see: 2023-09-12T15:55:58.109 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:58.110 INFO:teuthology.orchestra.run.smithi130.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-09-12T15:55:58.110 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:55:58.110 INFO:teuthology.orchestra.run.smithi130.stdout:Bootstrap complete. 2023-09-12T15:55:58.143 INFO:tasks.cephadm:Fetching config... 2023-09-12T15:55:58.144 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:55:58.144 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-09-12T15:55:58.160 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-09-12T15:55:58.161 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:55:58.161 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-09-12T15:55:58.217 INFO:tasks.cephadm:Fetching mon keyring... 2023-09-12T15:55:58.217 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:55:58.218 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/keyring of=/dev/stdout 2023-09-12T15:55:58.285 INFO:tasks.cephadm:Fetching pub ssh key... 2023-09-12T15:55:58.285 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:55:58.285 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-09-12T15:55:58.342 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-09-12T15:55:58.342 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6lLfFzPIrM9VL/P4+SAGHY+s/JnZr2O8zFpSGPxXOqVVmTG+wibIjl/jBD5S4yiUNzBy6RdVS8BlZbN5NhuNrkpTohXpnEs9kRY6bDwo98FpRPUdGLVUU1cZqvyL6AZLMNhof69JcTJPcq38unsQh7iZP2bt3n7Gi88QIt1SyTxhEOYEMUUL3RWQWbpmBEIguLEJ7UBd8AYei1DvXoVhJDgzxRX8jQVIS3MJjJnSYu90ei+RUJxtwGW10HSuhCzZXmdng4hswXZt+ECrmJ8ed2pk/8y7xB38mgqE8o0y/JqXbTlbb5JS7b9gx5OtpzoBWshEYza2d0DyCr5e2c85FiWB1YboYI59HQP2MORrG/WhSBUr6/pp4rcwBNqyjVgbyaQpLz1u4jC/m33iJPdZXz9HP56uAy9EWwexYFMkDwqXeSdTSERuMpuiP0I+m9i+X8ek8dzZaLVALcX8kuV1ntblKxhzV2rorFHmKF/CGZW2EwO6lsdFQTmwxQw+Dou8= ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-12T15:55:58.424 INFO:teuthology.orchestra.run.smithi130.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6lLfFzPIrM9VL/P4+SAGHY+s/JnZr2O8zFpSGPxXOqVVmTG+wibIjl/jBD5S4yiUNzBy6RdVS8BlZbN5NhuNrkpTohXpnEs9kRY6bDwo98FpRPUdGLVUU1cZqvyL6AZLMNhof69JcTJPcq38unsQh7iZP2bt3n7Gi88QIt1SyTxhEOYEMUUL3RWQWbpmBEIguLEJ7UBd8AYei1DvXoVhJDgzxRX8jQVIS3MJjJnSYu90ei+RUJxtwGW10HSuhCzZXmdng4hswXZt+ECrmJ8ed2pk/8y7xB38mgqE8o0y/JqXbTlbb5JS7b9gx5OtpzoBWshEYza2d0DyCr5e2c85FiWB1YboYI59HQP2MORrG/WhSBUr6/pp4rcwBNqyjVgbyaQpLz1u4jC/m33iJPdZXz9HP56uAy9EWwexYFMkDwqXeSdTSERuMpuiP0I+m9i+X8ek8dzZaLVALcX8kuV1ntblKxhzV2rorFHmKF/CGZW2EwO6lsdFQTmwxQw+Dou8= ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:58.435 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6lLfFzPIrM9VL/P4+SAGHY+s/JnZr2O8zFpSGPxXOqVVmTG+wibIjl/jBD5S4yiUNzBy6RdVS8BlZbN5NhuNrkpTohXpnEs9kRY6bDwo98FpRPUdGLVUU1cZqvyL6AZLMNhof69JcTJPcq38unsQh7iZP2bt3n7Gi88QIt1SyTxhEOYEMUUL3RWQWbpmBEIguLEJ7UBd8AYei1DvXoVhJDgzxRX8jQVIS3MJjJnSYu90ei+RUJxtwGW10HSuhCzZXmdng4hswXZt+ECrmJ8ed2pk/8y7xB38mgqE8o0y/JqXbTlbb5JS7b9gx5OtpzoBWshEYza2d0DyCr5e2c85FiWB1YboYI59HQP2MORrG/WhSBUr6/pp4rcwBNqyjVgbyaQpLz1u4jC/m33iJPdZXz9HP56uAy9EWwexYFMkDwqXeSdTSERuMpuiP0I+m9i+X8ek8dzZaLVALcX8kuV1ntblKxhzV2rorFHmKF/CGZW2EwO6lsdFQTmwxQw+Dou8= ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-12T15:55:58.480 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6lLfFzPIrM9VL/P4+SAGHY+s/JnZr2O8zFpSGPxXOqVVmTG+wibIjl/jBD5S4yiUNzBy6RdVS8BlZbN5NhuNrkpTohXpnEs9kRY6bDwo98FpRPUdGLVUU1cZqvyL6AZLMNhof69JcTJPcq38unsQh7iZP2bt3n7Gi88QIt1SyTxhEOYEMUUL3RWQWbpmBEIguLEJ7UBd8AYei1DvXoVhJDgzxRX8jQVIS3MJjJnSYu90ei+RUJxtwGW10HSuhCzZXmdng4hswXZt+ECrmJ8ed2pk/8y7xB38mgqE8o0y/JqXbTlbb5JS7b9gx5OtpzoBWshEYza2d0DyCr5e2c85FiWB1YboYI59HQP2MORrG/WhSBUr6/pp4rcwBNqyjVgbyaQpLz1u4jC/m33iJPdZXz9HP56uAy9EWwexYFMkDwqXeSdTSERuMpuiP0I+m9i+X8ek8dzZaLVALcX8kuV1ntblKxhzV2rorFHmKF/CGZW2EwO6lsdFQTmwxQw+Dou8= ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:58.493 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6lLfFzPIrM9VL/P4+SAGHY+s/JnZr2O8zFpSGPxXOqVVmTG+wibIjl/jBD5S4yiUNzBy6RdVS8BlZbN5NhuNrkpTohXpnEs9kRY6bDwo98FpRPUdGLVUU1cZqvyL6AZLMNhof69JcTJPcq38unsQh7iZP2bt3n7Gi88QIt1SyTxhEOYEMUUL3RWQWbpmBEIguLEJ7UBd8AYei1DvXoVhJDgzxRX8jQVIS3MJjJnSYu90ei+RUJxtwGW10HSuhCzZXmdng4hswXZt+ECrmJ8ed2pk/8y7xB38mgqE8o0y/JqXbTlbb5JS7b9gx5OtpzoBWshEYza2d0DyCr5e2c85FiWB1YboYI59HQP2MORrG/WhSBUr6/pp4rcwBNqyjVgbyaQpLz1u4jC/m33iJPdZXz9HP56uAy9EWwexYFMkDwqXeSdTSERuMpuiP0I+m9i+X8ek8dzZaLVALcX8kuV1ntblKxhzV2rorFHmKF/CGZW2EwO6lsdFQTmwxQw+Dou8= ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-12T15:55:58.539 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6lLfFzPIrM9VL/P4+SAGHY+s/JnZr2O8zFpSGPxXOqVVmTG+wibIjl/jBD5S4yiUNzBy6RdVS8BlZbN5NhuNrkpTohXpnEs9kRY6bDwo98FpRPUdGLVUU1cZqvyL6AZLMNhof69JcTJPcq38unsQh7iZP2bt3n7Gi88QIt1SyTxhEOYEMUUL3RWQWbpmBEIguLEJ7UBd8AYei1DvXoVhJDgzxRX8jQVIS3MJjJnSYu90ei+RUJxtwGW10HSuhCzZXmdng4hswXZt+ECrmJ8ed2pk/8y7xB38mgqE8o0y/JqXbTlbb5JS7b9gx5OtpzoBWshEYza2d0DyCr5e2c85FiWB1YboYI59HQP2MORrG/WhSBUr6/pp4rcwBNqyjVgbyaQpLz1u4jC/m33iJPdZXz9HP56uAy9EWwexYFMkDwqXeSdTSERuMpuiP0I+m9i+X8ek8dzZaLVALcX8kuV1ntblKxhzV2rorFHmKF/CGZW2EwO6lsdFQTmwxQw+Dou8= ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:55:58.552 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-09-12T15:55:58.715 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:58 smithi130 ceph-mon[98670]: mgrmap e13: x(active, since 6s) 2023-09-12T15:55:58.715 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:55:58 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2638562765' entity='client.admin' 2023-09-12T15:55:59.117 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:01.156 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:01.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:01.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:56:01.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:01.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:01.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:01.158 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:00 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1889000945' entity='client.admin' 2023-09-12T15:56:01.158 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:01.158 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:01.184 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-09-12T15:56:01.184 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-09-12T15:56:01.750 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:03.511 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2023-09-12T15:56:03.512 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:56:03.512 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2023-09-12T15:56:03.532 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:56:03.533 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:56:03.591 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2023-09-12T15:56:03.592 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch host add smithi159 2023-09-12T15:56:04.161 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:04.407 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:56:04.408 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:04.408 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:04.408 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:04.408 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:04.409 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: Updating smithi130:/etc/ceph/ceph.conf 2023-09-12T15:56:04.409 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: Updating smithi130:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:04.409 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:56:04.409 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: Updating smithi130:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-12T15:56:04.409 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:04.410 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:04.410 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:06.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:06 smithi130 ceph-mon[98670]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:56:07.862 INFO:teuthology.orchestra.run.smithi130.stdout:Added host 'smithi159' with addr '172.21.15.159' 2023-09-12T15:56:07.892 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:07 smithi130 ceph-mon[98670]: Deploying cephadm binary to smithi159 2023-09-12T15:56:08.224 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-12T15:56:08.790 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:09.158 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:09.159 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:08 smithi130 ceph-mon[98670]: Added host smithi159 2023-09-12T15:56:09.159 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:09.159 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:10.189 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:56:10.189 INFO:teuthology.orchestra.run.smithi130.stdout:[{"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2023-09-12T15:56:10.316 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:10.547 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2023-09-12T15:56:10.547 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:56:10.548 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2023-09-12T15:56:10.568 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:56:10.569 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:56:10.628 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2023-09-12T15:56:10.628 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch host add smithi203 2023-09-12T15:56:11.181 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:11.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:11 smithi130 ceph-mon[98670]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T15:56:11.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:11.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:11 smithi130 ceph-mon[98670]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:13 smithi130 ceph-mon[98670]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:56:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:13 smithi130 ceph-mon[98670]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:14.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:56:14.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:14.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:14.915 INFO:teuthology.orchestra.run.smithi130.stdout:Added host 'smithi203' with addr '172.21.15.203' 2023-09-12T15:56:15.050 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:14 smithi130 ceph-mon[98670]: Deploying cephadm binary to smithi203 2023-09-12T15:56:15.050 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:14 smithi130 ceph-mon[98670]: Updating smithi159:/etc/ceph/ceph.conf 2023-09-12T15:56:15.051 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:14 smithi130 ceph-mon[98670]: Updating smithi159:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:15.051 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:14 smithi130 ceph-mon[98670]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:56:15.051 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:15.051 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:15.051 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:15.238 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-12T15:56:15.804 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:16.197 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:15 smithi130 ceph-mon[98670]: Updating smithi159:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-12T15:56:16.197 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:15 smithi130 ceph-mon[98670]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:16.197 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:16.197 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:15 smithi130 ceph-mon[98670]: Added host smithi203 2023-09-12T15:56:16.198 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:16.198 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:17.148 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T15:56:17.149 INFO:teuthology.orchestra.run.smithi130.stdout:[{"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2023-09-12T15:56:17.176 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:17.176 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:17 smithi130 ceph-mon[98670]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:17.548 INFO:tasks.cephadm:Setting crush tunables to default 2023-09-12T15:56:17.549 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd crush tunables default 2023-09-12T15:56:18.103 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:18.582 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:18 smithi130 ceph-mon[98670]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T15:56:18.582 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:19.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:19 smithi130 ceph-mon[98670]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:20.291 INFO:teuthology.orchestra.run.smithi130.stderr:adjusted tunables profile to default 2023-09-12T15:56:20.659 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:20 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2643330924' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-12T15:56:20.660 INFO:tasks.cephadm:Adding mon.a on smithi130 2023-09-12T15:56:20.661 INFO:tasks.cephadm:Adding mon.b on smithi159 2023-09-12T15:56:20.661 INFO:tasks.cephadm:Adding mon.c on smithi203 2023-09-12T15:56:20.661 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch apply mon '3;smithi130:172.21.15.130=a;smithi159:172.21.15.159=b;smithi203:172.21.15.203=c' 2023-09-12T15:56:21.266 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:21.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2643330924' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-12T15:56:21.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: osdmap e4: 0 total, 0 up, 0 in 2023-09-12T15:56:21.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:21.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:21.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:21.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:21.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:56:21.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:21.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:21.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: Updating smithi203:/etc/ceph/ceph.conf 2023-09-12T15:56:21.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: Updating smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:21.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:21.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:56:21.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: Updating smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-12T15:56:21.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:21.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:21.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:22.739 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2023-09-12T15:56:23.144 DEBUG:teuthology.orchestra.run.smithi159:mon.b> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.b.service 2023-09-12T15:56:23.147 DEBUG:teuthology.orchestra.run.smithi203:mon.c> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.c.service 2023-09-12T15:56:23.150 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-12T15:56:23.150 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-12T15:56:23.179 INFO:journalctl@ceph.mon.c.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T15:56:23.183 INFO:journalctl@ceph.mon.b.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T15:56:23.998 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:24.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:24.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi130:172.21.15.130=a;smithi159:172.21.15.159=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:56:24.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: Saving service mon spec with placement smithi130:172.21.15.130=a;smithi159:172.21.15.159=b;smithi203:172.21.15.203=c;count:3 2023-09-12T15:56:24.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:24.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:24.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:24.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:24.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:24.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:24.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:24.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:23 smithi130 ceph-mon[98670]: Deploying daemon mon.c on smithi203 2023-09-12T15:56:26.187 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:56:26.187 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","modified":"2023-09-12T15:55:00.069564Z","created":"2023-09-12T15:55:00.069564Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-09-12T15:56:26.188 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2023-09-12T15:56:26.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:25 smithi130 ceph-mon[98670]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:26.488 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 podman[103370]: 2023-09-12T15:56:26.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 bash[103370]: db561d8c372354e61f15fa40649500eca2ec227b4258a233926a5d1c40d9057f 2023-09-12T15:56:26.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 systemd[1]: Started Ceph mon.c for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T15:56:26.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: set uid:gid to 167:167 (ceph:ceph) 2023-09-12T15:56:26.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev), process ceph-mon, pid 7 2023-09-12T15:56:26.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: pidfile_write: ignore empty --pid-file 2023-09-12T15:56:26.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: load: jerasure load: lrc 2023-09-12T15:56:26.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: RocksDB version: 7.9.2 2023-09-12T15:56:26.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Git sha 0 2023-09-12T15:56:26.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Compile date 2023-09-12 13:00:11 2023-09-12T15:56:26.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: DB SUMMARY 2023-09-12T15:56:26.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: DB Session ID: 8O7VCKMWSXZMRTXZ01S9 2023-09-12T15:56:26.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: CURRENT file: CURRENT 2023-09-12T15:56:26.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: IDENTITY file: IDENTITY 2023-09-12T15:56:27.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-12T15:56:27.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-09-12T15:56:27.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-09-12T15:56:27.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.error_if_exists: 0 2023-09-12T15:56:27.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.create_if_missing: 0 2023-09-12T15:56:27.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.paranoid_checks: 1 2023-09-12T15:56:27.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-12T15:56:27.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-12T15:56:27.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-12T15:56:27.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.env: 0x55f3f0f45da0 2023-09-12T15:56:27.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.fs: PosixFileSystem 2023-09-12T15:56:27.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.info_log: 0x55f3f1bdf4e0 2023-09-12T15:56:27.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_file_opening_threads: 16 2023-09-12T15:56:27.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.statistics: (nil) 2023-09-12T15:56:27.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.use_fsync: 0 2023-09-12T15:56:27.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_log_file_size: 0 2023-09-12T15:56:27.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-12T15:56:27.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-12T15:56:27.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.keep_log_file_num: 1000 2023-09-12T15:56:27.004 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.recycle_log_file_num: 0 2023-09-12T15:56:27.004 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.allow_fallocate: 1 2023-09-12T15:56:27.004 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.allow_mmap_reads: 0 2023-09-12T15:56:27.004 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.allow_mmap_writes: 0 2023-09-12T15:56:27.004 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.use_direct_reads: 0 2023-09-12T15:56:27.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-12T15:56:27.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.create_missing_column_families: 0 2023-09-12T15:56:27.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.db_log_dir: 2023-09-12T15:56:27.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.wal_dir: 2023-09-12T15:56:27.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-12T15:56:27.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-12T15:56:27.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-12T15:56:27.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-12T15:56:27.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-12T15:56:27.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-12T15:56:27.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.advise_random_on_open: 1 2023-09-12T15:56:27.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.db_write_buffer_size: 0 2023-09-12T15:56:27.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.write_buffer_manager: 0x55f3f1b994a0 2023-09-12T15:56:27.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-12T15:56:27.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-12T15:56:27.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-12T15:56:27.008 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.rate_limiter: (nil) 2023-09-12T15:56:27.008 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-12T15:56:27.008 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.wal_recovery_mode: 2 2023-09-12T15:56:27.008 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.enable_thread_tracking: 0 2023-09-12T15:56:27.009 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.enable_pipelined_write: 0 2023-09-12T15:56:27.009 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.unordered_write: 0 2023-09-12T15:56:27.009 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-12T15:56:27.009 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-12T15:56:27.009 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-12T15:56:27.010 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-12T15:56:27.010 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.row_cache: None 2023-09-12T15:56:27.010 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.wal_filter: None 2023-09-12T15:56:27.010 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-12T15:56:27.010 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.allow_ingest_behind: 0 2023-09-12T15:56:27.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.two_write_queues: 0 2023-09-12T15:56:27.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.manual_wal_flush: 0 2023-09-12T15:56:27.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.wal_compression: 0 2023-09-12T15:56:27.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.atomic_flush: 0 2023-09-12T15:56:27.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-12T15:56:27.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-12T15:56:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-12T15:56:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.log_readahead_size: 0 2023-09-12T15:56:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-12T15:56:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.best_efforts_recovery: 0 2023-09-12T15:56:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-12T15:56:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-12T15:56:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.allow_data_in_errors: 0 2023-09-12T15:56:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.db_host_id: __hostname__ 2023-09-12T15:56:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-12T15:56:27.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_background_jobs: 2 2023-09-12T15:56:27.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_background_compactions: -1 2023-09-12T15:56:27.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_subcompactions: 1 2023-09-12T15:56:27.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-12T15:56:27.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-12T15:56:27.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-12T15:56:27.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_total_wal_size: 0 2023-09-12T15:56:27.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-12T15:56:27.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-12T15:56:27.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-12T15:56:27.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-12T15:56:27.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_open_files: -1 2023-09-12T15:56:27.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bytes_per_sync: 0 2023-09-12T15:56:27.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-12T15:56:27.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-12T15:56:27.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_readahead_size: 0 2023-09-12T15:56:27.019 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_background_flushes: -1 2023-09-12T15:56:27.019 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Compression algorithms supported: 2023-09-12T15:56:27.019 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-12T15:56:27.020 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: kZSTD supported: 0 2023-09-12T15:56:27.020 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: kXpressCompression supported: 0 2023-09-12T15:56:27.020 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: kLZ4HCCompression supported: 1 2023-09-12T15:56:27.020 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: kZlibCompression supported: 1 2023-09-12T15:56:27.021 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: kSnappyCompression supported: 1 2023-09-12T15:56:27.021 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: kLZ4Compression supported: 1 2023-09-12T15:56:27.021 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: kBZip2Compression supported: 0 2023-09-12T15:56:27.021 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-12T15:56:27.022 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-12T15:56:27.022 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-09-12T15:56:27.022 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-12T15:56:27.022 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-12T15:56:27.022 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.merge_operator: 2023-09-12T15:56:27.023 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_filter: None 2023-09-12T15:56:27.023 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_filter_factory: None 2023-09-12T15:56:27.023 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.sst_partitioner_factory: None 2023-09-12T15:56:27.023 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-12T15:56:27.023 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-12T15:56:27.024 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f3f1bdf5e0) 2023-09-12T15:56:27.024 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks: 1 2023-09-12T15:56:27.024 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-12T15:56:27.024 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-12T15:56:27.024 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_top_level_index_and_filter: 1 2023-09-12T15:56:27.025 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_type: 0 2023-09-12T15:56:27.025 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_index_type: 0 2023-09-12T15:56:27.025 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_shortening: 1 2023-09-12T15:56:27.025 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-12T15:56:27.026 INFO:journalctl@ceph.mon.c.smithi203.stdout: checksum: 4 2023-09-12T15:56:27.026 INFO:journalctl@ceph.mon.c.smithi203.stdout: no_block_cache: 0 2023-09-12T15:56:27.026 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache: 0x55f3f1c371f0 2023-09-12T15:56:27.026 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_name: BinnedLRUCache 2023-09-12T15:56:27.027 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_options: 2023-09-12T15:56:27.027 INFO:journalctl@ceph.mon.c.smithi203.stdout: capacity : 536870912 2023-09-12T15:56:27.027 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_shard_bits : 4 2023-09-12T15:56:27.027 INFO:journalctl@ceph.mon.c.smithi203.stdout: strict_capacity_limit : 0 2023-09-12T15:56:27.027 INFO:journalctl@ceph.mon.c.smithi203.stdout: high_pri_pool_ratio: 0.000 2023-09-12T15:56:27.028 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_compressed: (nil) 2023-09-12T15:56:27.028 INFO:journalctl@ceph.mon.c.smithi203.stdout: persistent_cache: (nil) 2023-09-12T15:56:27.028 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size: 4096 2023-09-12T15:56:27.028 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size_deviation: 10 2023-09-12T15:56:27.028 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_restart_interval: 16 2023-09-12T15:56:27.028 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_block_restart_interval: 1 2023-09-12T15:56:27.029 INFO:journalctl@ceph.mon.c.smithi203.stdout: metadata_block_size: 4096 2023-09-12T15:56:27.029 INFO:journalctl@ceph.mon.c.smithi203.stdout: partition_filters: 0 2023-09-12T15:56:27.029 INFO:journalctl@ceph.mon.c.smithi203.stdout: use_delta_encoding: 1 2023-09-12T15:56:27.029 INFO:journalctl@ceph.mon.c.smithi203.stdout: filter_policy: bloomfilter 2023-09-12T15:56:27.029 INFO:journalctl@ceph.mon.c.smithi203.stdout: whole_key_filtering: 1 2023-09-12T15:56:27.030 INFO:journalctl@ceph.mon.c.smithi203.stdout: verify_compression: 0 2023-09-12T15:56:27.030 INFO:journalctl@ceph.mon.c.smithi203.stdout: read_amp_bytes_per_bit: 0 2023-09-12T15:56:27.030 INFO:journalctl@ceph.mon.c.smithi203.stdout: format_version: 5 2023-09-12T15:56:27.030 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_index_compression: 1 2023-09-12T15:56:27.030 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_align: 0 2023-09-12T15:56:27.031 INFO:journalctl@ceph.mon.c.smithi203.stdout: max_auto_readahead_size: 262144 2023-09-12T15:56:27.031 INFO:journalctl@ceph.mon.c.smithi203.stdout: prepopulate_block_cache: 0 2023-09-12T15:56:27.031 INFO:journalctl@ceph.mon.c.smithi203.stdout: initial_auto_readahead_size: 8192 2023-09-12T15:56:27.031 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_file_reads_for_auto_readahead: 2 2023-09-12T15:56:27.031 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.write_buffer_size: 33554432 2023-09-12T15:56:27.032 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_write_buffer_number: 2 2023-09-12T15:56:27.032 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression: NoCompression 2023-09-12T15:56:27.032 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression: Disabled 2023-09-12T15:56:27.032 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.prefix_extractor: nullptr 2023-09-12T15:56:27.032 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-12T15:56:27.032 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.num_levels: 7 2023-09-12T15:56:27.033 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-12T15:56:27.033 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-12T15:56:27.033 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-12T15:56:27.033 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-12T15:56:27.033 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-12T15:56:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-12T15:56:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-12T15:56:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-12T15:56:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-12T15:56:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-12T15:56:27.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-12T15:56:27.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-12T15:56:27.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-12T15:56:27.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression_opts.level: 32767 2023-09-12T15:56:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression_opts.strategy: 0 2023-09-12T15:56:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-12T15:56:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-12T15:56:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-12T15:56:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-12T15:56:27.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression_opts.enabled: false 2023-09-12T15:56:27.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-12T15:56:27.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-12T15:56:27.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-12T15:56:27.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-12T15:56:27.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.target_file_size_base: 67108864 2023-09-12T15:56:27.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-12T15:56:27.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-12T15:56:27.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-12T15:56:27.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-12T15:56:27.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-12T15:56:27.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-12T15:56:27.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-12T15:56:27.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-12T15:56:27.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-12T15:56:27.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-12T15:56:27.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-12T15:56:27.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-12T15:56:27.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-12T15:56:27.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-12T15:56:27.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.arena_block_size: 1048576 2023-09-12T15:56:27.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-12T15:56:27.041 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-12T15:56:27.041 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.disable_auto_compactions: 0 2023-09-12T15:56:27.041 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-12T15:56:27.041 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-12T15:56:27.041 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-12T15:56:27.042 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-12T15:56:27.042 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-12T15:56:27.042 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-12T15:56:27.042 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-12T15:56:27.042 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-12T15:56:27.043 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-12T15:56:27.043 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-12T15:56:27.043 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.table_properties_collectors: 2023-09-12T15:56:27.043 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.inplace_update_support: 0 2023-09-12T15:56:27.043 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-12T15:56:27.044 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-12T15:56:27.044 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-12T15:56:27.044 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-12T15:56:27.044 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.bloom_locality: 0 2023-09-12T15:56:27.045 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.max_successive_merges: 0 2023-09-12T15:56:27.045 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-12T15:56:27.045 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.paranoid_file_checks: 0 2023-09-12T15:56:27.046 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.force_consistency_checks: 1 2023-09-12T15:56:27.046 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.report_bg_io_stats: 0 2023-09-12T15:56:27.046 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.ttl: 2592000 2023-09-12T15:56:27.047 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-12T15:56:27.047 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-12T15:56:27.047 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-12T15:56:27.048 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.enable_blob_files: false 2023-09-12T15:56:27.048 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.min_blob_size: 0 2023-09-12T15:56:27.048 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.blob_file_size: 268435456 2023-09-12T15:56:27.049 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-12T15:56:27.049 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-12T15:56:27.050 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-12T15:56:27.050 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-12T15:56:27.050 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-12T15:56:27.051 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.blob_file_starting_level: 0 2023-09-12T15:56:27.051 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-12T15:56:27.052 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-12T15:56:27.052 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-12T15:56:27.052 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 59740b55-95b0-43de-b115-6bc2a6d396e1 2023-09-12T15:56:27.053 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694534186692312, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-12T15:56:27.053 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-12T15:56:27.053 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694534186692732, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694534186, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "59740b55-95b0-43de-b115-6bc2a6d396e1", "db_session_id": "8O7VCKMWSXZMRTXZ01S9", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-12T15:56:27.054 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694534186692800, "job": 1, "event": "recovery_finished"} 2023-09-12T15:56:27.054 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-12T15:56:27.054 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-12T15:56:27.055 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f3f1cd2000 2023-09-12T15:56:27.055 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: DB pointer 0x55f3f1cbe000 2023-09-12T15:56:27.055 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-12T15:56:27.056 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-12T15:56:27.056 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** DB Stats ** 2023-09-12T15:56:27.056 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-12T15:56:27.057 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-12T15:56:27.057 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-12T15:56:27.057 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-12T15:56:27.058 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-12T15:56:27.058 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-12T15:56:27.059 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-12T15:56:27.059 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-09-12T15:56:27.059 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2023-09-12T15:56:27.060 INFO:journalctl@ceph.mon.c.smithi203.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-12T15:56:27.060 INFO:journalctl@ceph.mon.c.smithi203.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-12T15:56:27.060 INFO:journalctl@ceph.mon.c.smithi203.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-12T15:56:27.061 INFO:journalctl@ceph.mon.c.smithi203.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-12T15:56:27.061 INFO:journalctl@ceph.mon.c.smithi203.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-12T15:56:27.062 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-09-12T15:56:27.062 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2023-09-12T15:56:27.062 INFO:journalctl@ceph.mon.c.smithi203.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-12T15:56:27.063 INFO:journalctl@ceph.mon.c.smithi203.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-12T15:56:27.063 INFO:journalctl@ceph.mon.c.smithi203.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-12T15:56:27.063 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-09-12T15:56:27.064 INFO:journalctl@ceph.mon.c.smithi203.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-12T15:56:27.064 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-09-12T15:56:27.064 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-12T15:56:27.065 INFO:journalctl@ceph.mon.c.smithi203.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-12T15:56:27.065 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-12T15:56:27.065 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-12T15:56:27.066 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-12T15:56:27.066 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-12T15:56:27.066 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-12T15:56:27.067 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-12T15:56:27.067 INFO:journalctl@ceph.mon.c.smithi203.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-12T15:56:27.067 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache BinnedLRUCache@0x55f3f1c371f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-09-12T15:56:27.068 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-12T15:56:27.068 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-09-12T15:56:27.068 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-12T15:56:27.068 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-09-12T15:56:27.069 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: using public_addr v2:172.21.15.203:0/0 -> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2023-09-12T15:56:27.069 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: starting mon.c rank -1 at public addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] at bind addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:56:27.069 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(???) e0 preinit fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:56:27.069 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).mds e1 new map 2023-09-12T15:56:27.069 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).mds e1 print_map 2023-09-12T15:56:27.070 INFO:journalctl@ceph.mon.c.smithi203.stdout: e1 2023-09-12T15:56:27.070 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-12T15:56:27.070 INFO:journalctl@ceph.mon.c.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-12T15:56:27.070 INFO:journalctl@ceph.mon.c.smithi203.stdout: legacy client fscid: -1 2023-09-12T15:56:27.070 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-09-12T15:56:27.071 INFO:journalctl@ceph.mon.c.smithi203.stdout: No filesystems configured 2023-09-12T15:56:27.071 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-12T15:56:27.071 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-12T15:56:27.071 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-12T15:56:27.071 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-12T15:56:27.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-12T15:56:27.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-12T15:56:27.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-12T15:56:27.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-12T15:56:27.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-12T15:56:27.073 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-12T15:56:27.073 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:27.073 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2643330924' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-12T15:56:27.073 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2643330924' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-12T15:56:27.073 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: osdmap e4: 0 total, 0 up, 0 in 2023-09-12T15:56:27.074 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:27.074 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:27.074 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:27.074 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:27.074 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:56:27.075 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:27.075 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:27.075 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: Updating smithi203:/etc/ceph/ceph.conf 2023-09-12T15:56:27.075 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: Updating smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:27.075 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:27.076 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:56:27.076 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: Updating smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-12T15:56:27.076 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:27.076 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:27.076 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:27.077 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:27.077 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi130:172.21.15.130=a;smithi159:172.21.15.159=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:56:27.077 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: Saving service mon spec with placement smithi130:172.21.15.130=a;smithi159:172.21.15.159=b;smithi203:172.21.15.203=c;count:3 2023-09-12T15:56:27.078 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:27.078 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:27.078 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:27.078 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:27.078 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:27.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:27.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:27.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: Deploying daemon mon.c on smithi203 2023-09-12T15:56:27.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:27.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-12T15:56:27.080 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-09-12T15:56:27.080 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta expand map: {default=false} 2023-09-12T15:56:27.080 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta from 'false' to 'false' 2023-09-12T15:56:27.080 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta expanded map: {default=false} 2023-09-12T15:56:27.080 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta expand map: {default=info} 2023-09-12T15:56:27.081 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta from 'info' to 'info' 2023-09-12T15:56:27.081 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta expanded map: {default=info} 2023-09-12T15:56:27.081 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta expand map: {default=daemon} 2023-09-12T15:56:27.081 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-12T15:56:27.082 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta expanded map: {default=daemon} 2023-09-12T15:56:27.082 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta expand map: {default=debug} 2023-09-12T15:56:27.082 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta from 'debug' to 'debug' 2023-09-12T15:56:27.083 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:26 smithi203 ceph-mon[103431]: expand_channel_meta expanded map: {default=debug} 2023-09-12T15:56:27.821 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-12T15:56:27.821 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-12T15:56:28.390 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T15:56:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:56:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: mon.a calling monitor election 2023-09-12T15:56:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: mon.c calling monitor election 2023-09-12T15:56:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-12T15:56:32.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: monmap e2: 2 mons at {a=[v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} 2023-09-12T15:56:32.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: fsmap 2023-09-12T15:56:32.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: osdmap e4: 0 total, 0 up, 0 in 2023-09-12T15:56:32.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: mgrmap e13: x(active, since 41s) 2023-09-12T15:56:32.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: overall HEALTH_OK 2023-09-12T15:56:32.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:32.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:32.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:32.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:32.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:32.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:32.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:56:32.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: mon.a calling monitor election 2023-09-12T15:56:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: mon.c calling monitor election 2023-09-12T15:56:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:32.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:32.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-12T15:56:32.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: monmap e2: 2 mons at {a=[v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} 2023-09-12T15:56:32.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: fsmap 2023-09-12T15:56:32.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: osdmap e4: 0 total, 0 up, 0 in 2023-09-12T15:56:32.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: mgrmap e13: x(active, since 41s) 2023-09-12T15:56:32.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: overall HEALTH_OK 2023-09-12T15:56:32.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:32.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:32.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:32.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:32.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:32.997 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:56:32.997 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","modified":"2023-09-12T15:56:26.722258Z","created":"2023-09-12T15:55:00.069564Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-12T15:56:32.999 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2023-09-12T15:56:33.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:32 smithi203 ceph-mon[103431]: Deploying daemon mon.b on smithi159 2023-09-12T15:56:33.141 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:32 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:33.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:32 smithi130 ceph-mon[98670]: Deploying daemon mon.b on smithi159 2023-09-12T15:56:33.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:32 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:34.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:33 smithi130 ceph-mon[98670]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:34.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:33 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/3948296732' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-12T15:56:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:33 smithi203 ceph-mon[103431]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:33 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/3948296732' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-12T15:56:34.341 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-12T15:56:34.344 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-12T15:56:34.623 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 systemd[1]: Starting Ceph mon.b for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T15:56:34.910 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T15:56:34.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 podman[102482]: 2023-09-12T15:56:34.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: set uid:gid to 167:167 (ceph:ceph) 2023-09-12T15:56:34.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev), process ceph-mon, pid 7 2023-09-12T15:56:34.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 bash[102482]: 9a218cd014a6b86d41ea6fedb12b10ab629d2f187da8786de30a150383ecf06b 2023-09-12T15:56:34.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: pidfile_write: ignore empty --pid-file 2023-09-12T15:56:34.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: load: jerasure load: lrc 2023-09-12T15:56:34.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: RocksDB version: 7.9.2 2023-09-12T15:56:34.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Git sha 0 2023-09-12T15:56:34.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Compile date 2023-09-12 13:00:11 2023-09-12T15:56:34.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: DB SUMMARY 2023-09-12T15:56:34.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: DB Session ID: VW9ZZ8OT5P0T5JGXMV2X 2023-09-12T15:56:34.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: CURRENT file: CURRENT 2023-09-12T15:56:34.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: IDENTITY file: IDENTITY 2023-09-12T15:56:34.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-12T15:56:34.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-09-12T15:56:34.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-09-12T15:56:34.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.error_if_exists: 0 2023-09-12T15:56:34.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.create_if_missing: 0 2023-09-12T15:56:34.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.paranoid_checks: 1 2023-09-12T15:56:34.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-12T15:56:34.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-12T15:56:34.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-12T15:56:34.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.env: 0x561f8d4efda0 2023-09-12T15:56:34.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.fs: PosixFileSystem 2023-09-12T15:56:34.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.info_log: 0x561f8fd394e0 2023-09-12T15:56:34.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_file_opening_threads: 16 2023-09-12T15:56:34.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.statistics: (nil) 2023-09-12T15:56:34.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.use_fsync: 0 2023-09-12T15:56:34.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_log_file_size: 0 2023-09-12T15:56:34.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-12T15:56:34.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-12T15:56:34.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.keep_log_file_num: 1000 2023-09-12T15:56:34.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.recycle_log_file_num: 0 2023-09-12T15:56:34.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.allow_fallocate: 1 2023-09-12T15:56:34.983 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.allow_mmap_reads: 0 2023-09-12T15:56:34.983 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.allow_mmap_writes: 0 2023-09-12T15:56:34.983 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.use_direct_reads: 0 2023-09-12T15:56:34.983 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-12T15:56:34.983 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.create_missing_column_families: 0 2023-09-12T15:56:34.984 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.db_log_dir: 2023-09-12T15:56:34.984 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.wal_dir: 2023-09-12T15:56:34.984 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-12T15:56:34.984 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-12T15:56:34.984 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-12T15:56:34.984 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-12T15:56:34.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-12T15:56:34.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-12T15:56:34.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.advise_random_on_open: 1 2023-09-12T15:56:34.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.db_write_buffer_size: 0 2023-09-12T15:56:34.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.write_buffer_manager: 0x561f8fcf34a0 2023-09-12T15:56:34.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-12T15:56:34.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-12T15:56:34.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-12T15:56:34.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.rate_limiter: (nil) 2023-09-12T15:56:34.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-12T15:56:34.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.wal_recovery_mode: 2 2023-09-12T15:56:34.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.enable_thread_tracking: 0 2023-09-12T15:56:34.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.enable_pipelined_write: 0 2023-09-12T15:56:34.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.unordered_write: 0 2023-09-12T15:56:34.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-12T15:56:34.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-12T15:56:34.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-12T15:56:34.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-12T15:56:34.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.row_cache: None 2023-09-12T15:56:34.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.wal_filter: None 2023-09-12T15:56:34.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-12T15:56:34.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.allow_ingest_behind: 0 2023-09-12T15:56:34.989 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.two_write_queues: 0 2023-09-12T15:56:34.989 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.manual_wal_flush: 0 2023-09-12T15:56:34.989 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.wal_compression: 0 2023-09-12T15:56:34.989 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.atomic_flush: 0 2023-09-12T15:56:34.989 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-12T15:56:34.990 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-12T15:56:34.990 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-12T15:56:34.990 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.log_readahead_size: 0 2023-09-12T15:56:34.990 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-12T15:56:34.990 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.best_efforts_recovery: 0 2023-09-12T15:56:34.990 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-12T15:56:34.991 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-12T15:56:34.991 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.allow_data_in_errors: 0 2023-09-12T15:56:34.991 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.db_host_id: __hostname__ 2023-09-12T15:56:34.991 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-12T15:56:34.991 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_background_jobs: 2 2023-09-12T15:56:34.992 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_background_compactions: -1 2023-09-12T15:56:34.992 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_subcompactions: 1 2023-09-12T15:56:34.992 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-12T15:56:34.992 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-12T15:56:34.992 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-12T15:56:34.992 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_total_wal_size: 0 2023-09-12T15:56:34.993 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-12T15:56:34.993 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-12T15:56:34.993 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-12T15:56:34.993 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-12T15:56:34.993 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_open_files: -1 2023-09-12T15:56:34.994 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bytes_per_sync: 0 2023-09-12T15:56:34.994 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-12T15:56:34.994 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-12T15:56:34.994 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_readahead_size: 0 2023-09-12T15:56:34.994 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_background_flushes: -1 2023-09-12T15:56:34.995 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Compression algorithms supported: 2023-09-12T15:56:34.995 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-12T15:56:34.995 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: kZSTD supported: 0 2023-09-12T15:56:34.995 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: kXpressCompression supported: 0 2023-09-12T15:56:34.995 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: kLZ4HCCompression supported: 1 2023-09-12T15:56:34.996 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: kZlibCompression supported: 1 2023-09-12T15:56:34.996 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: kSnappyCompression supported: 1 2023-09-12T15:56:34.996 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: kLZ4Compression supported: 1 2023-09-12T15:56:34.996 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: kBZip2Compression supported: 0 2023-09-12T15:56:34.996 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-12T15:56:34.996 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-12T15:56:34.997 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-09-12T15:56:34.997 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-12T15:56:34.997 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-12T15:56:34.997 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.merge_operator: 2023-09-12T15:56:34.997 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_filter: None 2023-09-12T15:56:34.997 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_filter_factory: None 2023-09-12T15:56:34.998 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.sst_partitioner_factory: None 2023-09-12T15:56:34.998 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-12T15:56:34.998 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-12T15:56:34.998 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561f8fd395e0) 2023-09-12T15:56:34.998 INFO:journalctl@ceph.mon.b.smithi159.stdout: cache_index_and_filter_blocks: 1 2023-09-12T15:56:34.999 INFO:journalctl@ceph.mon.b.smithi159.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-12T15:56:34.999 INFO:journalctl@ceph.mon.b.smithi159.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-12T15:56:34.999 INFO:journalctl@ceph.mon.b.smithi159.stdout: pin_top_level_index_and_filter: 1 2023-09-12T15:56:34.999 INFO:journalctl@ceph.mon.b.smithi159.stdout: index_type: 0 2023-09-12T15:56:34.999 INFO:journalctl@ceph.mon.b.smithi159.stdout: data_block_index_type: 0 2023-09-12T15:56:35.000 INFO:journalctl@ceph.mon.b.smithi159.stdout: index_shortening: 1 2023-09-12T15:56:35.000 INFO:journalctl@ceph.mon.b.smithi159.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-12T15:56:35.000 INFO:journalctl@ceph.mon.b.smithi159.stdout: checksum: 4 2023-09-12T15:56:35.000 INFO:journalctl@ceph.mon.b.smithi159.stdout: no_block_cache: 0 2023-09-12T15:56:35.000 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache: 0x561f8fd911f0 2023-09-12T15:56:35.000 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache_name: BinnedLRUCache 2023-09-12T15:56:35.001 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache_options: 2023-09-12T15:56:35.001 INFO:journalctl@ceph.mon.b.smithi159.stdout: capacity : 536870912 2023-09-12T15:56:35.001 INFO:journalctl@ceph.mon.b.smithi159.stdout: num_shard_bits : 4 2023-09-12T15:56:35.001 INFO:journalctl@ceph.mon.b.smithi159.stdout: strict_capacity_limit : 0 2023-09-12T15:56:35.001 INFO:journalctl@ceph.mon.b.smithi159.stdout: high_pri_pool_ratio: 0.000 2023-09-12T15:56:35.002 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache_compressed: (nil) 2023-09-12T15:56:35.002 INFO:journalctl@ceph.mon.b.smithi159.stdout: persistent_cache: (nil) 2023-09-12T15:56:35.002 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_size: 4096 2023-09-12T15:56:35.002 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_size_deviation: 10 2023-09-12T15:56:35.002 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_restart_interval: 16 2023-09-12T15:56:35.002 INFO:journalctl@ceph.mon.b.smithi159.stdout: index_block_restart_interval: 1 2023-09-12T15:56:35.003 INFO:journalctl@ceph.mon.b.smithi159.stdout: metadata_block_size: 4096 2023-09-12T15:56:35.003 INFO:journalctl@ceph.mon.b.smithi159.stdout: partition_filters: 0 2023-09-12T15:56:35.003 INFO:journalctl@ceph.mon.b.smithi159.stdout: use_delta_encoding: 1 2023-09-12T15:56:35.003 INFO:journalctl@ceph.mon.b.smithi159.stdout: filter_policy: bloomfilter 2023-09-12T15:56:35.003 INFO:journalctl@ceph.mon.b.smithi159.stdout: whole_key_filtering: 1 2023-09-12T15:56:35.004 INFO:journalctl@ceph.mon.b.smithi159.stdout: verify_compression: 0 2023-09-12T15:56:35.004 INFO:journalctl@ceph.mon.b.smithi159.stdout: read_amp_bytes_per_bit: 0 2023-09-12T15:56:35.004 INFO:journalctl@ceph.mon.b.smithi159.stdout: format_version: 5 2023-09-12T15:56:35.004 INFO:journalctl@ceph.mon.b.smithi159.stdout: enable_index_compression: 1 2023-09-12T15:56:35.004 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_align: 0 2023-09-12T15:56:35.004 INFO:journalctl@ceph.mon.b.smithi159.stdout: max_auto_readahead_size: 262144 2023-09-12T15:56:35.005 INFO:journalctl@ceph.mon.b.smithi159.stdout: prepopulate_block_cache: 0 2023-09-12T15:56:35.005 INFO:journalctl@ceph.mon.b.smithi159.stdout: initial_auto_readahead_size: 8192 2023-09-12T15:56:35.005 INFO:journalctl@ceph.mon.b.smithi159.stdout: num_file_reads_for_auto_readahead: 2 2023-09-12T15:56:35.005 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.write_buffer_size: 33554432 2023-09-12T15:56:35.005 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_write_buffer_number: 2 2023-09-12T15:56:35.006 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression: NoCompression 2023-09-12T15:56:35.006 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression: Disabled 2023-09-12T15:56:35.006 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.prefix_extractor: nullptr 2023-09-12T15:56:35.006 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-12T15:56:35.006 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.num_levels: 7 2023-09-12T15:56:35.006 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-12T15:56:35.007 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-12T15:56:35.007 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-12T15:56:35.007 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-12T15:56:35.007 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-12T15:56:35.007 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-12T15:56:35.009 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-12T15:56:35.009 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-12T15:56:35.009 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-12T15:56:35.009 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-12T15:56:35.010 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-12T15:56:35.010 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-12T15:56:35.010 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-12T15:56:35.010 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression_opts.level: 32767 2023-09-12T15:56:35.010 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression_opts.strategy: 0 2023-09-12T15:56:35.010 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-12T15:56:35.011 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-12T15:56:35.011 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-12T15:56:35.011 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-12T15:56:35.011 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression_opts.enabled: false 2023-09-12T15:56:35.011 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-12T15:56:35.012 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-12T15:56:35.012 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-12T15:56:35.012 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-12T15:56:35.012 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.target_file_size_base: 67108864 2023-09-12T15:56:35.012 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-12T15:56:35.013 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-12T15:56:35.013 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-12T15:56:35.013 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-12T15:56:35.013 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-12T15:56:35.013 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-12T15:56:35.013 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-12T15:56:35.014 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-12T15:56:35.014 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-12T15:56:35.014 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-12T15:56:35.014 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-12T15:56:35.014 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-12T15:56:35.014 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-12T15:56:35.015 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-12T15:56:35.015 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.arena_block_size: 1048576 2023-09-12T15:56:35.015 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-12T15:56:35.015 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-12T15:56:35.015 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.disable_auto_compactions: 0 2023-09-12T15:56:35.016 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-12T15:56:35.016 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-12T15:56:35.016 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-12T15:56:35.016 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-12T15:56:35.016 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-12T15:56:35.016 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-12T15:56:35.017 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-12T15:56:35.017 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-12T15:56:35.017 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-12T15:56:35.017 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-12T15:56:35.017 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.table_properties_collectors: 2023-09-12T15:56:35.018 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.inplace_update_support: 0 2023-09-12T15:56:35.018 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-12T15:56:35.018 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-12T15:56:35.018 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-12T15:56:35.018 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-12T15:56:35.018 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.bloom_locality: 0 2023-09-12T15:56:35.019 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.max_successive_merges: 0 2023-09-12T15:56:35.019 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-12T15:56:35.019 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.paranoid_file_checks: 0 2023-09-12T15:56:35.019 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.force_consistency_checks: 1 2023-09-12T15:56:35.019 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.report_bg_io_stats: 0 2023-09-12T15:56:35.020 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.ttl: 2592000 2023-09-12T15:56:35.020 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-12T15:56:35.020 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-12T15:56:35.020 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-12T15:56:35.020 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.enable_blob_files: false 2023-09-12T15:56:35.020 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.min_blob_size: 0 2023-09-12T15:56:35.021 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.blob_file_size: 268435456 2023-09-12T15:56:35.021 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-12T15:56:35.021 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-12T15:56:35.021 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-12T15:56:35.021 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-12T15:56:35.021 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-12T15:56:35.022 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.blob_file_starting_level: 0 2023-09-12T15:56:35.022 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-12T15:56:35.022 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-12T15:56:35.022 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-12T15:56:35.022 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cd43af45-b58a-4831-85b3-3549cbb3235f 2023-09-12T15:56:35.023 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694534194809950, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-12T15:56:35.023 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-12T15:56:35.023 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694534194810356, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694534194, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cd43af45-b58a-4831-85b3-3549cbb3235f", "db_session_id": "VW9ZZ8OT5P0T5JGXMV2X", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-12T15:56:35.023 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694534194810423, "job": 1, "event": "recovery_finished"} 2023-09-12T15:56:35.023 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-12T15:56:35.024 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-12T15:56:35.024 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561f8fe2c000 2023-09-12T15:56:35.024 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: DB pointer 0x561f8fe18000 2023-09-12T15:56:35.024 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-12T15:56:35.024 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-12T15:56:35.024 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** DB Stats ** 2023-09-12T15:56:35.025 INFO:journalctl@ceph.mon.b.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-12T15:56:35.025 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-12T15:56:35.025 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-12T15:56:35.025 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-12T15:56:35.025 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-12T15:56:35.026 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-12T15:56:35.026 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-12T15:56:35.026 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2023-09-12T15:56:35.026 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** Compaction Stats [default] ** 2023-09-12T15:56:35.026 INFO:journalctl@ceph.mon.b.smithi159.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-12T15:56:35.026 INFO:journalctl@ceph.mon.b.smithi159.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-12T15:56:35.027 INFO:journalctl@ceph.mon.b.smithi159.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-12T15:56:35.027 INFO:journalctl@ceph.mon.b.smithi159.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-12T15:56:35.027 INFO:journalctl@ceph.mon.b.smithi159.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-12T15:56:35.027 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2023-09-12T15:56:35.027 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** Compaction Stats [default] ** 2023-09-12T15:56:35.028 INFO:journalctl@ceph.mon.b.smithi159.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-12T15:56:35.028 INFO:journalctl@ceph.mon.b.smithi159.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-12T15:56:35.028 INFO:journalctl@ceph.mon.b.smithi159.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-12T15:56:35.028 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2023-09-12T15:56:35.028 INFO:journalctl@ceph.mon.b.smithi159.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-12T15:56:35.028 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2023-09-12T15:56:35.029 INFO:journalctl@ceph.mon.b.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-12T15:56:35.029 INFO:journalctl@ceph.mon.b.smithi159.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-12T15:56:35.029 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-12T15:56:35.029 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-12T15:56:35.029 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-12T15:56:35.030 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-12T15:56:35.030 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-12T15:56:35.030 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-12T15:56:35.030 INFO:journalctl@ceph.mon.b.smithi159.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-12T15:56:35.030 INFO:journalctl@ceph.mon.b.smithi159.stdout: Block cache BinnedLRUCache@0x561f8fd911f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-09-12T15:56:35.031 INFO:journalctl@ceph.mon.b.smithi159.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-12T15:56:35.031 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2023-09-12T15:56:35.031 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-12T15:56:35.031 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-09-12T15:56:35.031 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: using public_addr v2:172.21.15.159:0/0 -> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2023-09-12T15:56:35.031 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: starting mon.b rank -1 at public addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] at bind addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:56:35.032 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(???) e0 preinit fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:56:35.032 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 systemd[1]: Started Ceph mon.b for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T15:56:35.032 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).mds e1 new map 2023-09-12T15:56:35.032 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).mds e1 print_map 2023-09-12T15:56:35.032 INFO:journalctl@ceph.mon.b.smithi159.stdout: e1 2023-09-12T15:56:35.033 INFO:journalctl@ceph.mon.b.smithi159.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-12T15:56:35.033 INFO:journalctl@ceph.mon.b.smithi159.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-12T15:56:35.033 INFO:journalctl@ceph.mon.b.smithi159.stdout: legacy client fscid: -1 2023-09-12T15:56:35.033 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2023-09-12T15:56:35.033 INFO:journalctl@ceph.mon.b.smithi159.stdout: No filesystems configured 2023-09-12T15:56:35.033 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-12T15:56:35.034 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-12T15:56:35.034 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-12T15:56:35.034 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-12T15:56:35.034 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-12T15:56:35.034 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-12T15:56:35.035 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-12T15:56:35.035 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-12T15:56:35.035 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-12T15:56:35.035 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-12T15:56:35.035 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:35.035 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2643330924' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-12T15:56:35.036 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2643330924' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-12T15:56:35.036 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: osdmap e4: 0 total, 0 up, 0 in 2023-09-12T15:56:35.036 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.036 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.036 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.037 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.037 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:56:35.037 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:35.037 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:35.037 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: Updating smithi203:/etc/ceph/ceph.conf 2023-09-12T15:56:35.037 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: Updating smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:35.038 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:35.038 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:56:35.038 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: Updating smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-12T15:56:35.038 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.039 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.039 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.039 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:35.039 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi130:172.21.15.130=a;smithi159:172.21.15.159=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:56:35.039 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: Saving service mon spec with placement smithi130:172.21.15.130=a;smithi159:172.21.15.159=b;smithi203:172.21.15.203=c;count:3 2023-09-12T15:56:35.040 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.040 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:35.040 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:35.040 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:35.040 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.041 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:35.041 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:35.041 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: Deploying daemon mon.c on smithi203 2023-09-12T15:56:35.041 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:35.041 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:35.042 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:56:35.042 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.a calling monitor election 2023-09-12T15:56:35.042 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:35.042 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:35.042 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:35.043 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:35.043 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.c calling monitor election 2023-09-12T15:56:35.043 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:35.043 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:35.043 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:35.044 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:35.044 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-12T15:56:35.044 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: monmap e2: 2 mons at {a=[v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} 2023-09-12T15:56:35.044 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: fsmap 2023-09-12T15:56:35.044 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: osdmap e4: 0 total, 0 up, 0 in 2023-09-12T15:56:35.044 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mgrmap e13: x(active, since 41s) 2023-09-12T15:56:35.045 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: overall HEALTH_OK 2023-09-12T15:56:35.045 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.045 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.045 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:35.045 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:35.046 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:35.046 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: Deploying daemon mon.b on smithi159 2023-09-12T15:56:35.046 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:35.046 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:35.046 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/3948296732' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-12T15:56:35.046 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-12T15:56:35.047 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-09-12T15:56:35.047 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta expand map: {default=false} 2023-09-12T15:56:35.047 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta from 'false' to 'false' 2023-09-12T15:56:35.047 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta expanded map: {default=false} 2023-09-12T15:56:35.047 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta expand map: {default=info} 2023-09-12T15:56:35.048 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta from 'info' to 'info' 2023-09-12T15:56:35.048 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta expanded map: {default=info} 2023-09-12T15:56:35.048 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta expand map: {default=daemon} 2023-09-12T15:56:35.048 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-12T15:56:35.048 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta expanded map: {default=daemon} 2023-09-12T15:56:35.048 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta expand map: {default=debug} 2023-09-12T15:56:35.049 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta from 'debug' to 'debug' 2023-09-12T15:56:35.049 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:34 smithi159 ceph-mon[102522]: expand_channel_meta expanded map: {default=debug} 2023-09-12T15:56:40.161 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-09-12T15:56:40.161 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":3,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","modified":"2023-09-12T15:56:34.843708Z","created":"2023-09-12T15:55:00.069564Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-09-12T15:56:40.164 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 3 2023-09-12T15:56:40.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:40.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:56:40.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: mon.a calling monitor election 2023-09-12T15:56:40.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: mon.c calling monitor election 2023-09-12T15:56:40.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:40.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:40.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: mon.b calling monitor election 2023-09-12T15:56:40.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:40.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-12T15:56:40.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: monmap e3: 3 mons at {a=[v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0],b=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} 2023-09-12T15:56:40.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: fsmap 2023-09-12T15:56:40.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: osdmap e4: 0 total, 0 up, 0 in 2023-09-12T15:56:40.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: mgrmap e13: x(active, since 49s) 2023-09-12T15:56:40.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: overall HEALTH_OK 2023-09-12T15:56:40.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:40.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:40.231 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:40.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:40.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:56:40.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: mon.a calling monitor election 2023-09-12T15:56:40.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: mon.c calling monitor election 2023-09-12T15:56:40.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:40.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:40.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: mon.b calling monitor election 2023-09-12T15:56:40.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:40.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-12T15:56:40.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: monmap e3: 3 mons at {a=[v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0],b=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} 2023-09-12T15:56:40.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: fsmap 2023-09-12T15:56:40.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: osdmap e4: 0 total, 0 up, 0 in 2023-09-12T15:56:40.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: mgrmap e13: x(active, since 49s) 2023-09-12T15:56:40.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: overall HEALTH_OK 2023-09-12T15:56:40.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:40.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:40.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:56:40.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: mon.a calling monitor election 2023-09-12T15:56:40.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: mon.c calling monitor election 2023-09-12T15:56:40.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:56:40.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:40.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: mon.b calling monitor election 2023-09-12T15:56:40.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:40.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:40.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-12T15:56:40.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: monmap e3: 3 mons at {a=[v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0],b=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} 2023-09-12T15:56:40.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: fsmap 2023-09-12T15:56:40.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: osdmap e4: 0 total, 0 up, 0 in 2023-09-12T15:56:40.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: mgrmap e13: x(active, since 49s) 2023-09-12T15:56:40.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: overall HEALTH_OK 2023-09-12T15:56:40.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:40.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:40.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:40.575 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-09-12T15:56:40.576 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config generate-minimal-conf 2023-09-12T15:56:41.145 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:41.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:41.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:41.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:40 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/2915652954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-12T15:56:41.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:41.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:41.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:41.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:41.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:40 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/2915652954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-12T15:56:41.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:41.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:41.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:41.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:41.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:40 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/2915652954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-12T15:56:41.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:41.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:56:42.160 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:41 smithi130 ceph-mon[98670]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:42.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:42.162 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:42.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:41 smithi159 ceph-mon[102522]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:42.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:42.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:41 smithi203 ceph-mon[103431]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:42.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:42.556 INFO:teuthology.orchestra.run.smithi130.stdout:# minimal ceph.conf for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:56:42.556 INFO:teuthology.orchestra.run.smithi130.stdout:[global] 2023-09-12T15:56:42.556 INFO:teuthology.orchestra.run.smithi130.stdout: fsid = 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T15:56:42.556 INFO:teuthology.orchestra.run.smithi130.stdout: mon_host = [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2023-09-12T15:56:43.013 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'] 2023-09-12T15:56:43.014 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set client client_mount_timeout 600 2023-09-12T15:56:43.016 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'debug_client', '20'] 2023-09-12T15:56:43.016 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set client debug_client 20 2023-09-12T15:56:43.059 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'debug_ms', '1'] 2023-09-12T15:56:43.060 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set client debug_ms 1 2023-09-12T15:56:43.103 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'] 2023-09-12T15:56:43.103 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set client rados_mon_op_timeout 900 2023-09-12T15:56:43.147 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'] 2023-09-12T15:56:43.148 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set client rados_osd_op_timeout 900 2023-09-12T15:56:43.191 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_mds', '20'] 2023-09-12T15:56:43.192 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_mds 20 2023-09-12T15:56:43.223 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: Updating smithi130:/etc/ceph/ceph.conf 2023-09-12T15:56:43.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: Updating smithi159:/etc/ceph/ceph.conf 2023-09-12T15:56:43.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: Updating smithi203:/etc/ceph/ceph.conf 2023-09-12T15:56:43.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: Updating smithi130:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:43.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: Updating smithi159:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:43.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: Updating smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:43.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-12T15:56:43.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:43.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:42 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/861848452' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:43.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: Updating smithi130:/etc/ceph/ceph.conf 2023-09-12T15:56:43.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: Updating smithi159:/etc/ceph/ceph.conf 2023-09-12T15:56:43.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: Updating smithi203:/etc/ceph/ceph.conf 2023-09-12T15:56:43.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: Updating smithi130:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:43.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: Updating smithi159:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:43.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: Updating smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:43.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:43.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-12T15:56:43.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:43.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:42 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/861848452' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:43.232 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'] 2023-09-12T15:56:43.233 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_mds_balancer 20 2023-09-12T15:56:43.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: Updating smithi130:/etc/ceph/ceph.conf 2023-09-12T15:56:43.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: Updating smithi159:/etc/ceph/ceph.conf 2023-09-12T15:56:43.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: Updating smithi203:/etc/ceph/ceph.conf 2023-09-12T15:56:43.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: Updating smithi130:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:43.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: Updating smithi159:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:43.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: Updating smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-12T15:56:43.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:43.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:43.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-12T15:56:43.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:43.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:42 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/861848452' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:43.277 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_ms', '1'] 2023-09-12T15:56:43.278 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_ms 1 2023-09-12T15:56:43.321 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'] 2023-09-12T15:56:43.321 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_fragment_size_max 10000 2023-09-12T15:56:43.369 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'] 2023-09-12T15:56:43.370 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_merge_size 5 2023-09-12T15:56:43.414 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'] 2023-09-12T15:56:43.414 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_split_bits 3 2023-09-12T15:56:43.458 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'] 2023-09-12T15:56:43.458 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_split_size 100 2023-09-12T15:56:43.502 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'] 2023-09-12T15:56:43.503 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_debug_frag True 2023-09-12T15:56:43.552 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'] 2023-09-12T15:56:43.553 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_debug_scatterstat True 2023-09-12T15:56:43.613 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'] 2023-09-12T15:56:43.613 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_op_complaint_time 180 2023-09-12T15:56:43.692 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'] 2023-09-12T15:56:43.692 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_verify_scatter True 2023-09-12T15:56:43.753 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'] 2023-09-12T15:56:43.754 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds osd_op_complaint_time 180 2023-09-12T15:56:43.799 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'] 2023-09-12T15:56:43.799 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds rados_mon_op_timeout 900 2023-09-12T15:56:43.856 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'] 2023-09-12T15:56:43.856 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mds rados_osd_op_timeout 900 2023-09-12T15:56:43.872 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:43.878 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'] 2023-09-12T15:56:43.878 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set mon mon_op_complaint_time 120 2023-09-12T15:56:43.896 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:43.897 INFO:tasks.cephadm:['ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'] 2023-09-12T15:56:43.898 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config set osd osd_op_complaint_time 180 2023-09-12T15:56:43.946 DEBUG:tasks.cephadm:set 21 configs 2023-09-12T15:56:43.960 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:43.976 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:43.994 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:44.059 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:43 smithi130 ceph-mon[98670]: Reconfiguring mon.a (unknown last config time)... 2023-09-12T15:56:44.060 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:43 smithi130 ceph-mon[98670]: Reconfiguring daemon mon.a on smithi130 2023-09-12T15:56:44.060 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:43 smithi130 ceph-mon[98670]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:44.060 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:44.060 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:44.061 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:44.061 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-12T15:56:44.061 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:44.126 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:44.208 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:44.214 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:44.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:43 smithi159 ceph-mon[102522]: Reconfiguring mon.a (unknown last config time)... 2023-09-12T15:56:44.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:43 smithi159 ceph-mon[102522]: Reconfiguring daemon mon.a on smithi130 2023-09-12T15:56:44.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:43 smithi159 ceph-mon[102522]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:44.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:44.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:44.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:44.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-12T15:56:44.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:44.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:43 smithi203 ceph-mon[103431]: Reconfiguring mon.a (unknown last config time)... 2023-09-12T15:56:44.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:43 smithi203 ceph-mon[103431]: Reconfiguring daemon mon.a on smithi130 2023-09-12T15:56:44.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:43 smithi203 ceph-mon[103431]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:44.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:44.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:44.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:44.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-12T15:56:44.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:44.600 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:44.601 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:45.020 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:44 smithi159 ceph-mon[102522]: Reconfiguring mon.b (monmap changed)... 2023-09-12T15:56:45.020 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:44 smithi159 ceph-mon[102522]: Reconfiguring daemon mon.b on smithi159 2023-09-12T15:56:45.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:44 smithi130 ceph-mon[98670]: Reconfiguring mon.b (monmap changed)... 2023-09-12T15:56:45.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:44 smithi130 ceph-mon[98670]: Reconfiguring daemon mon.b on smithi159 2023-09-12T15:56:45.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:44 smithi203 ceph-mon[103431]: Reconfiguring mon.b (monmap changed)... 2023-09-12T15:56:45.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:44 smithi203 ceph-mon[103431]: Reconfiguring daemon mon.b on smithi159 2023-09-12T15:56:46.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:45 smithi203 ceph-mon[103431]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:46.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:46.185 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:46.185 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:46.185 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-12T15:56:46.185 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:46.190 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:45 smithi130 ceph-mon[98670]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:46.190 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:46.190 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:46.191 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:46.191 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-12T15:56:46.191 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:46.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:45 smithi159 ceph-mon[102522]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:46.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:46.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:46.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-12T15:56:46.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-12T15:56:46.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:47.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:46 smithi203 ceph-mon[103431]: Reconfiguring mon.c (monmap changed)... 2023-09-12T15:56:47.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:46 smithi203 ceph-mon[103431]: Reconfiguring daemon mon.c on smithi203 2023-09-12T15:56:47.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:47.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:47.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:47.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:47.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:47.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:47.395 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:46 smithi130 ceph-mon[98670]: Reconfiguring mon.c (monmap changed)... 2023-09-12T15:56:47.395 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:46 smithi130 ceph-mon[98670]: Reconfiguring daemon mon.c on smithi203 2023-09-12T15:56:47.395 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:47.395 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:47.396 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:47.396 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:47.396 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:47.396 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:47.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:46 smithi159 ceph-mon[102522]: Reconfiguring mon.c (monmap changed)... 2023-09-12T15:56:47.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:46 smithi159 ceph-mon[102522]: Reconfiguring daemon mon.c on smithi203 2023-09-12T15:56:47.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:47.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:47.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:56:47.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:56:47.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:56:47.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:56:47.579 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:48.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:47 smithi203 ceph-mon[103431]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:48.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:47 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1085240081' entity='client.admin' 2023-09-12T15:56:48.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:47 smithi130 ceph-mon[98670]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:48.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:47 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1085240081' entity='client.admin' 2023-09-12T15:56:48.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:47 smithi159 ceph-mon[102522]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:48.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:47 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1085240081' entity='client.admin' 2023-09-12T15:56:50.151 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:49 smithi130 ceph-mon[98670]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:50.152 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:49 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1187429763' entity='client.admin' 2023-09-12T15:56:50.152 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:49 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/396128779' entity='client.admin' 2023-09-12T15:56:50.152 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:49 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3755557858' entity='client.admin' 2023-09-12T15:56:50.152 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:49 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3685537845' entity='client.admin' 2023-09-12T15:56:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:49 smithi203 ceph-mon[103431]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:49 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1187429763' entity='client.admin' 2023-09-12T15:56:50.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:49 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/396128779' entity='client.admin' 2023-09-12T15:56:50.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:49 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3755557858' entity='client.admin' 2023-09-12T15:56:50.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:49 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3685537845' entity='client.admin' 2023-09-12T15:56:50.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:49 smithi159 ceph-mon[102522]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:50.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:49 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1187429763' entity='client.admin' 2023-09-12T15:56:50.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:49 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/396128779' entity='client.admin' 2023-09-12T15:56:50.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:49 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3755557858' entity='client.admin' 2023-09-12T15:56:50.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:49 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3685537845' entity='client.admin' 2023-09-12T15:56:50.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 -- 172.21.15.130:0/3489790599 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9050038630 msgr2=0x7f905003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:50.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 --2- 172.21.15.130:0/3489790599 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9050038630 0x7f905003aaf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f904c00b2e0 tx=0x7f904c005fb0 comp rx=0 tx=0).stop 2023-09-12T15:56:50.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 -- 172.21.15.130:0/3489790599 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9060100290 msgr2=0x7f9060195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:50.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 --2- 172.21.15.130:0/3489790599 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9060100290 0x7f9060195470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f905400d950 tx=0x7f905400dd10 comp rx=0 tx=0).stop 2023-09-12T15:56:50.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 -- 172.21.15.130:0/3489790599 shutdown_connections 2023-09-12T15:56:50.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 --2- 172.21.15.130:0/3489790599 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9050038630 0x7f905003aaf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:50.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 --2- 172.21.15.130:0/3489790599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90601038a0 0x7f9060199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:50.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 --2- 172.21.15.130:0/3489790599 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9060100290 0x7f9060195470 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:50.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 --2- 172.21.15.130:0/3489790599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9060100c40 0x7f90601959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:50.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 -- 172.21.15.130:0/3489790599 >> 172.21.15.130:0/3489790599 conn(0x7f9060075a00 msgr2=0x7f90601021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:50.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 -- 172.21.15.130:0/3489790599 shutdown_connections 2023-09-12T15:56:50.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:50.951+0000 7f9066e08700 1 -- 172.21.15.130:0/3489790599 wait complete. 2023-09-12T15:56:51.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.139+0000 7fbc1c70a700 1 -- 172.21.15.130:0/1518234233 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7fbbfc0051e0 con 0x7fbc14104530 2023-09-12T15:56:51.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.142+0000 7fbc0b7fe700 1 -- 172.21.15.130:0/1518234233 <== mon.0 v2:172.21.15.130:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v17)=0 v17) v1 ==== 108+0+0 (secure 0 0 0) 0x7fbc0c016070 con 0x7fbc14104530 2023-09-12T15:56:51.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 -- 172.21.15.130:0/1518234233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbc00038630 msgr2=0x7fbc0003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:51.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 --2- 172.21.15.130:0/1518234233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbc00038630 0x7fbc0003aaf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc14069c10 tx=0x7fbc10007040 comp rx=0 tx=0).stop 2023-09-12T15:56:51.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 -- 172.21.15.130:0/1518234233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbc14104530 msgr2=0x7fbc1419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:51.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 --2- 172.21.15.130:0/1518234233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbc14104530 0x7fbc1419af60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00cca0 tx=0x7fbc0c00a060 comp rx=0 tx=0).stop 2023-09-12T15:56:51.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 -- 172.21.15.130:0/1518234233 shutdown_connections 2023-09-12T15:56:51.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 --2- 172.21.15.130:0/1518234233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbc00038630 0x7fbc0003aaf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 --2- 172.21.15.130:0/1518234233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbc14104530 0x7fbc1419af60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 --2- 172.21.15.130:0/1518234233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc140ff520 0x7fbc141965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 --2- 172.21.15.130:0/1518234233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc140ffee0 0x7fbc14196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.144+0000 7fbc1c70a700 1 -- 172.21.15.130:0/1518234233 >> 172.21.15.130:0/1518234233 conn(0x7fbc14076600 msgr2=0x7fbc14068160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:51.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.145+0000 7fbc1c70a700 1 -- 172.21.15.130:0/1518234233 shutdown_connections 2023-09-12T15:56:51.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.145+0000 7fbc1c70a700 1 -- 172.21.15.130:0/1518234233 wait complete. 2023-09-12T15:56:51.385 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:51.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.441+0000 7f60ea343700 1 -- 172.21.15.130:0/4151803930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e40ff880 msgr2=0x7f60e40ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:51.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.441+0000 7f60ea343700 1 --2- 172.21.15.130:0/4151803930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e40ff880 0x7f60e40ffc60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009990 tx=0x7f60d4009ca0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.441+0000 7f60ea343700 1 -- 172.21.15.130:0/4151803930 shutdown_connections 2023-09-12T15:56:51.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.441+0000 7f60ea343700 1 --2- 172.21.15.130:0/4151803930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60e4101840 0x7f60e410c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.441+0000 7f60ea343700 1 --2- 172.21.15.130:0/4151803930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e40ff880 0x7f60e40ffc60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.441+0000 7f60ea343700 1 --2- 172.21.15.130:0/4151803930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60e41001a0 0x7f60e4101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.441+0000 7f60ea343700 1 -- 172.21.15.130:0/4151803930 >> 172.21.15.130:0/4151803930 conn(0x7f60e40765f0 msgr2=0x7f60e4076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:51.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.441+0000 7f60ea343700 1 -- 172.21.15.130:0/4151803930 shutdown_connections 2023-09-12T15:56:51.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.441+0000 7f60ea343700 1 -- 172.21.15.130:0/4151803930 wait complete. 2023-09-12T15:56:51.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.442+0000 7f60ea343700 1 Processor -- start 2023-09-12T15:56:51.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.442+0000 7f60ea343700 1 -- start start 2023-09-12T15:56:51.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.442+0000 7f60ea343700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60e40ff880 0x7f60e4197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:51.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.442+0000 7f60ea343700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e41001a0 0x7f60e4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:51.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.442+0000 7f60ea343700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60e4101840 0x7f60e41a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:51.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.442+0000 7f60ea343700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41983c0 con 0x7f60e41001a0 2023-09-12T15:56:51.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.442+0000 7f60ea343700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41a2b30 con 0x7f60e40ff880 2023-09-12T15:56:51.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.442+0000 7f60ea343700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41a2ca0 con 0x7f60e4101840 2023-09-12T15:56:51.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e37fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e41001a0 0x7f60e4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:51.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e37fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e41001a0 0x7f60e4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52216/0 (socket says 172.21.15.130:52216) 2023-09-12T15:56:51.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e37fe700 1 -- 172.21.15.130:0/2043200152 learned_addr learned my addr 172.21.15.130:0/2043200152 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:56:51.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e88e0700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60e4101840 0x7f60e41a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:51.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e37fe700 1 -- 172.21.15.130:0/2043200152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60e4101840 msgr2=0x7f60e41a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:51.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e37fe700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60e4101840 0x7f60e41a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e37fe700 1 -- 172.21.15.130:0/2043200152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60e40ff880 msgr2=0x7f60e4197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:51.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e37fe700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60e40ff880 0x7f60e4197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e37fe700 1 -- 172.21.15.130:0/2043200152 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d40095f0 con 0x7f60e41001a0 2023-09-12T15:56:51.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e37fe700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e41001a0 0x7f60e4197ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60d800d980 tx=0x7f60d800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:51.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e17fa700 1 -- 172.21.15.130:0/2043200152 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8009b40 con 0x7f60e41001a0 2023-09-12T15:56:51.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e17fa700 1 -- 172.21.15.130:0/2043200152 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(25 keys) v1 ==== 1080+0+0 (secure 0 0 0) 0x7f60d800de60 con 0x7f60e41001a0 2023-09-12T15:56:51.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e41a2f80 con 0x7f60e41001a0 2023-09-12T15:56:51.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60e17fa700 1 -- 172.21.15.130:0/2043200152 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d800f620 con 0x7f60e41001a0 2023-09-12T15:56:51.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.443+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e41a33f0 con 0x7f60e41001a0 2023-09-12T15:56:51.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.444+0000 7f60e17fa700 1 -- 172.21.15.130:0/2043200152 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f60d800f780 con 0x7f60e41001a0 2023-09-12T15:56:51.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.444+0000 7f60e17fa700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f60cc038630 0x7f60cc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:51.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.444+0000 7f60e17fa700 1 -- 172.21.15.130:0/2043200152 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f60d804aa50 con 0x7f60e41001a0 2023-09-12T15:56:51.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.444+0000 7f60e3fff700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f60cc038630 0x7f60cc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:51.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.444+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e404e300 con 0x7f60e41001a0 2023-09-12T15:56:51.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.445+0000 7f60e3fff700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f60cc038630 0x7f60cc03aaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f60d40095c0 tx=0x7f60d4009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:51.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.449+0000 7f60e17fa700 1 -- 172.21.15.130:0/2043200152 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f60d8020080 con 0x7f60e41001a0 2023-09-12T15:56:51.560 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.098+0000 7f0cdecd3700 1 -- 172.21.15.130:0/1245305836 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7f0cd8197820 con 0x7f0cd8106770 2023-09-12T15:56:51.560 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.101+0000 7f0cd5ffb700 1 -- 172.21.15.130:0/1245305836 <== mon.0 v2:172.21.15.130:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v16)=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0cd0016070 con 0x7f0cd8106770 2023-09-12T15:56:51.560 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 -- 172.21.15.130:0/1245305836 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0cc8038630 msgr2=0x7f0cc803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:51.561 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 --2- 172.21.15.130:0/1245305836 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0cc8038630 0x7f0cc803aaf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc006010 tx=0x7f0ccc00b260 comp rx=0 tx=0).stop 2023-09-12T15:56:51.561 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 -- 172.21.15.130:0/1245305836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0cd8106770 msgr2=0x7f0cd819ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:51.561 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 --2- 172.21.15.130:0/1245305836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0cd8106770 0x7f0cd819ae70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000cca0 tx=0x7f0cd000a060 comp rx=0 tx=0).stop 2023-09-12T15:56:51.561 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 -- 172.21.15.130:0/1245305836 shutdown_connections 2023-09-12T15:56:51.561 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 --2- 172.21.15.130:0/1245305836 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0cc8038630 0x7f0cc803aaf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.561 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 --2- 172.21.15.130:0/1245305836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0cd8106770 0x7f0cd819ae70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 --2- 172.21.15.130:0/1245305836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cd8068120 0x7f0cd81964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 --2- 172.21.15.130:0/1245305836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cd8068ae0 0x7f0cd8196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 -- 172.21.15.130:0/1245305836 >> 172.21.15.130:0/1245305836 conn(0x7f0cd8076600 msgr2=0x7f0cd80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:51.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 -- 172.21.15.130:0/1245305836 shutdown_connections 2023-09-12T15:56:51.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.104+0000 7f0cdecd3700 1 -- 172.21.15.130:0/1245305836 wait complete. 2023-09-12T15:56:51.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.661+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) v1 -- 0x7f60e410f4d0 con 0x7f60e41001a0 2023-09-12T15:56:51.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.664+0000 7f60e17fa700 1 -- 172.21.15.130:0/2043200152 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v18)=0 v18) v1 ==== 114+0+0 (secure 0 0 0) 0x7f60d801a950 con 0x7f60e41001a0 2023-09-12T15:56:51.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.665+0000 7f60e17fa700 1 -- 172.21.15.130:0/2043200152 <== mon.0 v2:172.21.15.130:3300/0 8 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60d801a950 con 0x7f60e41001a0 2023-09-12T15:56:51.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.667+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f60cc038630 msgr2=0x7f60cc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:51.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.667+0000 7f60ea343700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f60cc038630 0x7f60cc03aaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f60d40095c0 tx=0x7f60d4009530 comp rx=0 tx=0).stop 2023-09-12T15:56:51.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.667+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e41001a0 msgr2=0x7f60e4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:51.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.667+0000 7f60ea343700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e41001a0 0x7f60e4197ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60d800d980 tx=0x7f60d800dd40 comp rx=0 tx=0).stop 2023-09-12T15:56:51.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.667+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 shutdown_connections 2023-09-12T15:56:51.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.668+0000 7f60ea343700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f60cc038630 0x7f60cc03aaf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.668+0000 7f60ea343700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60e4101840 0x7f60e41a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.668+0000 7f60ea343700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60e40ff880 0x7f60e4197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.668+0000 7f60ea343700 1 --2- 172.21.15.130:0/2043200152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60e41001a0 0x7f60e4197ab0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:51.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.668+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 >> 172.21.15.130:0/2043200152 conn(0x7f60e40765f0 msgr2=0x7f60e40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:51.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.668+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 shutdown_connections 2023-09-12T15:56:51.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:51.668+0000 7f60ea343700 1 -- 172.21.15.130:0/2043200152 wait complete. 2023-09-12T15:56:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:51 smithi130 ceph-mon[98670]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:51 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3489790599' entity='client.admin' 2023-09-12T15:56:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:51 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1245305836' entity='client.admin' 2023-09-12T15:56:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:51 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1518234233' entity='client.admin' 2023-09-12T15:56:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:51 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2043200152' entity='client.admin' 2023-09-12T15:56:52.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:51 smithi159 ceph-mon[102522]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:52.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:51 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3489790599' entity='client.admin' 2023-09-12T15:56:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:51 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1245305836' entity='client.admin' 2023-09-12T15:56:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:51 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1518234233' entity='client.admin' 2023-09-12T15:56:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:51 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2043200152' entity='client.admin' 2023-09-12T15:56:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:51 smithi203 ceph-mon[103431]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:51 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3489790599' entity='client.admin' 2023-09-12T15:56:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:51 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1245305836' entity='client.admin' 2023-09-12T15:56:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:51 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1518234233' entity='client.admin' 2023-09-12T15:56:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:51 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2043200152' entity='client.admin' 2023-09-12T15:56:53.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.052+0000 7fe207641700 1 -- 172.21.15.130:0/3604140726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200100670 msgr2=0x7fe200100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.052+0000 7fe207641700 1 --2- 172.21.15.130:0/3604140726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200100670 0x7fe200100af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009990 tx=0x7fe1fc009ca0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.052+0000 7fe207641700 1 -- 172.21.15.130:0/3604140726 shutdown_connections 2023-09-12T15:56:53.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.052+0000 7fe207641700 1 --2- 172.21.15.130:0/3604140726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200101030 0x7fe20010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.052+0000 7fe207641700 1 --2- 172.21.15.130:0/3604140726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200106670 0x7fe200106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.052+0000 7fe207641700 1 --2- 172.21.15.130:0/3604140726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200100670 0x7fe200100af0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.052+0000 7fe207641700 1 -- 172.21.15.130:0/3604140726 >> 172.21.15.130:0/3604140726 conn(0x7fe2000fc160 msgr2=0x7fe2000fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:53.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.052+0000 7fe207641700 1 -- 172.21.15.130:0/3604140726 shutdown_connections 2023-09-12T15:56:53.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.052+0000 7fe207641700 1 -- 172.21.15.130:0/3604140726 wait complete. 2023-09-12T15:56:53.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe207641700 1 Processor -- start 2023-09-12T15:56:53.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe207641700 1 -- start start 2023-09-12T15:56:53.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe207641700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200100670 0x7fe200198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:53.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe207641700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200101030 0x7fe200198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:53.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe207641700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200106670 0x7fe20019d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:53.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe207641700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019d930 con 0x7fe200106670 2023-09-12T15:56:53.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe207641700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019daa0 con 0x7fe200100670 2023-09-12T15:56:53.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe207641700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019dc60 con 0x7fe200101030 2023-09-12T15:56:53.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe205bde700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200106670 0x7fe20019d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:53.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe2053dd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200100670 0x7fe200198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:53.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe205bde700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200106670 0x7fe20019d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52230/0 (socket says 172.21.15.130:52230) 2023-09-12T15:56:53.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe205bde700 1 -- 172.21.15.130:0/3040376893 learned_addr learned my addr 172.21.15.130:0/3040376893 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:56:53.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe204bdc700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200101030 0x7fe200198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:53.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe205bde700 1 -- 172.21.15.130:0/3040376893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200101030 msgr2=0x7fe200198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe205bde700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200101030 0x7fe200198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe205bde700 1 -- 172.21.15.130:0/3040376893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200100670 msgr2=0x7fe200198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe205bde700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200100670 0x7fe200198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.053+0000 7fe205bde700 1 -- 172.21.15.130:0/3040376893 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0095f0 con 0x7fe200106670 2023-09-12T15:56:53.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.054+0000 7fe205bde700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200106670 0x7fe20019d360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800ada0 tx=0x7fe1f800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:53.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.054+0000 7fe1f67fc700 1 -- 172.21.15.130:0/3040376893 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800f070 con 0x7fe200106670 2023-09-12T15:56:53.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.054+0000 7fe1f67fc700 1 -- 172.21.15.130:0/3040376893 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1f8007b30 con 0x7fe200106670 2023-09-12T15:56:53.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.054+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20019df40 con 0x7fe200106670 2023-09-12T15:56:53.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.054+0000 7fe1f67fc700 1 -- 172.21.15.130:0/3040376893 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8005280 con 0x7fe200106670 2023-09-12T15:56:53.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.054+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2001a57e0 con 0x7fe200106670 2023-09-12T15:56:53.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.054+0000 7fe1f67fc700 1 -- 172.21.15.130:0/3040376893 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7fe1f80053e0 con 0x7fe200106670 2023-09-12T15:56:53.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.055+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20004e300 con 0x7fe200106670 2023-09-12T15:56:53.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.055+0000 7fe1f67fc700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec038630 0x7fe1ec03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:53.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.055+0000 7fe1f67fc700 1 -- 172.21.15.130:0/3040376893 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe1f804a370 con 0x7fe200106670 2023-09-12T15:56:53.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.055+0000 7fe2053dd700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec038630 0x7fe1ec03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:53.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.055+0000 7fe2053dd700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec038630 0x7fe1ec03aaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005950 tx=0x7fe1f000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:53.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.060+0000 7fe1f67fc700 1 -- 172.21.15.130:0/3040376893 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe1f801a310 con 0x7fe200106670 2023-09-12T15:56:53.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.160+0000 7f34eca8d700 1 -- 172.21.15.130:0/4099013481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8068d80 msgr2=0x7f34e8069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.160+0000 7f34eca8d700 1 --2- 172.21.15.130:0/4099013481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8068d80 0x7f34e8069160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009990 tx=0x7f34d8009ca0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.160+0000 7f34eca8d700 1 -- 172.21.15.130:0/4099013481 shutdown_connections 2023-09-12T15:56:53.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.160+0000 7f34eca8d700 1 --2- 172.21.15.130:0/4099013481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34e8102b10 0x7f34e810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.160+0000 7f34eca8d700 1 --2- 172.21.15.130:0/4099013481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8068d80 0x7f34e8069160 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.160+0000 7f34eca8d700 1 --2- 172.21.15.130:0/4099013481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8102150 0x7f34e81025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.160+0000 7f34eca8d700 1 -- 172.21.15.130:0/4099013481 >> 172.21.15.130:0/4099013481 conn(0x7f34e80765f0 msgr2=0x7f34e8076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:53.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.160+0000 7f34eca8d700 1 -- 172.21.15.130:0/4099013481 shutdown_connections 2023-09-12T15:56:53.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.160+0000 7f34eca8d700 1 -- 172.21.15.130:0/4099013481 wait complete. 2023-09-12T15:56:53.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.161+0000 7f34eca8d700 1 Processor -- start 2023-09-12T15:56:53.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.161+0000 7f34eca8d700 1 -- start start 2023-09-12T15:56:53.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.161+0000 7f34eca8d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34e8068d80 0x7f34e819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:53.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34eca8d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8102150 0x7f34e819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:53.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34eca8d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8102b10 0x7f34e81965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:53.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34eca8d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e8196c40 con 0x7f34e8102150 2023-09-12T15:56:53.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34eca8d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e8196db0 con 0x7f34e8068d80 2023-09-12T15:56:53.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34eca8d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e8196f70 con 0x7f34e8102b10 2023-09-12T15:56:53.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e5d9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8102150 0x7f34e819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:53.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e5d9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8102150 0x7f34e819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52242/0 (socket says 172.21.15.130:52242) 2023-09-12T15:56:53.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e5d9b700 1 -- 172.21.15.130:0/1462718049 learned_addr learned my addr 172.21.15.130:0/1462718049 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:56:53.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e659c700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34e8068d80 0x7f34e819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:53.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e6d9d700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8102b10 0x7f34e81965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:53.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e5d9b700 1 -- 172.21.15.130:0/1462718049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8102b10 msgr2=0x7f34e81965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e5d9b700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8102b10 0x7f34e81965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e5d9b700 1 -- 172.21.15.130:0/1462718049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34e8068d80 msgr2=0x7f34e819c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e5d9b700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34e8068d80 0x7f34e819c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e5d9b700 1 -- 172.21.15.130:0/1462718049 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d80095f0 con 0x7f34e8102150 2023-09-12T15:56:53.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34e5d9b700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8102150 0x7f34e819ca90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00d950 tx=0x7f34dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:53.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34d77fe700 1 -- 172.21.15.130:0/1462718049 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc009940 con 0x7f34e8102150 2023-09-12T15:56:53.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e8197250 con 0x7f34e8102150 2023-09-12T15:56:53.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34d77fe700 1 -- 172.21.15.130:0/1462718049 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34dc010460 con 0x7f34e8102150 2023-09-12T15:56:53.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34d77fe700 1 -- 172.21.15.130:0/1462718049 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00f620 con 0x7f34e8102150 2023-09-12T15:56:53.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.162+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e81a3540 con 0x7f34e8102150 2023-09-12T15:56:53.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.163+0000 7f34d77fe700 1 -- 172.21.15.130:0/1462718049 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f34dc00f780 con 0x7f34e8102150 2023-09-12T15:56:53.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.163+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e804e300 con 0x7f34e8102150 2023-09-12T15:56:53.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.163+0000 7f34d77fe700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f34d0038630 0x7f34d003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:53.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.163+0000 7f34d77fe700 1 -- 172.21.15.130:0/1462718049 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f34dc04ac50 con 0x7f34e8102150 2023-09-12T15:56:53.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.164+0000 7f34e659c700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f34d0038630 0x7f34d003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:53.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.164+0000 7f34e659c700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f34d0038630 0x7f34d003aaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f34d800b5c0 tx=0x7f34d8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:53.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.168+0000 7f34d77fe700 1 -- 172.21.15.130:0/1462718049 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f34dc050020 con 0x7f34e8102150 2023-09-12T15:56:53.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.259+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7fe2001108e0 con 0x7fe200106670 2023-09-12T15:56:53.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.265+0000 7fe1f67fc700 1 -- 172.21.15.130:0/3040376893 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v19)=0 v19) v1 ==== 117+0+0 (secure 0 0 0) 0x7fe1f8012b80 con 0x7fe200106670 2023-09-12T15:56:53.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.268+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec038630 msgr2=0x7fe1ec03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.268+0000 7fe207641700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec038630 0x7fe1ec03aaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005950 tx=0x7fe1f000a300 comp rx=0 tx=0).stop 2023-09-12T15:56:53.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.268+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200106670 msgr2=0x7fe20019d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.268+0000 7fe207641700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200106670 0x7fe20019d360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800ada0 tx=0x7fe1f800c330 comp rx=0 tx=0).stop 2023-09-12T15:56:53.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.268+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 shutdown_connections 2023-09-12T15:56:53.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.269+0000 7fe207641700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec038630 0x7fe1ec03aaf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.269+0000 7fe207641700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200106670 0x7fe20019d360 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.269+0000 7fe207641700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200100670 0x7fe200198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.269+0000 7fe207641700 1 --2- 172.21.15.130:0/3040376893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200101030 0x7fe200198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.269+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 >> 172.21.15.130:0/3040376893 conn(0x7fe2000fc160 msgr2=0x7fe200072930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:53.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.269+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 shutdown_connections 2023-09-12T15:56:53.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.269+0000 7fe207641700 1 -- 172.21.15.130:0/3040376893 wait complete. 2023-09-12T15:56:53.362 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:53.370 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:53.380 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:53.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.385+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7f34e8110650 con 0x7f34e8102150 2023-09-12T15:56:53.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.393+0000 7f34d77fe700 1 -- 172.21.15.130:0/1462718049 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v20)=0 v20) v1 ==== 116+0+0 (secure 0 0 0) 0x7f34dc0226c0 con 0x7f34e8102150 2023-09-12T15:56:53.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.404+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f34d0038630 msgr2=0x7f34d003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.404+0000 7f34eca8d700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f34d0038630 0x7f34d003aaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f34d800b5c0 tx=0x7f34d8005fd0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.405+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8102150 msgr2=0x7f34e819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.405+0000 7f34eca8d700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8102150 0x7f34e819ca90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00d950 tx=0x7f34dc00dc60 comp rx=0 tx=0).stop 2023-09-12T15:56:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.405+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 shutdown_connections 2023-09-12T15:56:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.405+0000 7f34eca8d700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f34d0038630 0x7f34d003aaf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.405+0000 7f34eca8d700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8102b10 0x7f34e81965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.405+0000 7f34eca8d700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34e8068d80 0x7f34e819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.405+0000 7f34eca8d700 1 --2- 172.21.15.130:0/1462718049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e8102150 0x7f34e819ca90 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00d950 tx=0x7f34dc00dc60 comp rx=0 tx=0).stop 2023-09-12T15:56:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.405+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 >> 172.21.15.130:0/1462718049 conn(0x7f34e80765f0 msgr2=0x7f34e81054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.409+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 shutdown_connections 2023-09-12T15:56:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:53.409+0000 7f34eca8d700 1 -- 172.21.15.130:0/1462718049 wait complete. 2023-09-12T15:56:53.613 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:54.112 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:54.141 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:54.180 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:54.182 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:54.207 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:53 smithi130 ceph-mon[98670]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:54.208 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:53 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3040376893' entity='client.admin' 2023-09-12T15:56:54.208 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:53 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1462718049' entity='client.admin' 2023-09-12T15:56:54.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:53 smithi159 ceph-mon[102522]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:54.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:53 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3040376893' entity='client.admin' 2023-09-12T15:56:54.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:53 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1462718049' entity='client.admin' 2023-09-12T15:56:54.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:53 smithi203 ceph-mon[103431]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:53 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3040376893' entity='client.admin' 2023-09-12T15:56:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:53 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1462718049' entity='client.admin' 2023-09-12T15:56:54.525 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:56:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:55 smithi130 ceph-mon[98670]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:56.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:55 smithi159 ceph-mon[102522]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:56.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:55 smithi203 ceph-mon[103431]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:57.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.126+0000 7f89c8ad5700 1 -- 172.21.15.130:0/436615075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c4101500 msgr2=0x7f89c40feaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.126+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/436615075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c4101500 0x7f89c40feaf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009990 tx=0x7f89b8009ca0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.127+0000 7f89c8ad5700 1 -- 172.21.15.130:0/436615075 shutdown_connections 2023-09-12T15:56:57.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.127+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/436615075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c410d270 0x7f89c410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.127+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/436615075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c4101500 0x7f89c40feaf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.127+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/436615075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c4101970 0x7f89c40ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.127+0000 7f89c8ad5700 1 -- 172.21.15.130:0/436615075 >> 172.21.15.130:0/436615075 conn(0x7f89c4075a70 msgr2=0x7f89c40fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:57.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.127+0000 7f89c8ad5700 1 -- 172.21.15.130:0/436615075 shutdown_connections 2023-09-12T15:56:57.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.127+0000 7f89c8ad5700 1 -- 172.21.15.130:0/436615075 wait complete. 2023-09-12T15:56:57.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c8ad5700 1 Processor -- start 2023-09-12T15:56:57.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c8ad5700 1 -- start start 2023-09-12T15:56:57.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c8ad5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c4101500 0x7f89c419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:57.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c8ad5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c4101970 0x7f89c419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:57.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c8ad5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c410d270 0x7f89c41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:57.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c8ad5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4198d90 con 0x7f89c410d270 2023-09-12T15:56:57.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c8ad5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4198f00 con 0x7f89c4101970 2023-09-12T15:56:57.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c8ad5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c41990c0 con 0x7f89c4101500 2023-09-12T15:56:57.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c2d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c410d270 0x7f89c41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:57.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c1d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c4101970 0x7f89c419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:57.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c2d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c410d270 0x7f89c41986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52258/0 (socket says 172.21.15.130:52258) 2023-09-12T15:56:57.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c2d9d700 1 -- 172.21.15.130:0/2333249076 learned_addr learned my addr 172.21.15.130:0/2333249076 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:56:57.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c1d9b700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c4101970 0x7f89c419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:35514/0 (socket says 172.21.15.130:35514) 2023-09-12T15:56:57.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c259c700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c4101500 0x7f89c419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:57.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c2d9d700 1 -- 172.21.15.130:0/2333249076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c4101500 msgr2=0x7f89c419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c2d9d700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c4101500 0x7f89c419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c2d9d700 1 -- 172.21.15.130:0/2333249076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c4101970 msgr2=0x7f89c419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c2d9d700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c4101970 0x7f89c419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c2d9d700 1 -- 172.21.15.130:0/2333249076 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b80095f0 con 0x7f89c410d270 2023-09-12T15:56:57.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.128+0000 7f89c2d9d700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c410d270 0x7f89c41986e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f89bc007d40 tx=0x7f89bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:57.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.129+0000 7f89b37fe700 1 -- 172.21.15.130:0/2333249076 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00ac90 con 0x7f89c410d270 2023-09-12T15:56:57.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.129+0000 7f89b37fe700 1 -- 172.21.15.130:0/2333249076 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89bc007e90 con 0x7f89c410d270 2023-09-12T15:56:57.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.129+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c41993a0 con 0x7f89c410d270 2023-09-12T15:56:57.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.129+0000 7f89b37fe700 1 -- 172.21.15.130:0/2333249076 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0115e0 con 0x7f89c410d270 2023-09-12T15:56:57.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.129+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c41a5670 con 0x7f89c410d270 2023-09-12T15:56:57.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.129+0000 7f89b37fe700 1 -- 172.21.15.130:0/2333249076 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f89bc011780 con 0x7f89c410d270 2023-09-12T15:56:57.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.130+0000 7f89b37fe700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89ac038630 0x7f89ac03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:57.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.130+0000 7f89b37fe700 1 -- 172.21.15.130:0/2333249076 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f89bc04b040 con 0x7f89c410d270 2023-09-12T15:56:57.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.130+0000 7f89c259c700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89ac038630 0x7f89ac03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:57.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.130+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c404e300 con 0x7f89c410d270 2023-09-12T15:56:57.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.130+0000 7f89c259c700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89ac038630 0x7f89ac03aaf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f89b800b5c0 tx=0x7f89b8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:57.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.135+0000 7f89b37fe700 1 -- 172.21.15.130:0/2333249076 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f89bc016070 con 0x7f89c410d270 2023-09-12T15:56:57.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.357+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f89c4199c10 con 0x7f89c410d270 2023-09-12T15:56:57.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.361+0000 7f89b37fe700 1 -- 172.21.15.130:0/2333249076 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v21)=0 v21) v1 ==== 116+0+0 (secure 0 0 0) 0x7f89bc010b40 con 0x7f89c410d270 2023-09-12T15:56:57.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89ac038630 msgr2=0x7f89ac03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89ac038630 0x7f89ac03aaf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f89b800b5c0 tx=0x7f89b8005fd0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c410d270 msgr2=0x7f89c41986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c410d270 0x7f89c41986e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f89bc007d40 tx=0x7f89bc00a340 comp rx=0 tx=0).stop 2023-09-12T15:56:57.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 shutdown_connections 2023-09-12T15:56:57.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89ac038630 0x7f89ac03aaf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89c410d270 0x7f89c41986e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c4101500 0x7f89c419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 --2- 172.21.15.130:0/2333249076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c4101970 0x7f89c419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 >> 172.21.15.130:0/2333249076 conn(0x7f89c4075a70 msgr2=0x7f89c40fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:57.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.363+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 shutdown_connections 2023-09-12T15:56:57.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.364+0000 7f89c8ad5700 1 -- 172.21.15.130:0/2333249076 wait complete. 2023-09-12T15:56:57.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.704+0000 7f8b61946700 1 -- 172.21.15.130:0/4212404700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c100fe0 msgr2=0x7f8b5c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.704+0000 7f8b61946700 1 --2- 172.21.15.130:0/4212404700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c100fe0 0x7f8b5c10d4f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400b0a0 tx=0x7f8b5400b3b0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.704+0000 7f8b61946700 1 -- 172.21.15.130:0/4212404700 shutdown_connections 2023-09-12T15:56:57.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.704+0000 7f8b61946700 1 --2- 172.21.15.130:0/4212404700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c100fe0 0x7f8b5c10d4f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.704+0000 7f8b61946700 1 --2- 172.21.15.130:0/4212404700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b5c106620 0x7f8b5c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.704+0000 7f8b61946700 1 --2- 172.21.15.130:0/4212404700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b5c100620 0x7f8b5c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.704+0000 7f8b61946700 1 -- 172.21.15.130:0/4212404700 >> 172.21.15.130:0/4212404700 conn(0x7f8b5c0fc0f0 msgr2=0x7f8b5c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:57.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.704+0000 7f8b61946700 1 -- 172.21.15.130:0/4212404700 shutdown_connections 2023-09-12T15:56:57.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.704+0000 7f8b61946700 1 -- 172.21.15.130:0/4212404700 wait complete. 2023-09-12T15:56:57.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b61946700 1 Processor -- start 2023-09-12T15:56:57.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b61946700 1 -- start start 2023-09-12T15:56:57.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b61946700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b5c100620 0x7f8b5c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:57.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b61946700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b5c100fe0 0x7f8b5c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:57.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b61946700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c106620 0x7f8b5c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:57.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b61946700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c196d40 con 0x7f8b5c106620 2023-09-12T15:56:57.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b61946700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c196eb0 con 0x7f8b5c100fe0 2023-09-12T15:56:57.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b61946700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c197070 con 0x7f8b5c100620 2023-09-12T15:56:57.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b5b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c106620 0x7f8b5c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:57.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b5b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c106620 0x7f8b5c1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52272/0 (socket says 172.21.15.130:52272) 2023-09-12T15:56:57.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.705+0000 7f8b5b7fe700 1 -- 172.21.15.130:0/2727594211 learned_addr learned my addr 172.21.15.130:0/2727594211 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:56:57.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b5b7fe700 1 -- 172.21.15.130:0/2727594211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b5c100620 msgr2=0x7f8b5c19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b5b7fe700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b5c100620 0x7f8b5c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b5b7fe700 1 -- 172.21.15.130:0/2727594211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b5c100fe0 msgr2=0x7f8b5c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b5a7fc700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b5c100fe0 0x7f8b5c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:57.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b5b7fe700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b5c100fe0 0x7f8b5c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b5b7fe700 1 -- 172.21.15.130:0/2727594211 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b54009d00 con 0x7f8b5c106620 2023-09-12T15:56:57.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b5b7fe700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c106620 0x7f8b5c1966d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009fd0 tx=0x7f8b5400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:57.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b60944700 1 -- 172.21.15.130:0/2727594211 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5400e040 con 0x7f8b5c106620 2023-09-12T15:56:57.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b60944700 1 -- 172.21.15.130:0/2727594211 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b54004020 con 0x7f8b5c106620 2023-09-12T15:56:57.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c1972f0 con 0x7f8b5c106620 2023-09-12T15:56:57.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b60944700 1 -- 172.21.15.130:0/2727594211 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5401b690 con 0x7f8b5c106620 2023-09-12T15:56:57.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.706+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c1a3640 con 0x7f8b5c106620 2023-09-12T15:56:57.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.707+0000 7f8b60944700 1 -- 172.21.15.130:0/2727594211 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f8b5401b7f0 con 0x7f8b5c106620 2023-09-12T15:56:57.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.707+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5c04e300 con 0x7f8b5c106620 2023-09-12T15:56:57.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.707+0000 7f8b60944700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8b44038630 0x7f8b4403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:57.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.707+0000 7f8b60944700 1 -- 172.21.15.130:0/2727594211 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8b5404c4a0 con 0x7f8b5c106620 2023-09-12T15:56:57.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.708+0000 7f8b5affd700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8b44038630 0x7f8b4403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:57.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.708+0000 7f8b5affd700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8b44038630 0x7f8b4403aaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c005fd0 tx=0x7f8b4c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:57.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.713+0000 7f8b60944700 1 -- 172.21.15.130:0/2727594211 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8b5401a3b0 con 0x7f8b5c106620 2023-09-12T15:56:57.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.925+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f8b5c1106f0 con 0x7f8b5c106620 2023-09-12T15:56:57.930 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.930+0000 7f8b60944700 1 -- 172.21.15.130:0/2727594211 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v22)=0 v22) v1 ==== 120+0+0 (secure 0 0 0) 0x7f8b54024d10 con 0x7f8b5c106620 2023-09-12T15:56:57.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8b44038630 msgr2=0x7f8b4403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8b44038630 0x7f8b4403aaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c005fd0 tx=0x7f8b4c005ee0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c106620 msgr2=0x7f8b5c1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c106620 0x7f8b5c1966d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009fd0 tx=0x7f8b5400cf40 comp rx=0 tx=0).stop 2023-09-12T15:56:57.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 shutdown_connections 2023-09-12T15:56:57.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8b44038630 0x7f8b4403aaf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b5c106620 0x7f8b5c1966d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b5c100620 0x7f8b5c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 --2- 172.21.15.130:0/2727594211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b5c100fe0 0x7f8b5c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 >> 172.21.15.130:0/2727594211 conn(0x7f8b5c0fc0f0 msgr2=0x7f8b5c0fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:57.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 shutdown_connections 2023-09-12T15:56:57.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.933+0000 7f8b61946700 1 -- 172.21.15.130:0/2727594211 wait complete. 2023-09-12T15:56:57.960 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:57 smithi130 ceph-mon[98670]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:57.961 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:57 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2333249076' entity='client.admin' 2023-09-12T15:56:57.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:57 smithi159 ceph-mon[102522]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:57.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:57 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2333249076' entity='client.admin' 2023-09-12T15:56:57.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 -- 172.21.15.130:0/2750752780 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 msgr2=0x7f4df0104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:57.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 --2- 172.21.15.130:0/2750752780 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 0x7f4df0104910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009990 tx=0x7f4de4009ca0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 -- 172.21.15.130:0/2750752780 shutdown_connections 2023-09-12T15:56:57.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 --2- 172.21.15.130:0/2750752780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df00ffee0 0x7f4df010c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 --2- 172.21.15.130:0/2750752780 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 0x7f4df0104910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 --2- 172.21.15.130:0/2750752780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df00ff520 0x7f4df00ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:57.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 -- 172.21.15.130:0/2750752780 >> 172.21.15.130:0/2750752780 conn(0x7f4df0076600 msgr2=0x7f4df0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:57.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 -- 172.21.15.130:0/2750752780 shutdown_connections 2023-09-12T15:56:57.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 -- 172.21.15.130:0/2750752780 wait complete. 2023-09-12T15:56:57.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:57 smithi203 ceph-mon[103431]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:56:57.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:57 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2333249076' entity='client.admin' 2023-09-12T15:56:57.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.994+0000 7f4df6df6700 1 Processor -- start 2023-09-12T15:56:57.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df6df6700 1 -- start start 2023-09-12T15:56:57.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df6df6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df00ff520 0x7f4df01974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:58.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df6df6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df00ffee0 0x7f4df0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:58.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df6df6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 0x7f4df01a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:58.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df6df6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0198350 con 0x7f4df0104530 2023-09-12T15:56:58.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df6df6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df01a2a90 con 0x7f4df00ffee0 2023-09-12T15:56:58.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df6df6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df01a2c50 con 0x7f4df00ff520 2023-09-12T15:56:58.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df5393700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 0x7f4df01a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:58.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df5393700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 0x7f4df01a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52286/0 (socket says 172.21.15.130:52286) 2023-09-12T15:56:58.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df5393700 1 -- 172.21.15.130:0/738664336 learned_addr learned my addr 172.21.15.130:0/738664336 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:56:58.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4deffff700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df00ffee0 0x7f4df0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:58.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df4b92700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df00ff520 0x7f4df01974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:58.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df5393700 1 -- 172.21.15.130:0/738664336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df00ff520 msgr2=0x7f4df01974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:58.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df5393700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df00ff520 0x7f4df01974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df5393700 1 -- 172.21.15.130:0/738664336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df00ffee0 msgr2=0x7f4df0197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:58.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df5393700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df00ffee0 0x7f4df0197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.995+0000 7f4df5393700 1 -- 172.21.15.130:0/738664336 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de40095f0 con 0x7f4df0104530 2023-09-12T15:56:58.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.996+0000 7f4df5393700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 0x7f4df01a2550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4de800efc0 tx=0x7f4de800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:58.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.996+0000 7f4dedffb700 1 -- 172.21.15.130:0/738664336 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8018070 con 0x7f4df0104530 2023-09-12T15:56:58.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.996+0000 7f4dedffb700 1 -- 172.21.15.130:0/738664336 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de800f040 con 0x7f4df0104530 2023-09-12T15:56:58.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.996+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df01952c0 con 0x7f4df0104530 2023-09-12T15:56:58.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.996+0000 7f4dedffb700 1 -- 172.21.15.130:0/738664336 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de800cc10 con 0x7f4df0104530 2023-09-12T15:56:58.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.996+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df0195770 con 0x7f4df0104530 2023-09-12T15:56:58.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.997+0000 7f4dedffb700 1 -- 172.21.15.130:0/738664336 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f4de8004000 con 0x7f4df0104530 2023-09-12T15:56:58.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.997+0000 7f4dedffb700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4de0038630 0x7f4de003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:58.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.997+0000 7f4dedffb700 1 -- 172.21.15.130:0/738664336 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4de804b800 con 0x7f4df0104530 2023-09-12T15:56:58.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.997+0000 7f4df4b92700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4de0038630 0x7f4de003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:58.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.997+0000 7f4df4b92700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4de0038630 0x7f4de003aaf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4de400b2e0 tx=0x7f4de4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:58.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:57.997+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df004e300 con 0x7f4df0104530 2023-09-12T15:56:58.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.002+0000 7f4dedffb700 1 -- 172.21.15.130:0/738664336 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4de8007500 con 0x7f4df0104530 2023-09-12T15:56:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.212+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f4df010f450 con 0x7f4df0104530 2023-09-12T15:56:58.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.217+0000 7f4dedffb700 1 -- 172.21.15.130:0/738664336 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v23)=0 v23) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4de803e030 con 0x7f4df0104530 2023-09-12T15:56:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4de0038630 msgr2=0x7f4de003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4de0038630 0x7f4de003aaf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4de400b2e0 tx=0x7f4de4005fd0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 msgr2=0x7f4df01a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 0x7f4df01a2550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4de800efc0 tx=0x7f4de800c3a0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 shutdown_connections 2023-09-12T15:56:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4de0038630 0x7f4de003aaf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df0104530 0x7f4df01a2550 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df00ff520 0x7f4df01974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 --2- 172.21.15.130:0/738664336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df00ffee0 0x7f4df0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 >> 172.21.15.130:0/738664336 conn(0x7f4df0076600 msgr2=0x7f4df00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:58.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 shutdown_connections 2023-09-12T15:56:58.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.220+0000 7f4df6df6700 1 -- 172.21.15.130:0/738664336 wait complete. 2023-09-12T15:56:58.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.057+0000 7ff641c63700 1 -- 172.21.15.130:0/2968856827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c0ff880 msgr2=0x7ff63c0ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:58.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.057+0000 7ff641c63700 1 --2- 172.21.15.130:0/2968856827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c0ff880 0x7ff63c0ffc60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009990 tx=0x7ff62c009ca0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.058+0000 7ff641c63700 1 -- 172.21.15.130:0/2968856827 shutdown_connections 2023-09-12T15:56:58.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.058+0000 7ff641c63700 1 --2- 172.21.15.130:0/2968856827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c101840 0x7ff63c10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.058+0000 7ff641c63700 1 --2- 172.21.15.130:0/2968856827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c0ff880 0x7ff63c0ffc60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.058+0000 7ff641c63700 1 --2- 172.21.15.130:0/2968856827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff63c1001a0 0x7ff63c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.058+0000 7ff641c63700 1 -- 172.21.15.130:0/2968856827 >> 172.21.15.130:0/2968856827 conn(0x7ff63c0765f0 msgr2=0x7ff63c076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:58.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.058+0000 7ff641c63700 1 -- 172.21.15.130:0/2968856827 shutdown_connections 2023-09-12T15:56:58.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.058+0000 7ff641c63700 1 -- 172.21.15.130:0/2968856827 wait complete. 2023-09-12T15:56:58.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.058+0000 7ff641c63700 1 Processor -- start 2023-09-12T15:56:58.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff641c63700 1 -- start start 2023-09-12T15:56:58.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff641c63700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c0ff880 0x7ff63c19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:58.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff641c63700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c1001a0 0x7ff63c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:58.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff641c63700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff63c101840 0x7ff63c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:58.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff641c63700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c196ce0 con 0x7ff63c1001a0 2023-09-12T15:56:58.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff641c63700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c196e50 con 0x7ff63c101840 2023-09-12T15:56:58.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff641c63700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c197010 con 0x7ff63c0ff880 2023-09-12T15:56:58.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c1001a0 0x7ff63c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:58.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c1001a0 0x7ff63c19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52298/0 (socket says 172.21.15.130:52298) 2023-09-12T15:56:58.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63affd700 1 -- 172.21.15.130:0/2157363112 learned_addr learned my addr 172.21.15.130:0/2157363112 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:56:58.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63b7fe700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c0ff880 0x7ff63c19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:58.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63bfff700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff63c101840 0x7ff63c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:58.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63affd700 1 -- 172.21.15.130:0/2157363112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c0ff880 msgr2=0x7ff63c19c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:58.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63affd700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c0ff880 0x7ff63c19c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63affd700 1 -- 172.21.15.130:0/2157363112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff63c101840 msgr2=0x7ff63c196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:58.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63affd700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff63c101840 0x7ff63c196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63affd700 1 -- 172.21.15.130:0/2157363112 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c0095f0 con 0x7ff63c1001a0 2023-09-12T15:56:58.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff63affd700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c1001a0 0x7ff63c19cae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff63000d980 tx=0x7ff63000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:58.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff638ff9700 1 -- 172.21.15.130:0/2157363112 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630009b40 con 0x7ff63c1001a0 2023-09-12T15:56:58.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff638ff9700 1 -- 172.21.15.130:0/2157363112 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff63000de60 con 0x7ff63c1001a0 2023-09-12T15:56:58.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff638ff9700 1 -- 172.21.15.130:0/2157363112 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63000f620 con 0x7ff63c1001a0 2023-09-12T15:56:58.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.059+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c1972f0 con 0x7ff63c1001a0 2023-09-12T15:56:58.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.060+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c1a3590 con 0x7ff63c1001a0 2023-09-12T15:56:58.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.060+0000 7ff638ff9700 1 -- 172.21.15.130:0/2157363112 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7ff63000f780 con 0x7ff63c1001a0 2023-09-12T15:56:58.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.061+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c04e300 con 0x7ff63c1001a0 2023-09-12T15:56:58.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.061+0000 7ff638ff9700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff624038630 0x7ff62403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:58.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.061+0000 7ff638ff9700 1 -- 172.21.15.130:0/2157363112 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff63004ac20 con 0x7ff63c1001a0 2023-09-12T15:56:58.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.061+0000 7ff63b7fe700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff624038630 0x7ff62403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:58.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.061+0000 7ff63b7fe700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff624038630 0x7ff62403aaf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff62c006010 tx=0x7ff62c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:58.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.065+0000 7ff638ff9700 1 -- 172.21.15.130:0/2157363112 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff63001a720 con 0x7ff63c1001a0 2023-09-12T15:56:58.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.275+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7ff63c197b30 con 0x7ff63c1001a0 2023-09-12T15:56:58.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.278+0000 7ff638ff9700 1 -- 172.21.15.130:0/2157363112 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v24)=0 v24) v1 ==== 119+0+0 (secure 0 0 0) 0x7ff630020020 con 0x7ff63c1001a0 2023-09-12T15:56:58.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.280+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff624038630 msgr2=0x7ff62403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:58.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.280+0000 7ff641c63700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff624038630 0x7ff62403aaf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff62c006010 tx=0x7ff62c00b540 comp rx=0 tx=0).stop 2023-09-12T15:56:58.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.280+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c1001a0 msgr2=0x7ff63c19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:58.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.280+0000 7ff641c63700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c1001a0 0x7ff63c19cae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff63000d980 tx=0x7ff63000dd40 comp rx=0 tx=0).stop 2023-09-12T15:56:58.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.281+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 shutdown_connections 2023-09-12T15:56:58.376 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.281+0000 7ff641c63700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff624038630 0x7ff62403aaf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.376 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.281+0000 7ff641c63700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff63c101840 0x7ff63c196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.376 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.281+0000 7ff641c63700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c0ff880 0x7ff63c19c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.376 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.281+0000 7ff641c63700 1 --2- 172.21.15.130:0/2157363112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff63c1001a0 0x7ff63c19cae0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:58.376 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.281+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 >> 172.21.15.130:0/2157363112 conn(0x7ff63c0765f0 msgr2=0x7ff63c0689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:58.377 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.281+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 shutdown_connections 2023-09-12T15:56:58.377 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:58.281+0000 7ff641c63700 1 -- 172.21.15.130:0/2157363112 wait complete. 2023-09-12T15:56:59.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:58 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2727594211' entity='client.admin' 2023-09-12T15:56:59.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:58 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/738664336' entity='client.admin' 2023-09-12T15:56:59.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:58 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2157363112' entity='client.admin' 2023-09-12T15:56:59.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:58 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2727594211' entity='client.admin' 2023-09-12T15:56:59.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:58 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/738664336' entity='client.admin' 2023-09-12T15:56:59.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:58 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2157363112' entity='client.admin' 2023-09-12T15:56:59.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:58 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2727594211' entity='client.admin' 2023-09-12T15:56:59.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:58 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/738664336' entity='client.admin' 2023-09-12T15:56:59.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:58 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2157363112' entity='client.admin' 2023-09-12T15:56:59.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.752+0000 7f407ae08700 1 -- 172.21.15.130:0/1898112285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 msgr2=0x7f407410f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:59.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.752+0000 7f407ae08700 1 --2- 172.21.15.130:0/1898112285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 0x7f407410f2d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f406c00b0a0 tx=0x7f406c00b3b0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.752+0000 7f407ae08700 1 -- 172.21.15.130:0/1898112285 shutdown_connections 2023-09-12T15:56:59.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.752+0000 7f407ae08700 1 --2- 172.21.15.130:0/1898112285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 0x7f407410f2d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.752+0000 7f407ae08700 1 --2- 172.21.15.130:0/1898112285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40740729d0 0x7f4074072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.752+0000 7f407ae08700 1 --2- 172.21.15.130:0/1898112285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40740732f0 0x7f407410c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.752+0000 7f407ae08700 1 -- 172.21.15.130:0/1898112285 >> 172.21.15.130:0/1898112285 conn(0x7f40740fc0f0 msgr2=0x7f40740fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:59.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.752+0000 7f407ae08700 1 -- 172.21.15.130:0/1898112285 shutdown_connections 2023-09-12T15:56:59.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.752+0000 7f407ae08700 1 -- 172.21.15.130:0/1898112285 wait complete. 2023-09-12T15:56:59.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f407ae08700 1 Processor -- start 2023-09-12T15:56:59.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f407ae08700 1 -- start start 2023-09-12T15:56:59.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f407ae08700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40740729d0 0x7f407419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:59.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f407ae08700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40740732f0 0x7f407419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:59.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f407ae08700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 0x7f40741988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:59.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f407ae08700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4074198f60 con 0x7f407410cec0 2023-09-12T15:56:59.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f407ae08700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40741990d0 con 0x7f40740729d0 2023-09-12T15:56:59.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f407ae08700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4074199290 con 0x7f40740732f0 2023-09-12T15:56:59.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f40793a5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 0x7f40741988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:59.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f40793a5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 0x7f40741988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52314/0 (socket says 172.21.15.130:52314) 2023-09-12T15:56:59.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.753+0000 7f40793a5700 1 -- 172.21.15.130:0/1963166014 learned_addr learned my addr 172.21.15.130:0/1963166014 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:56:59.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f4073fff700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40740732f0 0x7f407419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:59.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f4078ba4700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40740729d0 0x7f407419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:59.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f40793a5700 1 -- 172.21.15.130:0/1963166014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40740732f0 msgr2=0x7f407419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:59.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f40793a5700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40740732f0 0x7f407419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f40793a5700 1 -- 172.21.15.130:0/1963166014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40740729d0 msgr2=0x7f407419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:59.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f40793a5700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40740729d0 0x7f407419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f40793a5700 1 -- 172.21.15.130:0/1963166014 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c009d00 con 0x7f407410cec0 2023-09-12T15:56:59.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f4073fff700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40740732f0 0x7f407419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T15:56:59.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f40793a5700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 0x7f40741988f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f406c00b070 tx=0x7f406c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:59.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f4071ffb700 1 -- 172.21.15.130:0/1963166014 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c00e040 con 0x7f407410cec0 2023-09-12T15:56:59.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f4071ffb700 1 -- 172.21.15.130:0/1963166014 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f406c004e50 con 0x7f407410cec0 2023-09-12T15:56:59.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f4071ffb700 1 -- 172.21.15.130:0/1963166014 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c0048f0 con 0x7f407410cec0 2023-09-12T15:56:59.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4074199510 con 0x7f407410cec0 2023-09-12T15:56:59.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.754+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40741a5860 con 0x7f407410cec0 2023-09-12T15:56:59.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.755+0000 7f4071ffb700 1 -- 172.21.15.130:0/1963166014 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f406c003a00 con 0x7f407410cec0 2023-09-12T15:56:59.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.755+0000 7f4071ffb700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4064038630 0x7f406403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:59.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.755+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407404e300 con 0x7f407410cec0 2023-09-12T15:56:59.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.756+0000 7f4071ffb700 1 -- 172.21.15.130:0/1963166014 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f406c04c3d0 con 0x7f407410cec0 2023-09-12T15:56:59.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.756+0000 7f4078ba4700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4064038630 0x7f406403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:59.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.756+0000 7f4078ba4700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4064038630 0x7f406403aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4068005950 tx=0x7f40680058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:59.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.760+0000 7f4071ffb700 1 -- 172.21.15.130:0/1963166014 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f406c024a10 con 0x7f407410cec0 2023-09-12T15:56:59.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 -- 172.21.15.130:0/3469308771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c068ae0 msgr2=0x7f7c8c10d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:59.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 --2- 172.21.15.130:0/3469308771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c068ae0 0x7f7c8c10d440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400b0a0 tx=0x7f7c8400b3b0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 -- 172.21.15.130:0/3469308771 shutdown_connections 2023-09-12T15:56:59.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 --2- 172.21.15.130:0/3469308771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c068ae0 0x7f7c8c10d440 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 --2- 172.21.15.130:0/3469308771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c106770 0x7f7c8c106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 --2- 172.21.15.130:0/3469308771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c8c068120 0x7f7c8c0685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 -- 172.21.15.130:0/3469308771 >> 172.21.15.130:0/3469308771 conn(0x7f7c8c076600 msgr2=0x7f7c8c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:56:59.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 -- 172.21.15.130:0/3469308771 shutdown_connections 2023-09-12T15:56:59.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 -- 172.21.15.130:0/3469308771 wait complete. 2023-09-12T15:56:59.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 Processor -- start 2023-09-12T15:56:59.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.831+0000 7f7c93f13700 1 -- start start 2023-09-12T15:56:59.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c93f13700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c068120 0x7f7c8c0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:59.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c93f13700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c8c068ae0 0x7f7c8c0ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:59.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c93f13700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c106770 0x7f7c8c100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:59.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c93f13700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1a4cb0 con 0x7f7c8c106770 2023-09-12T15:56:59.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c93f13700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1a4e20 con 0x7f7c8c068120 2023-09-12T15:56:59.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c93f13700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1a4fe0 con 0x7f7c8c068ae0 2023-09-12T15:56:59.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c924b0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c106770 0x7f7c8c100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:59.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c924b0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c106770 0x7f7c8c100070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52326/0 (socket says 172.21.15.130:52326) 2023-09-12T15:56:59.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c924b0700 1 -- 172.21.15.130:0/942087734 learned_addr learned my addr 172.21.15.130:0/942087734 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:56:59.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c91caf700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c068120 0x7f7c8c0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:59.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c914ae700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c8c068ae0 0x7f7c8c0ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:59.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c924b0700 1 -- 172.21.15.130:0/942087734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c8c068ae0 msgr2=0x7f7c8c0ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:59.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c924b0700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c8c068ae0 0x7f7c8c0ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c924b0700 1 -- 172.21.15.130:0/942087734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c068120 msgr2=0x7f7c8c0ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:56:59.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c924b0700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c068120 0x7f7c8c0ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:56:59.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c924b0700 1 -- 172.21.15.130:0/942087734 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84009d00 con 0x7f7c8c106770 2023-09-12T15:56:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c91caf700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c068120 0x7f7c8c0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T15:56:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.832+0000 7f7c924b0700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c106770 0x7f7c8c100070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400b780 tx=0x7f7c84007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.833+0000 7f7c82ffd700 1 -- 172.21.15.130:0/942087734 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400e040 con 0x7f7c8c106770 2023-09-12T15:56:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.833+0000 7f7c82ffd700 1 -- 172.21.15.130:0/942087734 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c8400be40 con 0x7f7c8c106770 2023-09-12T15:56:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.833+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c1a5260 con 0x7f7c8c106770 2023-09-12T15:56:59.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.833+0000 7f7c82ffd700 1 -- 172.21.15.130:0/942087734 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c840052a0 con 0x7f7c8c106770 2023-09-12T15:56:59.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.833+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c1a5720 con 0x7f7c8c106770 2023-09-12T15:56:59.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.834+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c04e300 con 0x7f7c8c106770 2023-09-12T15:56:59.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.834+0000 7f7c82ffd700 1 -- 172.21.15.130:0/942087734 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f7c8401d470 con 0x7f7c8c106770 2023-09-12T15:56:59.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.834+0000 7f7c82ffd700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c78038630 0x7f7c7803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:56:59.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.834+0000 7f7c82ffd700 1 -- 172.21.15.130:0/942087734 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7c8404c330 con 0x7f7c8c106770 2023-09-12T15:56:59.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.834+0000 7f7c91caf700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c78038630 0x7f7c7803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:56:59.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.835+0000 7f7c91caf700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c78038630 0x7f7c7803aaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009c90 tx=0x7f7c88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:56:59.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.839+0000 7f7c82ffd700 1 -- 172.21.15.130:0/942087734 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7c84025ba0 con 0x7f7c8c106770 2023-09-12T15:56:59.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.996+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7f4074112a90 con 0x7f407410cec0 2023-09-12T15:57:00.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:56:59.999+0000 7f4071ffb700 1 -- 172.21.15.130:0/1963166014 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v25)=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f406c0243c0 con 0x7f407410cec0 2023-09-12T15:57:00.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4064038630 msgr2=0x7f406403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:00.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4064038630 0x7f406403aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4068005950 tx=0x7f40680058e0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 msgr2=0x7f40741988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:00.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 0x7f40741988f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f406c00b070 tx=0x7f406c0095c0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 shutdown_connections 2023-09-12T15:57:00.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4064038630 0x7f406403aaf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f407410cec0 0x7f40741988f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40740729d0 0x7f407419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 --2- 172.21.15.130:0/1963166014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40740732f0 0x7f407419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 >> 172.21.15.130:0/1963166014 conn(0x7f40740fc0f0 msgr2=0x7f407410efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:00.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 shutdown_connections 2023-09-12T15:57:00.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.002+0000 7f407ae08700 1 -- 172.21.15.130:0/1963166014 wait complete. 2023-09-12T15:57:00.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.044+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f7c8c110720 con 0x7f7c8c106770 2023-09-12T15:57:00.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.047+0000 7f7c82ffd700 1 -- 172.21.15.130:0/942087734 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v26)=0 v26) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7c84021020 con 0x7f7c8c106770 2023-09-12T15:57:00.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c78038630 msgr2=0x7f7c7803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:00.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c78038630 0x7f7c7803aaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009c90 tx=0x7f7c88009260 comp rx=0 tx=0).stop 2023-09-12T15:57:00.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c106770 msgr2=0x7f7c8c100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:00.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c106770 0x7f7c8c100070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400b780 tx=0x7f7c84007990 comp rx=0 tx=0).stop 2023-09-12T15:57:00.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 shutdown_connections 2023-09-12T15:57:00.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c78038630 0x7f7c7803aaf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c8c106770 0x7f7c8c100070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c068120 0x7f7c8c0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 --2- 172.21.15.130:0/942087734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c8c068ae0 0x7f7c8c0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 >> 172.21.15.130:0/942087734 conn(0x7f7c8c076600 msgr2=0x7f7c8c1042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:00.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 shutdown_connections 2023-09-12T15:57:00.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.049+0000 7f7c93f13700 1 -- 172.21.15.130:0/942087734 wait complete. 2023-09-12T15:57:00.191 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:56:59 smithi130 ceph-mon[98670]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:00.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:56:59 smithi159 ceph-mon[102522]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:00.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:56:59 smithi203 ceph-mon[103431]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:00.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.468+0000 7f0e02d6d700 1 -- 172.21.15.130:0/588648688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068120 msgr2=0x7f0dfc0685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:00.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.468+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/588648688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068120 0x7f0dfc0685a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009990 tx=0x7f0de4009ca0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.468+0000 7f0e02d6d700 1 -- 172.21.15.130:0/588648688 shutdown_connections 2023-09-12T15:57:00.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.468+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/588648688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dfc068ae0 0x7f0dfc10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.468+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/588648688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc106770 0x7f0dfc106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.468+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/588648688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068120 0x7f0dfc0685a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.468+0000 7f0e02d6d700 1 -- 172.21.15.130:0/588648688 >> 172.21.15.130:0/588648688 conn(0x7f0dfc076600 msgr2=0x7f0dfc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:00.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.468+0000 7f0e02d6d700 1 -- 172.21.15.130:0/588648688 shutdown_connections 2023-09-12T15:57:00.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.468+0000 7f0e02d6d700 1 -- 172.21.15.130:0/588648688 wait complete. 2023-09-12T15:57:00.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e02d6d700 1 Processor -- start 2023-09-12T15:57:00.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e02d6d700 1 -- start start 2023-09-12T15:57:00.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e02d6d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dfc068120 0x7f0dfc196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:00.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e02d6d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068ae0 0x7f0dfc196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:00.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e02d6d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc106770 0x7f0dfc199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:00.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e02d6d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc19a590 con 0x7f0dfc068ae0 2023-09-12T15:57:00.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e02d6d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc19a700 con 0x7f0dfc106770 2023-09-12T15:57:00.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e02d6d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc19a8c0 con 0x7f0dfc068120 2023-09-12T15:57:00.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0dfbfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068ae0 0x7f0dfc196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:00.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0dfbfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068ae0 0x7f0dfc196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52340/0 (socket says 172.21.15.130:52340) 2023-09-12T15:57:00.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0dfbfff700 1 -- 172.21.15.130:0/2243503980 learned_addr learned my addr 172.21.15.130:0/2243503980 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:57:00.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e00b09700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dfc068120 0x7f0dfc196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:00.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0e0130a700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc106770 0x7f0dfc199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:00.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.469+0000 7f0dfbfff700 1 -- 172.21.15.130:0/2243503980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dfc068120 msgr2=0x7f0dfc196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:00.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0dfbfff700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dfc068120 0x7f0dfc196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0dfbfff700 1 -- 172.21.15.130:0/2243503980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc106770 msgr2=0x7f0dfc199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:00.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0dfbfff700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc106770 0x7f0dfc199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0dfbfff700 1 -- 172.21.15.130:0/2243503980 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de40095f0 con 0x7f0dfc068ae0 2023-09-12T15:57:00.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0e00b09700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dfc068120 0x7f0dfc196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T15:57:00.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0e0130a700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc106770 0x7f0dfc199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T15:57:00.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0dfbfff700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068ae0 0x7f0dfc196bd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0de40095c0 tx=0x7f0de400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:00.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0df9ffb700 1 -- 172.21.15.130:0/2243503980 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de401d070 con 0x7f0dfc068ae0 2023-09-12T15:57:00.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0df9ffb700 1 -- 172.21.15.130:0/2243503980 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0de40054f0 con 0x7f0dfc068ae0 2023-09-12T15:57:00.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0df9ffb700 1 -- 172.21.15.130:0/2243503980 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4004bd0 con 0x7f0dfc068ae0 2023-09-12T15:57:00.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc19ab40 con 0x7f0dfc068ae0 2023-09-12T15:57:00.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.470+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc19ae10 con 0x7f0dfc068ae0 2023-09-12T15:57:00.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.471+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dfc04e300 con 0x7f0dfc068ae0 2023-09-12T15:57:00.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.471+0000 7f0df9ffb700 1 -- 172.21.15.130:0/2243503980 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f0de400fc30 con 0x7f0dfc068ae0 2023-09-12T15:57:00.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.472+0000 7f0df9ffb700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0de8038630 0x7f0de803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:00.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.472+0000 7f0df9ffb700 1 -- 172.21.15.130:0/2243503980 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0de404c2b0 con 0x7f0dfc068ae0 2023-09-12T15:57:00.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.472+0000 7f0e00b09700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0de8038630 0x7f0de803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:00.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.472+0000 7f0e00b09700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0de8038630 0x7f0de803aaf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0df400d340 tx=0x7f0df4009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:00.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.476+0000 7f0df9ffb700 1 -- 172.21.15.130:0/2243503980 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0de4005660 con 0x7f0dfc068ae0 2023-09-12T15:57:00.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.674+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f0dfc1979f0 con 0x7f0dfc068ae0 2023-09-12T15:57:00.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.677+0000 7f0df9ffb700 1 -- 172.21.15.130:0/2243503980 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v27)=0 v27) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0de4026030 con 0x7f0dfc068ae0 2023-09-12T15:57:00.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0de8038630 msgr2=0x7f0de803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:00.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0de8038630 0x7f0de803aaf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0df400d340 tx=0x7f0df4009040 comp rx=0 tx=0).stop 2023-09-12T15:57:00.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068ae0 msgr2=0x7f0dfc196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:00.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068ae0 0x7f0dfc196bd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0de40095c0 tx=0x7f0de400f740 comp rx=0 tx=0).stop 2023-09-12T15:57:00.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 shutdown_connections 2023-09-12T15:57:00.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0de8038630 0x7f0de803aaf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc106770 0x7f0dfc199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dfc068120 0x7f0dfc196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 --2- 172.21.15.130:0/2243503980 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dfc068ae0 0x7f0dfc196bd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:00.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 >> 172.21.15.130:0/2243503980 conn(0x7f0dfc076600 msgr2=0x7f0dfc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:00.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 shutdown_connections 2023-09-12T15:57:00.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:00.680+0000 7f0e02d6d700 1 -- 172.21.15.130:0/2243503980 wait complete. 2023-09-12T15:57:01.084 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1963166014' entity='client.admin' 2023-09-12T15:57:01.084 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/942087734' entity='client.admin' 2023-09-12T15:57:01.085 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2243503980' entity='client.admin' 2023-09-12T15:57:01.085 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:01 smithi130 ceph-mon[98670]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:01.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.248+0000 7f0d39c84700 1 -- 172.21.15.130:0/1954968076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34106740 msgr2=0x7f0d34106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.248+0000 7f0d39c84700 1 --2- 172.21.15.130:0/1954968076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34106740 0x7f0d34106b20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009990 tx=0x7f0d1c009ca0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.248+0000 7f0d39c84700 1 -- 172.21.15.130:0/1954968076 shutdown_connections 2023-09-12T15:57:01.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.248+0000 7f0d39c84700 1 --2- 172.21.15.130:0/1954968076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d34101250 0x7f0d3410d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.248+0000 7f0d39c84700 1 --2- 172.21.15.130:0/1954968076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34106740 0x7f0d34106b20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.248+0000 7f0d39c84700 1 --2- 172.21.15.130:0/1954968076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d34100890 0x7f0d34100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.248+0000 7f0d39c84700 1 -- 172.21.15.130:0/1954968076 >> 172.21.15.130:0/1954968076 conn(0x7f0d340785f0 msgr2=0x7f0d34078a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:01.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.248+0000 7f0d39c84700 1 -- 172.21.15.130:0/1954968076 shutdown_connections 2023-09-12T15:57:01.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.248+0000 7f0d39c84700 1 -- 172.21.15.130:0/1954968076 wait complete. 2023-09-12T15:57:01.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d39c84700 1 Processor -- start 2023-09-12T15:57:01.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d39c84700 1 -- start start 2023-09-12T15:57:01.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d39c84700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d34100890 0x7f0d341966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d39c84700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34101250 0x7f0d34196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d39c84700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d34106740 0x7f0d34199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d39c84700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3419a5a0 con 0x7f0d34101250 2023-09-12T15:57:01.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d39c84700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3419a710 con 0x7f0d34106740 2023-09-12T15:57:01.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d39c84700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3419a8d0 con 0x7f0d34100890 2023-09-12T15:57:01.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d32ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34101250 0x7f0d34196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d32ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34101250 0x7f0d34196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52354/0 (socket says 172.21.15.130:52354) 2023-09-12T15:57:01.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d32ffd700 1 -- 172.21.15.130:0/3035150903 learned_addr learned my addr 172.21.15.130:0/3035150903 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:57:01.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.249+0000 7f0d337fe700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d34100890 0x7f0d341966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d32ffd700 1 -- 172.21.15.130:0/3035150903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d34100890 msgr2=0x7f0d341966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d32ffd700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d34100890 0x7f0d341966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d33fff700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d34106740 0x7f0d34199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d32ffd700 1 -- 172.21.15.130:0/3035150903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d34106740 msgr2=0x7f0d34199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d32ffd700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d34106740 0x7f0d34199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d32ffd700 1 -- 172.21.15.130:0/3035150903 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0095f0 con 0x7f0d34101250 2023-09-12T15:57:01.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d337fe700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d34100890 0x7f0d341966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T15:57:01.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d32ffd700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34101250 0x7f0d34196be0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400d980 tx=0x7f0d2400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:01.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d30ff9700 1 -- 172.21.15.130:0/3035150903 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2400bb60 con 0x7f0d34101250 2023-09-12T15:57:01.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d30ff9700 1 -- 172.21.15.130:0/3035150903 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d24007640 con 0x7f0d34101250 2023-09-12T15:57:01.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3419abb0 con 0x7f0d34101250 2023-09-12T15:57:01.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d30ff9700 1 -- 172.21.15.130:0/3035150903 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24004e90 con 0x7f0d34101250 2023-09-12T15:57:01.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.250+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d341a3500 con 0x7f0d34101250 2023-09-12T15:57:01.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.251+0000 7f0d30ff9700 1 -- 172.21.15.130:0/3035150903 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f0d24005020 con 0x7f0d34101250 2023-09-12T15:57:01.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.251+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3404e300 con 0x7f0d34101250 2023-09-12T15:57:01.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.251+0000 7f0d30ff9700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d20038630 0x7f0d2003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.251+0000 7f0d30ff9700 1 -- 172.21.15.130:0/3035150903 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0d2404acd0 con 0x7f0d34101250 2023-09-12T15:57:01.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.251+0000 7f0d337fe700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d20038630 0x7f0d2003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.252+0000 7f0d337fe700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d20038630 0x7f0d2003aaf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00b5c0 tx=0x7f0d1c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:01.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.256+0000 7f0d30ff9700 1 -- 172.21.15.130:0/3035150903 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0d24022760 con 0x7f0d34101250 2023-09-12T15:57:01.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.454+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f0d34197a00 con 0x7f0d34101250 2023-09-12T15:57:01.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.457+0000 7f0d30ff9700 1 -- 172.21.15.130:0/3035150903 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v28)=0 v28) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0d2400de60 con 0x7f0d34101250 2023-09-12T15:57:01.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d20038630 msgr2=0x7f0d2003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d20038630 0x7f0d2003aaf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00b5c0 tx=0x7f0d1c005fb0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34101250 msgr2=0x7f0d34196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34101250 0x7f0d34196be0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400d980 tx=0x7f0d2400dd40 comp rx=0 tx=0).stop 2023-09-12T15:57:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 shutdown_connections 2023-09-12T15:57:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d20038630 0x7f0d2003aaf0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00b5c0 tx=0x7f0d1c005fb0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d34106740 0x7f0d34199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d34100890 0x7f0d341966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 --2- 172.21.15.130:0/3035150903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d34101250 0x7f0d34196be0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 >> 172.21.15.130:0/3035150903 conn(0x7f0d340785f0 msgr2=0x7f0d340ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:01.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 shutdown_connections 2023-09-12T15:57:01.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.460+0000 7f0d39c84700 1 -- 172.21.15.130:0/3035150903 wait complete. 2023-09-12T15:57:01.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1963166014' entity='client.admin' 2023-09-12T15:57:01.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/942087734' entity='client.admin' 2023-09-12T15:57:01.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2243503980' entity='client.admin' 2023-09-12T15:57:01.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:01 smithi159 ceph-mon[102522]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:01.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1963166014' entity='client.admin' 2023-09-12T15:57:01.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/942087734' entity='client.admin' 2023-09-12T15:57:01.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2243503980' entity='client.admin' 2023-09-12T15:57:01.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:01 smithi203 ceph-mon[103431]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:01.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.722+0000 7fe5abde9700 1 -- 172.21.15.130:0/1107341507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a41066a0 msgr2=0x7fe5a4106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.722+0000 7fe5abde9700 1 --2- 172.21.15.130:0/1107341507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a41066a0 0x7fe5a4106a80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009990 tx=0x7fe5a0009ca0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 -- 172.21.15.130:0/1107341507 shutdown_connections 2023-09-12T15:57:01.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 --2- 172.21.15.130:0/1107341507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5a4100fd0 0x7fe5a410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 --2- 172.21.15.130:0/1107341507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a41066a0 0x7fe5a4106a80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 --2- 172.21.15.130:0/1107341507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a4100610 0x7fe5a4100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 -- 172.21.15.130:0/1107341507 >> 172.21.15.130:0/1107341507 conn(0x7fe5a40fc100 msgr2=0x7fe5a40fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:01.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 -- 172.21.15.130:0/1107341507 shutdown_connections 2023-09-12T15:57:01.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 -- 172.21.15.130:0/1107341507 wait complete. 2023-09-12T15:57:01.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 Processor -- start 2023-09-12T15:57:01.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 -- start start 2023-09-12T15:57:01.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a4100610 0x7fe5a4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a4100fd0 0x7fe5a4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5a41066a0 0x7fe5a40733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a41a4bc0 con 0x7fe5a4100610 2023-09-12T15:57:01.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a41a4d30 con 0x7fe5a4100fd0 2023-09-12T15:57:01.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.723+0000 7fe5abde9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a41a4ef0 con 0x7fe5a41066a0 2023-09-12T15:57:01.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9b85700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a4100610 0x7fe5a4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9b85700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a4100610 0x7fe5a4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52366/0 (socket says 172.21.15.130:52366) 2023-09-12T15:57:01.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9b85700 1 -- 172.21.15.130:0/3003560901 learned_addr learned my addr 172.21.15.130:0/3003560901 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:57:01.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5aa386700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5a41066a0 0x7fe5a40733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.730 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9384700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a4100fd0 0x7fe5a4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.730 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9b85700 1 -- 172.21.15.130:0/3003560901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5a41066a0 msgr2=0x7fe5a40733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.730 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9b85700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5a41066a0 0x7fe5a40733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.730 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9b85700 1 -- 172.21.15.130:0/3003560901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a4100fd0 msgr2=0x7fe5a4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.730 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9b85700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a4100fd0 0x7fe5a4072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.731 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9b85700 1 -- 172.21.15.130:0/3003560901 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00095f0 con 0x7fe5a4100610 2023-09-12T15:57:01.731 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5aa386700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5a41066a0 0x7fe5a40733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T15:57:01.731 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5a9b85700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a4100610 0x7fe5a4072900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0003730 tx=0x7fe5a000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:01.731 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe59affd700 1 -- 172.21.15.130:0/3003560901 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a001d070 con 0x7fe5a4100610 2023-09-12T15:57:01.731 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe59affd700 1 -- 172.21.15.130:0/3003560901 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5a0004c90 con 0x7fe5a4100610 2023-09-12T15:57:01.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe59affd700 1 -- 172.21.15.130:0/3003560901 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00053f0 con 0x7fe5a4100610 2023-09-12T15:57:01.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a41a5170 con 0x7fe5a4100610 2023-09-12T15:57:01.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.724+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a41a5530 con 0x7fe5a4100610 2023-09-12T15:57:01.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.725+0000 7fe59affd700 1 -- 172.21.15.130:0/3003560901 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7fe5a00055d0 con 0x7fe5a4100610 2023-09-12T15:57:01.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.725+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a404e300 con 0x7fe5a4100610 2023-09-12T15:57:01.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.726+0000 7fe59affd700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe590038630 0x7fe59003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.726+0000 7fe59affd700 1 -- 172.21.15.130:0/3003560901 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe5a004bd90 con 0x7fe5a4100610 2023-09-12T15:57:01.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.726+0000 7fe5a9384700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe590038630 0x7fe59003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.726+0000 7fe5a9384700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe590038630 0x7fe59003aaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40740f0 tx=0x7fe594005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:01.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.731+0000 7fe59affd700 1 -- 172.21.15.130:0/3003560901 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe5a0016970 con 0x7fe5a4100610 2023-09-12T15:57:01.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.859+0000 7f2db62be700 1 -- 172.21.15.130:0/3381309239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db00ffc00 msgr2=0x7f2db0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.859+0000 7f2db62be700 1 --2- 172.21.15.130:0/3381309239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db00ffc00 0x7f2db0100080 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009990 tx=0x7f2da4009ca0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.860+0000 7f2db62be700 1 -- 172.21.15.130:0/3381309239 shutdown_connections 2023-09-12T15:57:01.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.860+0000 7f2db62be700 1 --2- 172.21.15.130:0/3381309239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db01005c0 0x7f2db010c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.860+0000 7f2db62be700 1 --2- 172.21.15.130:0/3381309239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db0105540 0x7f2db0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.860+0000 7f2db62be700 1 --2- 172.21.15.130:0/3381309239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db00ffc00 0x7f2db0100080 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.860+0000 7f2db62be700 1 -- 172.21.15.130:0/3381309239 >> 172.21.15.130:0/3381309239 conn(0x7f2db0075a70 msgr2=0x7f2db00fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:01.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.860+0000 7f2db62be700 1 -- 172.21.15.130:0/3381309239 shutdown_connections 2023-09-12T15:57:01.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.860+0000 7f2db62be700 1 -- 172.21.15.130:0/3381309239 wait complete. 2023-09-12T15:57:01.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.860+0000 7f2db62be700 1 Processor -- start 2023-09-12T15:57:01.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.860+0000 7f2db62be700 1 -- start start 2023-09-12T15:57:01.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db62be700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db00ffc00 0x7f2db0197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db62be700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db01005c0 0x7f2db0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db62be700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0105540 0x7f2db01a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db62be700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db0198400 con 0x7f2db0105540 2023-09-12T15:57:01.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db62be700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db01a2b40 con 0x7f2db00ffc00 2023-09-12T15:57:01.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db62be700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db01a2d00 con 0x7f2db01005c0 2023-09-12T15:57:01.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db485b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0105540 0x7f2db01a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db485b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0105540 0x7f2db01a2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52382/0 (socket says 172.21.15.130:52382) 2023-09-12T15:57:01.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db485b700 1 -- 172.21.15.130:0/852901276 learned_addr learned my addr 172.21.15.130:0/852901276 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:57:01.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2daffff700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db00ffc00 0x7f2db0197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2daf7fe700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db01005c0 0x7f2db0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db485b700 1 -- 172.21.15.130:0/852901276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db01005c0 msgr2=0x7f2db0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db485b700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db01005c0 0x7f2db0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db485b700 1 -- 172.21.15.130:0/852901276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db00ffc00 msgr2=0x7f2db0197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db485b700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db00ffc00 0x7f2db0197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.861+0000 7f2db485b700 1 -- 172.21.15.130:0/852901276 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da40095f0 con 0x7f2db0105540 2023-09-12T15:57:01.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.862+0000 7f2db485b700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0105540 0x7f2db01a2600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2da800ada0 tx=0x7f2da800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:01.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.862+0000 7f2dad7fa700 1 -- 172.21.15.130:0/852901276 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da800f070 con 0x7f2db0105540 2023-09-12T15:57:01.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.862+0000 7f2dad7fa700 1 -- 172.21.15.130:0/852901276 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da8007b30 con 0x7f2db0105540 2023-09-12T15:57:01.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.862+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db01a2fe0 con 0x7f2db0105540 2023-09-12T15:57:01.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.862+0000 7f2dad7fa700 1 -- 172.21.15.130:0/852901276 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8005280 con 0x7f2db0105540 2023-09-12T15:57:01.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.862+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db01a3500 con 0x7f2db0105540 2023-09-12T15:57:01.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.863+0000 7f2dad7fa700 1 -- 172.21.15.130:0/852901276 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f2da80053e0 con 0x7f2db0105540 2023-09-12T15:57:01.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.863+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db004e300 con 0x7f2db0105540 2023-09-12T15:57:01.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.863+0000 7f2dad7fa700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d98038630 0x7f2d9803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:01.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.863+0000 7f2dad7fa700 1 -- 172.21.15.130:0/852901276 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2da804a600 con 0x7f2db0105540 2023-09-12T15:57:01.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.863+0000 7f2daffff700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d98038630 0x7f2d9803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:01.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.863+0000 7f2daffff700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d98038630 0x7f2d9803aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2da0005950 tx=0x7f2da000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:01.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.868+0000 7f2dad7fa700 1 -- 172.21.15.130:0/852901276 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2da801a310 con 0x7f2db0105540 2023-09-12T15:57:01.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.940+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7fe5a41106c0 con 0x7fe5a4100610 2023-09-12T15:57:01.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.943+0000 7fe59affd700 1 -- 172.21.15.130:0/3003560901 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v29)=0 v29) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe5a0026030 con 0x7fe5a4100610 2023-09-12T15:57:01.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe590038630 msgr2=0x7fe59003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe590038630 0x7fe59003aaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40740f0 tx=0x7fe594005cf0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a4100610 msgr2=0x7fe5a4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:01.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a4100610 0x7fe5a4072900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0003730 tx=0x7fe5a000f740 comp rx=0 tx=0).stop 2023-09-12T15:57:01.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 shutdown_connections 2023-09-12T15:57:01.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe590038630 0x7fe59003aaf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5a41066a0 0x7fe5a40733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5a4100610 0x7fe5a4072900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 --2- 172.21.15.130:0/3003560901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a4100fd0 0x7fe5a4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:01.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 >> 172.21.15.130:0/3003560901 conn(0x7fe5a40fc100 msgr2=0x7fe5a40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:01.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 shutdown_connections 2023-09-12T15:57:01.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:01.945+0000 7fe5abde9700 1 -- 172.21.15.130:0/3003560901 wait complete. 2023-09-12T15:57:02.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.068+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f2db010f650 con 0x7f2db0105540 2023-09-12T15:57:02.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.072+0000 7f2dad7fa700 1 -- 172.21.15.130:0/852901276 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) v1 ==== 121+0+0 (secure 0 0 0) 0x7f2da8017030 con 0x7f2db0105540 2023-09-12T15:57:02.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d98038630 msgr2=0x7f2d9803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:02.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d98038630 0x7f2d9803aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2da0005950 tx=0x7f2da000a300 comp rx=0 tx=0).stop 2023-09-12T15:57:02.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0105540 msgr2=0x7f2db01a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:02.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0105540 0x7f2db01a2600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2da800ada0 tx=0x7f2da800c330 comp rx=0 tx=0).stop 2023-09-12T15:57:02.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 shutdown_connections 2023-09-12T15:57:02.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d98038630 0x7f2d9803aaf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:02.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2db0105540 0x7f2db01a2600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:02.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db00ffc00 0x7f2db0197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:02.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 --2- 172.21.15.130:0/852901276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db01005c0 0x7f2db0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:02.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 >> 172.21.15.130:0/852901276 conn(0x7f2db0075a70 msgr2=0x7f2db00fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:02.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 shutdown_connections 2023-09-12T15:57:02.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:02.074+0000 7f2db62be700 1 -- 172.21.15.130:0/852901276 wait complete. 2023-09-12T15:57:02.724 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:02 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3035150903' entity='client.admin' 2023-09-12T15:57:02.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:02 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3003560901' entity='client.admin' 2023-09-12T15:57:02.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:02 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/852901276' entity='client.admin' 2023-09-12T15:57:02.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:02 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3035150903' entity='client.admin' 2023-09-12T15:57:02.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:02 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3003560901' entity='client.admin' 2023-09-12T15:57:02.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:02 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/852901276' entity='client.admin' 2023-09-12T15:57:02.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:02 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3035150903' entity='client.admin' 2023-09-12T15:57:02.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:02 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3003560901' entity='client.admin' 2023-09-12T15:57:02.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:02 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/852901276' entity='client.admin' 2023-09-12T15:57:02.862 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-09-12T15:57:02.863 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:57:02.863 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.conf 2023-09-12T15:57:02.894 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:57:02.894 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:57:02.961 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:57:02.961 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2023-09-12T15:57:02.999 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:57:02.999 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:57:03.073 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:57:03.073 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2023-09-12T15:57:03.111 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:57:03.111 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-12T15:57:03.181 INFO:tasks.cephadm:Adding mgr.x on smithi130 2023-09-12T15:57:03.182 INFO:tasks.cephadm:Adding mgr.y on smithi159 2023-09-12T15:57:03.182 INFO:tasks.cephadm:Adding mgr.z on smithi203 2023-09-12T15:57:03.183 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch apply mgr '3;smithi130=x;smithi159=y;smithi203=z' 2023-09-12T15:57:03.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:03 smithi130 ceph-mon[98670]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:03.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:03 smithi159 ceph-mon[102522]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:03.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:03 smithi203 ceph-mon[103431]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:03.771 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T15:57:05.000 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 -- 172.21.15.203:0/429610578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 msgr2=0x7f152c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:05.000 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 --2- 172.21.15.203:0/429610578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 0x7f152c108ae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f152400b0a0 tx=0x7f152400b3b0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.000 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 -- 172.21.15.203:0/429610578 shutdown_connections 2023-09-12T15:57:05.001 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 --2- 172.21.15.203:0/429610578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 0x7f152c108ae0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.001 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 --2- 172.21.15.203:0/429610578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100690 0x7f152c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.001 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 --2- 172.21.15.203:0/429610578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f152c101890 0x7f152c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.001 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 -- 172.21.15.203:0/429610578 >> 172.21.15.203:0/429610578 conn(0x7f152c0fbbf0 msgr2=0x7f152c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:05.001 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 -- 172.21.15.203:0/429610578 shutdown_connections 2023-09-12T15:57:05.001 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 -- 172.21.15.203:0/429610578 wait complete. 2023-09-12T15:57:05.002 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 Processor -- start 2023-09-12T15:57:05.002 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.998+0000 7f1533aa2700 1 -- start start 2023-09-12T15:57:05.002 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1533aa2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100690 0x7f152c0743a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:05.002 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1533aa2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f152c101890 0x7f152c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:05.002 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1533aa2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 0x7f152c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1533aa2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0736b0 con 0x7f152c101890 2023-09-12T15:57:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1533aa2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c073820 con 0x7f152c100690 2023-09-12T15:57:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1533aa2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1a0480 con 0x7f152c102230 2023-09-12T15:57:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153203f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 0x7f152c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:05.004 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153203f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 0x7f152c072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:41380/0 (socket says 172.21.15.203:41380) 2023-09-12T15:57:05.004 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153203f700 1 -- 172.21.15.203:0/971691544 learned_addr learned my addr 172.21.15.203:0/971691544 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-09-12T15:57:05.004 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153103d700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f152c101890 0x7f152c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:05.004 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153203f700 1 -- 172.21.15.203:0/971691544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100690 msgr2=0x7f152c0743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:05.004 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153203f700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100690 0x7f152c0743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.005 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153203f700 1 -- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f152c101890 msgr2=0x7f152c0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:05.005 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153203f700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f152c101890 0x7f152c0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.005 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153203f700 1 -- 172.21.15.203:0/971691544 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524009d00 con 0x7f152c102230 2023-09-12T15:57:05.005 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f153203f700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 0x7f152c072f30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f152400b780 tx=0x7f152400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:05.005 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1522ffd700 1 -- 172.21.15.203:0/971691544 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152400e040 con 0x7f152c102230 2023-09-12T15:57:05.005 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1522ffd700 1 -- 172.21.15.203:0/971691544 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1524004e60 con 0x7f152c102230 2023-09-12T15:57:05.006 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1522ffd700 1 -- 172.21.15.203:0/971691544 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240053b0 con 0x7f152c102230 2023-09-12T15:57:05.006 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c1a0700 con 0x7f152c102230 2023-09-12T15:57:05.006 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:04.999+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c1a0b70 con 0x7f152c102230 2023-09-12T15:57:05.006 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.000+0000 7f1522ffd700 1 -- 172.21.15.203:0/971691544 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f1524005510 con 0x7f152c102230 2023-09-12T15:57:05.006 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.000+0000 7f1522ffd700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1518038630 0x7f151803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:05.007 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.000+0000 7f1522ffd700 1 -- 172.21.15.203:0/971691544 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f152404c620 con 0x7f152c102230 2023-09-12T15:57:05.007 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.000+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c10bea0 con 0x7f152c102230 2023-09-12T15:57:05.007 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.001+0000 7f153183e700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1518038630 0x7f151803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:05.007 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.001+0000 7f153183e700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1518038630 0x7f151803aaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f152c1016f0 tx=0x7f1528009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:05.008 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.005+0000 7f1522ffd700 1 -- 172.21.15.203:0/971691544 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1524021030 con 0x7f152c102230 2023-09-12T15:57:05.203 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.204+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi130=x;smithi159=y;smithi203=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f152c060a40 con 0x7f1518038630 2023-09-12T15:57:05.210 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.211+0000 7f1522ffd700 1 -- 172.21.15.203:0/971691544 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f152c060a40 con 0x7f1518038630 2023-09-12T15:57:05.210 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mgr update... 2023-09-12T15:57:05.213 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1518038630 msgr2=0x7f151803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:05.213 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1518038630 0x7f151803aaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f152c1016f0 tx=0x7f1528009470 comp rx=0 tx=0).stop 2023-09-12T15:57:05.213 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 msgr2=0x7f152c072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:05.213 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 0x7f152c072f30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f152400b780 tx=0x7f152400bd60 comp rx=0 tx=0).stop 2023-09-12T15:57:05.214 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 shutdown_connections 2023-09-12T15:57:05.214 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1518038630 0x7f151803aaf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.214 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152c102230 0x7f152c072f30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.214 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100690 0x7f152c0743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.214 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 --2- 172.21.15.203:0/971691544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f152c101890 0x7f152c0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:05.214 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 >> 172.21.15.203:0/971691544 conn(0x7f152c0fbbf0 msgr2=0x7f152c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:05.215 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 shutdown_connections 2023-09-12T15:57:05.215 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:05.213+0000 7f1533aa2700 1 -- 172.21.15.203:0/971691544 wait complete. 2023-09-12T15:57:05.625 DEBUG:teuthology.orchestra.run.smithi159:mgr.y> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.y.service 2023-09-12T15:57:05.628 DEBUG:teuthology.orchestra.run.smithi203:mgr.z> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.z.service 2023-09-12T15:57:05.630 INFO:tasks.cephadm:Deploying OSDs... 2023-09-12T15:57:05.630 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T15:57:05.631 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/scratch_devs of=/dev/stdout 2023-09-12T15:57:05.647 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-12T15:57:05.647 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_1 2023-09-12T15:57:05.659 INFO:journalctl@ceph.mgr.y.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T15:57:05.662 INFO:journalctl@ceph.mgr.z.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T15:57:05.705 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-12T15:57:05.705 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:05.705 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 382628 Links: 1 2023-09-12T15:57:05.705 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:05.706 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:05.706 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2023-09-12 15:56:00.726497782 +0000 2023-09-12T15:57:05.706 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2023-09-12 15:50:44.661965511 +0000 2023-09-12T15:57:05.706 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2023-09-12 15:50:44.661965511 +0000 2023-09-12T15:57:05.706 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: - 2023-09-12T15:57:05.707 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-12T15:57:05.771 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2023-09-12T15:57:05.771 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2023-09-12T15:57:05.771 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000223413 s, 2.3 MB/s 2023-09-12T15:57:05.772 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-12T15:57:05.829 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_2 2023-09-12T15:57:05.886 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-12T15:57:05.886 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:05.886 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 389523 Links: 1 2023-09-12T15:57:05.886 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:05.887 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:05.887 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2023-09-12 15:56:00.766497092 +0000 2023-09-12T15:57:05.887 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2023-09-12 15:50:45.166956751 +0000 2023-09-12T15:57:05.887 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2023-09-12 15:50:45.166956751 +0000 2023-09-12T15:57:05.887 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: - 2023-09-12T15:57:05.888 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-12T15:57:05.951 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2023-09-12T15:57:05.951 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2023-09-12T15:57:05.952 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000240316 s, 2.1 MB/s 2023-09-12T15:57:05.953 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-12T15:57:05.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:05.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:05.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:05.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:05.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-12T15:57:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-12T15:57:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-12T15:57:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:05.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:05.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:05.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:05.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-12T15:57:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-12T15:57:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-12T15:57:05.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:06.010 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_3 2023-09-12T15:57:06.068 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-12T15:57:06.068 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:06.068 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 389610 Links: 1 2023-09-12T15:57:06.068 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:06.069 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:06.069 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2023-09-12 15:56:00.807496386 +0000 2023-09-12T15:57:06.069 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2023-09-12 15:50:45.649948373 +0000 2023-09-12T15:57:06.069 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2023-09-12 15:50:45.649948373 +0000 2023-09-12T15:57:06.069 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: - 2023-09-12T15:57:06.070 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-12T15:57:06.131 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:06.132 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:06.132 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:06.132 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:06.132 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:06.132 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:06.133 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-12T15:57:06.133 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-12T15:57:06.133 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-12T15:57:06.133 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:06.134 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2023-09-12T15:57:06.134 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2023-09-12T15:57:06.134 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000215136 s, 2.4 MB/s 2023-09-12T15:57:06.135 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-12T15:57:06.193 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_4 2023-09-12T15:57:06.250 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-12T15:57:06.251 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:06.251 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 390578 Links: 1 2023-09-12T15:57:06.251 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:06.251 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:06.252 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2023-09-12 15:56:00.848495679 +0000 2023-09-12T15:57:06.252 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2023-09-12 15:50:46.161939491 +0000 2023-09-12T15:57:06.252 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2023-09-12 15:50:46.161939491 +0000 2023-09-12T15:57:06.252 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: - 2023-09-12T15:57:06.253 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-12T15:57:06.316 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2023-09-12T15:57:06.316 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2023-09-12T15:57:06.316 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000189156 s, 2.7 MB/s 2023-09-12T15:57:06.317 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-12T15:57:06.375 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-09-12T15:57:06.375 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2023-09-12T15:57:06.396 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-12T15:57:06.397 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2023-09-12T15:57:06.457 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-12T15:57:06.457 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:06.458 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 389337 Links: 1 2023-09-12T15:57:06.458 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:06.458 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:06.458 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-09-12 15:56:34.288872931 +0000 2023-09-12T15:57:06.458 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-09-12 15:51:01.490906984 +0000 2023-09-12T15:57:06.458 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-09-12 15:51:01.490906984 +0000 2023-09-12T15:57:06.459 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-09-12T15:57:06.459 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-12T15:57:06.527 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-09-12T15:57:06.528 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-09-12T15:57:06.528 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000236085 s, 2.2 MB/s 2023-09-12T15:57:06.529 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-12T15:57:06.589 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2023-09-12T15:57:06.650 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-12T15:57:06.650 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:06.650 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 387449 Links: 1 2023-09-12T15:57:06.651 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:06.651 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:06.651 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-09-12 15:56:34.288872931 +0000 2023-09-12T15:57:06.651 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-09-12 15:51:02.000897259 +0000 2023-09-12T15:57:06.652 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-09-12 15:51:02.000897259 +0000 2023-09-12T15:57:06.652 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-09-12T15:57:06.652 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-12T15:57:06.717 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-09-12T15:57:06.718 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-09-12T15:57:06.718 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000236993 s, 2.2 MB/s 2023-09-12T15:57:06.719 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-12T15:57:06.779 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2023-09-12T15:57:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-12T15:57:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:06.839 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 383780 Links: 1 2023-09-12T15:57:06.839 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:06.840 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:06.840 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-09-12 15:56:34.288872931 +0000 2023-09-12T15:57:06.840 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-09-12 15:51:02.500887723 +0000 2023-09-12T15:57:06.840 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-09-12 15:51:02.500887723 +0000 2023-09-12T15:57:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-09-12T15:57:06.841 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-12T15:57:06.902 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:06 smithi159 ceph-mon[102522]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi130=x;smithi159=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:06.902 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:06 smithi159 ceph-mon[102522]: Saving service mgr spec with placement smithi130=x;smithi159=y;smithi203=z;count:3 2023-09-12T15:57:06.903 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:06 smithi159 ceph-mon[102522]: Deploying daemon mgr.z on smithi203 2023-09-12T15:57:06.904 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-09-12T15:57:06.905 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-09-12T15:57:06.905 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000244873 s, 2.1 MB/s 2023-09-12T15:57:06.906 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-12T15:57:06.965 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2023-09-12T15:57:06.979 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:06 smithi203 ceph-mon[103431]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi130=x;smithi159=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:06.980 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:06 smithi203 ceph-mon[103431]: Saving service mgr spec with placement smithi130=x;smithi159=y;smithi203=z;count:3 2023-09-12T15:57:06.980 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:06 smithi203 ceph-mon[103431]: Deploying daemon mgr.z on smithi203 2023-09-12T15:57:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-12T15:57:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:07.022 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 384940 Links: 1 2023-09-12T15:57:07.022 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:07.023 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:07.023 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-09-12 15:56:34.288872931 +0000 2023-09-12T15:57:07.023 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-09-12 15:51:02.993878321 +0000 2023-09-12T15:57:07.023 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-09-12 15:51:02.993878321 +0000 2023-09-12T15:57:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-09-12T15:57:07.024 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-12T15:57:07.088 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-09-12T15:57:07.088 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-09-12T15:57:07.089 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000189576 s, 2.7 MB/s 2023-09-12T15:57:07.090 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-12T15:57:07.147 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-09-12T15:57:07.148 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2023-09-12T15:57:07.165 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-12T15:57:07.166 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2023-09-12T15:57:07.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:06 smithi130 ceph-mon[98670]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi130=x;smithi159=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:07.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:06 smithi130 ceph-mon[98670]: Saving service mgr spec with placement smithi130=x;smithi159=y;smithi203=z;count:3 2023-09-12T15:57:07.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:06 smithi130 ceph-mon[98670]: Deploying daemon mgr.z on smithi203 2023-09-12T15:57:07.226 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-12T15:57:07.226 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:07.227 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 384871 Links: 1 2023-09-12T15:57:07.227 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:07.227 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:07.227 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-09-12 15:57:07.204153101 +0000 2023-09-12T15:57:07.227 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-09-12 15:51:01.688584234 +0000 2023-09-12T15:57:07.227 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-09-12 15:51:01.688584234 +0000 2023-09-12T15:57:07.228 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-09-12T15:57:07.228 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-12T15:57:07.296 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-09-12T15:57:07.297 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-09-12T15:57:07.297 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000251105 s, 2.0 MB/s 2023-09-12T15:57:07.299 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-12T15:57:07.359 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2023-09-12T15:57:07.417 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-12T15:57:07.418 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:07.418 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 382423 Links: 1 2023-09-12T15:57:07.418 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:07.418 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:07.419 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-09-12 15:57:07.204153101 +0000 2023-09-12T15:57:07.419 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-09-12 15:51:02.214571295 +0000 2023-09-12T15:57:07.419 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-09-12 15:51:02.214571295 +0000 2023-09-12T15:57:07.419 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-09-12T15:57:07.419 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-12T15:57:07.483 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-09-12T15:57:07.484 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-09-12T15:57:07.484 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000229219 s, 2.2 MB/s 2023-09-12T15:57:07.485 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-12T15:57:07.543 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2023-09-12T15:57:07.579 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:07 smithi203 systemd[1]: Starting Ceph mgr.z for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T15:57:07.596 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-12T15:57:07.596 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:07.596 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 381393 Links: 1 2023-09-12T15:57:07.596 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:07.596 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:07.597 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-09-12 15:57:07.204153101 +0000 2023-09-12T15:57:07.597 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-09-12 15:51:02.723558775 +0000 2023-09-12T15:57:07.597 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-09-12 15:51:02.723558775 +0000 2023-09-12T15:57:07.597 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-09-12T15:57:07.598 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-12T15:57:07.662 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-09-12T15:57:07.663 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-09-12T15:57:07.663 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00023097 s, 2.2 MB/s 2023-09-12T15:57:07.664 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-12T15:57:07.722 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2023-09-12T15:57:07.779 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-12T15:57:07.779 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T15:57:07.780 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 385865 Links: 1 2023-09-12T15:57:07.780 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T15:57:07.780 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-09-12T15:57:07.780 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-09-12 15:57:07.204153101 +0000 2023-09-12T15:57:07.781 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-09-12 15:51:03.216546647 +0000 2023-09-12T15:57:07.781 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-09-12 15:51:03.216546647 +0000 2023-09-12T15:57:07.781 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-09-12T15:57:07.781 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-12T15:57:07.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:07 smithi203 ceph-mon[103431]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:07.843 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:07 smithi203 podman[105390]: 2023-09-12T15:57:07.845 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-09-12T15:57:07.845 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-09-12T15:57:07.845 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000191285 s, 2.7 MB/s 2023-09-12T15:57:07.846 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-12T15:57:07.904 INFO:tasks.cephadm:Deploying osd.0 on smithi130 with /dev/vg_nvme/lv_4... 2023-09-12T15:57:07.904 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-12T15:57:08.162 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:07 smithi203 ceph-mgr[105593]: -- 172.21.15.203:0/4069284918 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5637738051e0 con 0x5637737db000 2023-09-12T15:57:08.162 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:07 smithi203 bash[105390]: 5df2b073fd917520a036c23ac20998eddf6c799e121a63dfa01931c822be23ac 2023-09-12T15:57:08.162 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:07 smithi203 systemd[1]: Started Ceph mgr.z for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T15:57:08.163 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:07 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:07.992+0000 7fe1c8213200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-12T15:57:08.163 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:08.073+0000 7fe1c8213200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-12T15:57:08.163 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:08.162+0000 7fe1c8213200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-12T15:57:08.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:07 smithi130 ceph-mon[98670]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:08.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:07 smithi159 ceph-mon[102522]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:08.426 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:08.255+0000 7fe1c8213200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-12T15:57:08.470 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:57:08.745 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:08.425+0000 7fe1c8213200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-12T15:57:09.101 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:08.790+0000 7fe1c8213200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-12T15:57:09.101 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:08.919+0000 7fe1c8213200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-12T15:57:09.101 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:09 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:09.007+0000 7fe1c8213200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-12T15:57:09.102 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:09.102 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:09.102 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:09.102 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-12T15:57:09.102 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-12T15:57:09.103 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-12T15:57:09.103 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:09.103 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-mon[103431]: Deploying daemon mgr.y on smithi159 2023-09-12T15:57:09.103 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:08 smithi203 ceph-mon[103431]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:09.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:09.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:09.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:09.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-12T15:57:09.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-12T15:57:09.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-12T15:57:09.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:08 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:09.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:08 smithi130 ceph-mon[98670]: Deploying daemon mgr.y on smithi159 2023-09-12T15:57:09.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:08 smithi130 ceph-mon[98670]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:09.287 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:08 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:09.287 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:08 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:09.287 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:08 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:09.288 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:08 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-12T15:57:09.288 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:08 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-12T15:57:09.288 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:08 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-12T15:57:09.288 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:08 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:09.288 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:08 smithi159 ceph-mon[102522]: Deploying daemon mgr.y on smithi159 2023-09-12T15:57:09.289 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:08 smithi159 ceph-mon[102522]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:09.726 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:09 smithi159 systemd[1]: Starting Ceph mgr.y for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T15:57:09.833 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-12T15:57:09.834 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-12T15:57:09.834 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2023-09-12T15:57:09.834 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2023-09-12T15:57:09.834 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111162 s, 943 MB/s 2023-09-12T15:57:09.835 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 2023-09-12T15:57:09.835 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2023-09-12T15:57:09.996 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:09 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:09.101+0000 7fe1c8213200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-12T15:57:09.996 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:09 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:09.521+0000 7fe1c8213200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-12T15:57:10.091 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:09 smithi159 podman[103985]: 2023-09-12T15:57:10.092 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:10 smithi159 bash[103985]: 0d23bafa325684c53f53c11f35d69c2b225c442a36a370a94e2ec4dd2975b7f2 2023-09-12T15:57:10.092 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:10 smithi159 ceph-mgr[104025]: -- 172.21.15.159:0/3727008310 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5566f86e11e0 con 0x5566f86b6800 2023-09-12T15:57:10.311 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi130:vg_nvme/lv_4 2023-09-12T15:57:10.422 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:10 smithi159 systemd[1]: Started Ceph mgr.y for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T15:57:10.422 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:10.154+0000 7faa72e67200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-12T15:57:10.422 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:10.235+0000 7faa72e67200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-12T15:57:10.422 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:10.325+0000 7faa72e67200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-12T15:57:10.423 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:10.422+0000 7faa72e67200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-12T15:57:10.495 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:10 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:10.189+0000 7fe1c8213200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-12T15:57:10.495 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:10 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:10.271+0000 7fe1c8213200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-12T15:57:10.725 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:10.604+0000 7faa72e67200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-12T15:57:10.869 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:57:11.318 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:10.979+0000 7faa72e67200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-12T15:57:11.318 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:11.121+0000 7faa72e67200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-12T15:57:11.318 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:11.218+0000 7faa72e67200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-12T15:57:11.319 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:11.318+0000 7faa72e67200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-12T15:57:11.319 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.319 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.319 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:11.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-mon[102522]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:11.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.383 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:11.098+0000 7fe1c8213200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:57:11.384 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:11.195+0000 7fe1c8213200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-12T15:57:11.384 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:11.383+0000 7fe1c8213200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:57:11.384 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.384 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.385 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.385 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.385 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:11.385 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-mon[103431]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:11.385 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.415 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.415 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.416 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.416 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.416 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:11.416 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:11 smithi130 ceph-mon[98670]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:11.416 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:11.995 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:11.688+0000 7fe1c8213200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-12T15:57:11.995 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:11.928+0000 7fe1c8213200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-12T15:57:12.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 -- 172.21.15.130:0/4218044588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41006b0 msgr2=0x7f72a4100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:12.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/4218044588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41006b0 0x7f72a4100ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f72980099c0 tx=0x7f7298009cd0 comp rx=0 tx=0).stop 2023-09-12T15:57:12.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 -- 172.21.15.130:0/4218044588 shutdown_connections 2023-09-12T15:57:12.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/4218044588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a4102250 0x7f72a4108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:12.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/4218044588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41006b0 0x7f72a4100ad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:12.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/4218044588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a41018b0 0x7f72a4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:12.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 -- 172.21.15.130:0/4218044588 >> 172.21.15.130:0/4218044588 conn(0x7f72a40fbc10 msgr2=0x7f72a40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:12.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 -- 172.21.15.130:0/4218044588 shutdown_connections 2023-09-12T15:57:12.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 -- 172.21.15.130:0/4218044588 wait complete. 2023-09-12T15:57:12.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.013+0000 7f72a9cdf700 1 Processor -- start 2023-09-12T15:57:12.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a9cdf700 1 -- start start 2023-09-12T15:57:12.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a9cdf700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a41006b0 0x7f72a41963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:12.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a9cdf700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41018b0 0x7f72a41968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:12.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a9cdf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a4102250 0x7f72a419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:12.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a9cdf700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419df30 con 0x7f72a41018b0 2023-09-12T15:57:12.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a9cdf700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419e070 con 0x7f72a41006b0 2023-09-12T15:57:12.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a9cdf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419e200 con 0x7f72a4102250 2023-09-12T15:57:12.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41018b0 0x7f72a41968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:12.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41018b0 0x7f72a41968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52396/0 (socket says 172.21.15.130:52396) 2023-09-12T15:57:12.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a2ffd700 1 -- 172.21.15.130:0/2789672654 learned_addr learned my addr 172.21.15.130:0/2789672654 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:57:12.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a37fe700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a41006b0 0x7f72a41963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:12.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a3fff700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a4102250 0x7f72a419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:12.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a2ffd700 1 -- 172.21.15.130:0/2789672654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a4102250 msgr2=0x7f72a419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:12.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a2ffd700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a4102250 0x7f72a419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:12.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a2ffd700 1 -- 172.21.15.130:0/2789672654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a41006b0 msgr2=0x7f72a41963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:12.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a2ffd700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a41006b0 0x7f72a41963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:12.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a2ffd700 1 -- 172.21.15.130:0/2789672654 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72980095f0 con 0x7f72a41018b0 2023-09-12T15:57:12.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a2ffd700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41018b0 0x7f72a41968e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f729400e9d0 tx=0x7f729400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:12.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.014+0000 7f72a0ff9700 1 -- 172.21.15.130:0/2789672654 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294009790 con 0x7f72a41018b0 2023-09-12T15:57:12.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.015+0000 7f72a0ff9700 1 -- 172.21.15.130:0/2789672654 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f729400ee70 con 0x7f72a41018b0 2023-09-12T15:57:12.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.015+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a419e4e0 con 0x7f72a41018b0 2023-09-12T15:57:12.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.015+0000 7f72a0ff9700 1 -- 172.21.15.130:0/2789672654 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294010ea0 con 0x7f72a41018b0 2023-09-12T15:57:12.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.015+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a419ea30 con 0x7f72a41018b0 2023-09-12T15:57:12.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.015+0000 7f72a0ff9700 1 -- 172.21.15.130:0/2789672654 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) v1 ==== 45388+0+0 (secure 0 0 0) 0x7f7294018680 con 0x7f72a41018b0 2023-09-12T15:57:12.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.016+0000 7f72a0ff9700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f728c038630 0x7f728c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:12.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.016+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a410bec0 con 0x7f72a41018b0 2023-09-12T15:57:12.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.016+0000 7f72a0ff9700 1 -- 172.21.15.130:0/2789672654 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7294014070 con 0x7f72a41018b0 2023-09-12T15:57:12.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.016+0000 7f72a37fe700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f728c038630 0x7f728c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:12.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.016+0000 7f72a37fe700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f728c038630 0x7f728c03aaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7298009990 tx=0x7f7298005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:12.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.020+0000 7f72a0ff9700 1 -- 172.21.15.130:0/2789672654 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f72940173d0 con 0x7f72a41018b0 2023-09-12T15:57:12.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:12.222+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f72a4060a40 con 0x7f728c038630 2023-09-12T15:57:12.225 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:11 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:11.762+0000 7faa72e67200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-12T15:57:12.336 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:12.077+0000 7fe1c8213200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-12T15:57:12.725 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:12.459+0000 7faa72e67200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-12T15:57:12.726 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:12.546+0000 7faa72e67200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-12T15:57:12.745 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:12.337+0000 7fe1c8213200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-12T15:57:13.176 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:13.045+0000 7fe1c8213200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-12T15:57:13.176 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.177 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.177 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.177 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.177 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:57:13.177 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:57:13.178 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:13.178 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:13.178 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:13.179 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.179 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-12T15:57:13.179 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-12T15:57:13.179 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:13.180 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.180 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.180 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.181 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.181 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:57:13.181 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:57:13.181 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:13.181 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:13.182 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:13.182 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.182 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-12T15:57:13.183 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-12T15:57:13.183 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:13.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:57:13.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:57:13.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:13.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:13.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:13.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:13.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-12T15:57:13.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-12T15:57:13.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:13.483 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:13.176+0000 7fe1c8213200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-12T15:57:13.483 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:13.369+0000 7fe1c8213200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-12T15:57:13.483 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:13.483+0000 7fe1c8213200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-12T15:57:13.575 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:13.315+0000 7faa72e67200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:57:13.576 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:13.405+0000 7faa72e67200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-12T15:57:13.745 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:13.699+0000 7fe1c8213200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-12T15:57:13.850 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:13.575+0000 7faa72e67200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-12T15:57:14.163 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:13 smithi130 ceph-mon[98670]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:14.164 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:13 smithi130 ceph-mon[98670]: Reconfiguring mgr.x (unknown last config time)... 2023-09-12T15:57:14.164 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:13 smithi130 ceph-mon[98670]: Reconfiguring daemon mgr.x on smithi130 2023-09-12T15:57:14.164 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:13 smithi130 ceph-mon[98670]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:14.225 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:13.850+0000 7faa72e67200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-12T15:57:14.226 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:14.085+0000 7faa72e67200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-12T15:57:14.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:13 smithi159 ceph-mon[102522]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:14.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:13 smithi159 ceph-mon[102522]: Reconfiguring mgr.x (unknown last config time)... 2023-09-12T15:57:14.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:13 smithi159 ceph-mon[102522]: Reconfiguring daemon mgr.x on smithi130 2023-09-12T15:57:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:13 smithi159 ceph-mon[102522]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:14.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:13 smithi203 ceph-mon[103431]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:14.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:13 smithi203 ceph-mon[103431]: Reconfiguring mgr.x (unknown last config time)... 2023-09-12T15:57:14.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:13 smithi203 ceph-mon[103431]: Reconfiguring daemon mgr.x on smithi130 2023-09-12T15:57:14.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:13 smithi203 ceph-mon[103431]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:14.491 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:14.232+0000 7faa72e67200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-12T15:57:14.904 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:14.491+0000 7faa72e67200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-12T15:57:14.904 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z[105503]: 2023-09-12T15:57:14.566+0000 7fe1c8213200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-12T15:57:14.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:14.951+0000 7f72a0ff9700 1 -- 172.21.15.130:0/2789672654 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mgrmap(e 14) v1 ==== 90296+0+0 (secure 0 0 0) 0x7f729400cbd0 con 0x7f72a41018b0 2023-09-12T15:57:15.191 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:15.192 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:15.192 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:15.192 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:15.192 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:15.192 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: Standby manager daemon z started 2023-09-12T15:57:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-12T15:57:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-12T15:57:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-12T15:57:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:14 smithi159 ceph-mon[102522]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-12T15:57:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: Standby manager daemon z started 2023-09-12T15:57:15.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-12T15:57:15.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-12T15:57:15.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-12T15:57:15.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:14 smithi130 ceph-mon[98670]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-12T15:57:15.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:15.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:15.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:15.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:15.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:15.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:15.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: Standby manager daemon z started 2023-09-12T15:57:15.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-12T15:57:15.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-12T15:57:15.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-12T15:57:15.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:14 smithi203 ceph-mon[103431]: from='mgr.? 172.21.15.203:0/3176330637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-12T15:57:15.475 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:15.191+0000 7faa72e67200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-12T15:57:15.476 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:15.322+0000 7faa72e67200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-12T15:57:15.843 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:15.513+0000 7faa72e67200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-12T15:57:15.844 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:15.627+0000 7faa72e67200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-12T15:57:16.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:15 smithi130 ceph-mon[98670]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:16.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:15 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/937927042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c"}]: dispatch 2023-09-12T15:57:16.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:15 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c"}]: dispatch 2023-09-12T15:57:16.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:15 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c"}]': finished 2023-09-12T15:57:16.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:15 smithi130 ceph-mon[98670]: osdmap e5: 1 total, 0 up, 1 in 2023-09-12T15:57:16.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:16.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:15 smithi130 ceph-mon[98670]: mgrmap e14: x(active, since 84s), standbys: z 2023-09-12T15:57:16.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-12T15:57:16.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:15 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/849865548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:57:16.227 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:15.844+0000 7faa72e67200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-12T15:57:16.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-mon[102522]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:16.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/937927042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c"}]: dispatch 2023-09-12T15:57:16.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c"}]: dispatch 2023-09-12T15:57:16.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c"}]': finished 2023-09-12T15:57:16.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-mon[102522]: osdmap e5: 1 total, 0 up, 1 in 2023-09-12T15:57:16.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:16.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-mon[102522]: mgrmap e14: x(active, since 84s), standbys: z 2023-09-12T15:57:16.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-12T15:57:16.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:15 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/849865548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:57:16.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:15 smithi203 ceph-mon[103431]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:16.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:15 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/937927042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c"}]: dispatch 2023-09-12T15:57:16.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:15 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c"}]: dispatch 2023-09-12T15:57:16.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:15 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c"}]': finished 2023-09-12T15:57:16.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:15 smithi203 ceph-mon[103431]: osdmap e5: 1 total, 0 up, 1 in 2023-09-12T15:57:16.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:15 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:16.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:15 smithi203 ceph-mon[103431]: mgrmap e14: x(active, since 84s), standbys: z 2023-09-12T15:57:16.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:15 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-12T15:57:16.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:15 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/849865548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:57:16.975 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 15:57:16 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y[104001]: 2023-09-12T15:57:16.721+0000 7faa72e67200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-12T15:57:17.532 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:17.532+0000 7f72a0ff9700 1 -- 172.21.15.130:0/2789672654 <== mon.0 v2:172.21.15.130:3300/0 8 ==== mgrmap(e 15) v1 ==== 135204+0+0 (secure 0 0 0) 0x7f729401dd30 con 0x7f72a41018b0 2023-09-12T15:57:17.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:17 smithi130 ceph-mon[98670]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:17.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:17 smithi130 ceph-mon[98670]: Standby manager daemon y started 2023-09-12T15:57:17.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:17 smithi130 ceph-mon[98670]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-12T15:57:17.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:17 smithi130 ceph-mon[98670]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-12T15:57:17.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:17 smithi130 ceph-mon[98670]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-12T15:57:17.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:17 smithi130 ceph-mon[98670]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-12T15:57:17.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:17 smithi159 ceph-mon[102522]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:17.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:17 smithi159 ceph-mon[102522]: Standby manager daemon y started 2023-09-12T15:57:17.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:17 smithi159 ceph-mon[102522]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-12T15:57:17.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:17 smithi159 ceph-mon[102522]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-12T15:57:17.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:17 smithi159 ceph-mon[102522]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-12T15:57:17.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:17 smithi159 ceph-mon[102522]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-12T15:57:17.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:17 smithi203 ceph-mon[103431]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:17.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:17 smithi203 ceph-mon[103431]: Standby manager daemon y started 2023-09-12T15:57:17.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:17 smithi203 ceph-mon[103431]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-12T15:57:17.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:17 smithi203 ceph-mon[103431]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-12T15:57:17.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:17 smithi203 ceph-mon[103431]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-12T15:57:17.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:17 smithi203 ceph-mon[103431]: from='mgr.? 172.21.15.159:0/3509633202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-12T15:57:18.882 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:18 smithi130 ceph-mon[98670]: mgrmap e15: x(active, since 86s), standbys: y, z 2023-09-12T15:57:18.883 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-12T15:57:18.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:18 smithi159 ceph-mon[102522]: mgrmap e15: x(active, since 86s), standbys: y, z 2023-09-12T15:57:18.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-12T15:57:18.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:18 smithi203 ceph-mon[103431]: mgrmap e15: x(active, since 86s), standbys: y, z 2023-09-12T15:57:18.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-12T15:57:19.888 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:19 smithi130 ceph-mon[98670]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:19.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:19 smithi159 ceph-mon[102522]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:19.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:19 smithi203 ceph-mon[103431]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:22.013 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:21 smithi130 ceph-mon[98670]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:22.014 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-12T15:57:22.014 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:22.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:21 smithi159 ceph-mon[102522]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:22.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:21 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-12T15:57:22.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:21 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:22.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:21 smithi203 ceph-mon[103431]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:22.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:21 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-12T15:57:22.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:21 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:22.872 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:22 smithi130 ceph-mon[98670]: Deploying daemon osd.0 on smithi130 2023-09-12T15:57:23.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:22 smithi159 ceph-mon[102522]: Deploying daemon osd.0 on smithi130 2023-09-12T15:57:23.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:22 smithi203 ceph-mon[103431]: Deploying daemon osd.0 on smithi130 2023-09-12T15:57:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:23 smithi130 ceph-mon[98670]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:24.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:23 smithi159 ceph-mon[102522]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:24.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:23 smithi203 ceph-mon[103431]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:26.160 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:25 smithi130 ceph-mon[98670]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:26.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:25 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:26.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:25 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:26.161 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:25 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:25 smithi159 ceph-mon[102522]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:25 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:25 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:25 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:25 smithi203 ceph-mon[103431]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:25 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:26.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:25 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:26.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:25 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:27.679 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:27 smithi130 ceph-mon[98670]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:27.680 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:27.681 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:27.681 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:27.681 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:27.681 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:27.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.781+0000 7f72a0ff9700 1 -- 172.21.15.130:0/2789672654 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f72a4060a40 con 0x7f728c038630 2023-09-12T15:57:27.782 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 0 on host 'smithi130' 2023-09-12T15:57:27.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.783+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f728c038630 msgr2=0x7f728c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:27.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.783+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f728c038630 0x7f728c03aaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7298009990 tx=0x7f7298005fb0 comp rx=0 tx=0).stop 2023-09-12T15:57:27.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.783+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41018b0 msgr2=0x7f72a41968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:27.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.783+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41018b0 0x7f72a41968e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f729400e9d0 tx=0x7f729400ece0 comp rx=0 tx=0).stop 2023-09-12T15:57:27.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.784+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 shutdown_connections 2023-09-12T15:57:27.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.784+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f728c038630 0x7f728c03aaf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:27.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.784+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a4102250 0x7f72a419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:27.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.784+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a41006b0 0x7f72a41963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:27.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.784+0000 7f72a9cdf700 1 --2- 172.21.15.130:0/2789672654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72a41018b0 0x7f72a41968e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:27.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.784+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 >> 172.21.15.130:0/2789672654 conn(0x7f72a40fbc10 msgr2=0x7f72a40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:27.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.784+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 shutdown_connections 2023-09-12T15:57:27.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:57:27.784+0000 7f72a9cdf700 1 -- 172.21.15.130:0/2789672654 wait complete. 2023-09-12T15:57:27.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:27 smithi159 ceph-mon[102522]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:27.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:27 smithi203 ceph-mon[103431]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:27.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:27.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:27.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:27.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:28.146 DEBUG:teuthology.orchestra.run.smithi130:osd.0> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.0.service 2023-09-12T15:57:28.148 INFO:tasks.cephadm:Deploying osd.1 on smithi159 with /dev/vg_nvme/lv_4... 2023-09-12T15:57:28.149 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-12T15:57:28.177 INFO:journalctl@ceph.osd.0.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T15:57:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T15:57:28.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:28.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:28.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:28.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:28.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:28.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:29.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:29.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:29.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:29.557 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:57:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-0[121577]: 2023-09-12T15:57:29.348+0000 7f63421bc640 -1 osd.0 0 log_to_monitors true 2023-09-12T15:57:29.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:29 smithi203 ceph-mon[103431]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:29.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:29 smithi203 ceph-mon[103431]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-12T15:57:30.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:29 smithi130 ceph-mon[98670]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:30.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:29 smithi130 ceph-mon[98670]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-12T15:57:30.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:29 smithi159 ceph-mon[102522]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:30.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:29 smithi159 ceph-mon[102522]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-12T15:57:30.245 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-12T15:57:30.246 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-12T15:57:30.246 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-09-12T15:57:30.246 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-09-12T15:57:30.246 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111198 s, 943 MB/s 2023-09-12T15:57:30.246 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-09-12T15:57:30.669 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2023-09-12T15:57:30.974 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:57:30 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-0[121577]: 2023-09-12T15:57:30.753+0000 7f6337833700 -1 osd.0 0 waiting for initial osdmap 2023-09-12T15:57:30.975 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:57:30 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-0[121577]: 2023-09-12T15:57:30.755+0000 7f6331e25700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T15:57:30.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:30 smithi130 ceph-mon[98670]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-12T15:57:30.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:30 smithi130 ceph-mon[98670]: osdmap e6: 1 total, 0 up, 1 in 2023-09-12T15:57:30.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:30 smithi130 ceph-mon[98670]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T15:57:30.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:30 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:30.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:30 smithi159 ceph-mon[102522]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-12T15:57:30.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:30 smithi159 ceph-mon[102522]: osdmap e6: 1 total, 0 up, 1 in 2023-09-12T15:57:30.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:30 smithi159 ceph-mon[102522]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T15:57:30.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:30 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:31.226 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T15:57:31.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:30 smithi203 ceph-mon[103431]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-12T15:57:31.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:30 smithi203 ceph-mon[103431]: osdmap e6: 1 total, 0 up, 1 in 2023-09-12T15:57:31.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:30 smithi203 ceph-mon[103431]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T15:57:31.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:30 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:31.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:31.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T15:57:31.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: osdmap e7: 1 total, 0 up, 1 in 2023-09-12T15:57:31.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:31.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:31.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi130 2023-09-12T15:57:31.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:31.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:31.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:57:31.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi130 to 17168M 2023-09-12T15:57:31.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:31.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:31.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:31.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T15:57:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: osdmap e7: 1 total, 0 up, 1 in 2023-09-12T15:57:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi130 2023-09-12T15:57:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:57:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi130 to 17168M 2023-09-12T15:57:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:32.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-12T15:57:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T15:57:32.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: osdmap e7: 1 total, 0 up, 1 in 2023-09-12T15:57:32.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:32.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:32.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi130 2023-09-12T15:57:32.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:57:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi130 to 17168M 2023-09-12T15:57:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:32.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:32.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:32.419 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.416+0000 7f73b8160700 1 -- 172.21.15.159:0/4127283078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0100790 msgr2=0x7f73b0100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:32.420 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.416+0000 7f73b8160700 1 --2- 172.21.15.159:0/4127283078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0100790 0x7f73b0100bf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0099c0 tx=0x7f73ac009cd0 comp rx=0 tx=0).stop 2023-09-12T15:57:32.420 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.416+0000 7f73b8160700 1 -- 172.21.15.159:0/4127283078 shutdown_connections 2023-09-12T15:57:32.420 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.416+0000 7f73b8160700 1 --2- 172.21.15.159:0/4127283078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b0101130 0x7f73b01079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:32.420 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.416+0000 7f73b8160700 1 --2- 172.21.15.159:0/4127283078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73b00ff590 0x7f73b00ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:32.420 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.416+0000 7f73b8160700 1 --2- 172.21.15.159:0/4127283078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0100790 0x7f73b0100bf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:32.421 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.416+0000 7f73b8160700 1 -- 172.21.15.159:0/4127283078 >> 172.21.15.159:0/4127283078 conn(0x7f73b00fab10 msgr2=0x7f73b00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:32.421 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.416+0000 7f73b8160700 1 -- 172.21.15.159:0/4127283078 shutdown_connections 2023-09-12T15:57:32.421 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.416+0000 7f73b8160700 1 -- 172.21.15.159:0/4127283078 wait complete. 2023-09-12T15:57:32.421 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b8160700 1 Processor -- start 2023-09-12T15:57:32.421 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b8160700 1 -- start start 2023-09-12T15:57:32.422 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b8160700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73b00ff590 0x7f73b0195230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:32.422 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b8160700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b0100790 0x7f73b0195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:32.422 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b8160700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0101130 0x7f73b019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:32.422 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b8160700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b019cdc0 con 0x7f73b00ff590 2023-09-12T15:57:32.422 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b8160700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00680c0 con 0x7f73b0101130 2023-09-12T15:57:32.423 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b8160700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0068280 con 0x7f73b0100790 2023-09-12T15:57:32.423 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b66fd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0101130 0x7f73b019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:32.423 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b66fd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0101130 0x7f73b019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33050/0 (socket says 172.21.15.159:33050) 2023-09-12T15:57:32.423 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b66fd700 1 -- 172.21.15.159:0/1306139552 learned_addr learned my addr 172.21.15.159:0/1306139552 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-09-12T15:57:32.423 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b5efc700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73b00ff590 0x7f73b0195230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:32.424 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.417+0000 7f73b56fb700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b0100790 0x7f73b0195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:32.424 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73b66fd700 1 -- 172.21.15.159:0/1306139552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b0100790 msgr2=0x7f73b0195770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:32.424 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73b66fd700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b0100790 0x7f73b0195770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:32.424 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73b66fd700 1 -- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73b00ff590 msgr2=0x7f73b0195230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:32.424 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73b66fd700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73b00ff590 0x7f73b0195230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:32.424 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73b66fd700 1 -- 172.21.15.159:0/1306139552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac0095f0 con 0x7f73b0101130 2023-09-12T15:57:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73b66fd700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0101130 0x7f73b019c7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73a8007ee0 tx=0x7f73a800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73a6ffd700 1 -- 172.21.15.159:0/1306139552 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8017070 con 0x7f73b0101130 2023-09-12T15:57:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73a6ffd700 1 -- 172.21.15.159:0/1306139552 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a800f040 con 0x7f73b0101130 2023-09-12T15:57:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73a6ffd700 1 -- 172.21.15.159:0/1306139552 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8005580 con 0x7f73b0101130 2023-09-12T15:57:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b0068560 con 0x7f73b0101130 2023-09-12T15:57:32.426 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.418+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b0068a60 con 0x7f73b0101130 2023-09-12T15:57:32.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.419+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b0106b90 con 0x7f73b0101130 2023-09-12T15:57:32.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.420+0000 7f73a6ffd700 1 -- 172.21.15.159:0/1306139552 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) v1 ==== 135204+0+0 (secure 0 0 0) 0x7f73a800db20 con 0x7f73b0101130 2023-09-12T15:57:32.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.420+0000 7f73a6ffd700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f739c0a0630 0x7f739c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:32.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.421+0000 7f73a6ffd700 1 -- 172.21.15.159:0/1306139552 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f73a80c9e90 con 0x7f73b0101130 2023-09-12T15:57:32.428 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.421+0000 7f73b5efc700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f739c0a0630 0x7f739c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:32.428 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.421+0000 7f73b5efc700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f739c0a0630 0x7f739c0a2af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73b01005f0 tx=0x7f73a000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:32.428 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.424+0000 7f73a6ffd700 1 -- 172.21.15.159:0/1306139552 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f73a8095390 con 0x7f73b0101130 2023-09-12T15:57:32.625 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:32.624+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f73b0068ed0 con 0x7f739c0a0630 2023-09-12T15:57:33.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:32 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T15:57:33.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:32 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T15:57:33.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:32 smithi130 ceph-mon[98670]: osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] boot 2023-09-12T15:57:33.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:32 smithi130 ceph-mon[98670]: osdmap e8: 1 total, 1 up, 1 in 2023-09-12T15:57:33.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:32 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:33.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:32 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:57:33.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:32 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:57:33.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:32 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:33.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:32 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T15:57:33.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:32 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T15:57:33.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:32 smithi159 ceph-mon[102522]: osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] boot 2023-09-12T15:57:33.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:32 smithi159 ceph-mon[102522]: osdmap e8: 1 total, 1 up, 1 in 2023-09-12T15:57:33.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:32 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:33.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:32 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:57:33.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:32 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:57:33.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:32 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:33.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:32 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T15:57:33.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:32 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T15:57:33.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:32 smithi203 ceph-mon[103431]: osd.0 [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] boot 2023-09-12T15:57:33.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:32 smithi203 ceph-mon[103431]: osdmap e8: 1 total, 1 up, 1 in 2023-09-12T15:57:33.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:32 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-12T15:57:33.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:32 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:57:33.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:32 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:57:33.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:32 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:33.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:33 smithi159 ceph-mon[102522]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:33.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:33 smithi159 ceph-mon[102522]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:33.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:33 smithi159 ceph-mon[102522]: osdmap e9: 1 total, 1 up, 1 in 2023-09-12T15:57:34.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:33 smithi130 ceph-mon[98670]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:34.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:33 smithi130 ceph-mon[98670]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:34.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:33 smithi130 ceph-mon[98670]: osdmap e9: 1 total, 1 up, 1 in 2023-09-12T15:57:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:33 smithi203 ceph-mon[103431]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:33 smithi203 ceph-mon[103431]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:33 smithi203 ceph-mon[103431]: osdmap e9: 1 total, 1 up, 1 in 2023-09-12T15:57:36.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:35 smithi130 ceph-mon[98670]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:36.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:35 smithi130 ceph-mon[98670]: from='client.? 172.21.15.159:0/1685321518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fe17faa-52a3-4d63-878a-da07c02679b1"}]: dispatch 2023-09-12T15:57:36.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:35 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fe17faa-52a3-4d63-878a-da07c02679b1"}]: dispatch 2023-09-12T15:57:36.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:35 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fe17faa-52a3-4d63-878a-da07c02679b1"}]': finished 2023-09-12T15:57:36.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:35 smithi130 ceph-mon[98670]: osdmap e10: 2 total, 1 up, 2 in 2023-09-12T15:57:36.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:36.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:35 smithi130 ceph-mon[98670]: from='client.? 172.21.15.159:0/2903689279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:57:36.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:35 smithi159 ceph-mon[102522]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:36.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:35 smithi159 ceph-mon[102522]: from='client.? 172.21.15.159:0/1685321518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fe17faa-52a3-4d63-878a-da07c02679b1"}]: dispatch 2023-09-12T15:57:36.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:35 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fe17faa-52a3-4d63-878a-da07c02679b1"}]: dispatch 2023-09-12T15:57:36.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:35 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fe17faa-52a3-4d63-878a-da07c02679b1"}]': finished 2023-09-12T15:57:36.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:35 smithi159 ceph-mon[102522]: osdmap e10: 2 total, 1 up, 2 in 2023-09-12T15:57:36.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:36.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:35 smithi159 ceph-mon[102522]: from='client.? 172.21.15.159:0/2903689279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:57:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:35 smithi203 ceph-mon[103431]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:35 smithi203 ceph-mon[103431]: from='client.? 172.21.15.159:0/1685321518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fe17faa-52a3-4d63-878a-da07c02679b1"}]: dispatch 2023-09-12T15:57:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:35 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fe17faa-52a3-4d63-878a-da07c02679b1"}]: dispatch 2023-09-12T15:57:36.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:35 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fe17faa-52a3-4d63-878a-da07c02679b1"}]': finished 2023-09-12T15:57:36.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:35 smithi203 ceph-mon[103431]: osdmap e10: 2 total, 1 up, 2 in 2023-09-12T15:57:36.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:36.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:35 smithi203 ceph-mon[103431]: from='client.? 172.21.15.159:0/2903689279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:57:37.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:37 smithi130 ceph-mon[98670]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:37.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:37 smithi159 ceph-mon[102522]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:37.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:37 smithi203 ceph-mon[103431]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:40.009 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:39 smithi159 ceph-mon[102522]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:40.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:39 smithi130 ceph-mon[98670]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:39 smithi203 ceph-mon[103431]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:42.024 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:41 smithi159 ceph-mon[102522]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:42.025 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-12T15:57:42.025 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:42.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:41 smithi130 ceph-mon[98670]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:42.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-12T15:57:42.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:41 smithi203 ceph-mon[103431]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-12T15:57:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:42.860 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:42 smithi159 ceph-mon[102522]: Deploying daemon osd.1 on smithi159 2023-09-12T15:57:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:42 smithi130 ceph-mon[98670]: Deploying daemon osd.1 on smithi159 2023-09-12T15:57:43.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:42 smithi203 ceph-mon[103431]: Deploying daemon osd.1 on smithi159 2023-09-12T15:57:43.870 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:43 smithi159 ceph-mon[102522]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:44.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:43 smithi130 ceph-mon[98670]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:44.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:43 smithi203 ceph-mon[103431]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:46.121 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:45 smithi159 ceph-mon[102522]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:46.121 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:46.122 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:46.122 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:45 smithi130 ceph-mon[98670]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:45 smithi203 ceph-mon[103431]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:46.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:47.618 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:47 smithi159 ceph-mon[102522]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:47.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:47 smithi130 ceph-mon[98670]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:47.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:47 smithi203 ceph-mon[103431]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:48.483 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.483+0000 7f73a6ffd700 1 -- 172.21.15.159:0/1306139552 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f73b0068ed0 con 0x7f739c0a0630 2023-09-12T15:57:48.484 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 1 on host 'smithi159' 2023-09-12T15:57:48.485 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f739c0a0630 msgr2=0x7f739c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:48.485 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f739c0a0630 0x7f739c0a2af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73b01005f0 tx=0x7f73a000b410 comp rx=0 tx=0).stop 2023-09-12T15:57:48.485 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0101130 msgr2=0x7f73b019c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:48.486 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0101130 0x7f73b019c7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73a8007ee0 tx=0x7f73a800d3b0 comp rx=0 tx=0).stop 2023-09-12T15:57:48.486 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 shutdown_connections 2023-09-12T15:57:48.486 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f739c0a0630 0x7f739c0a2af0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:48.486 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b0101130 0x7f73b019c7f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:48.486 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73b00ff590 0x7f73b0195230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:48.487 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 --2- 172.21.15.159:0/1306139552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b0100790 0x7f73b0195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:48.487 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 >> 172.21.15.159:0/1306139552 conn(0x7f73b00fab10 msgr2=0x7f73b0103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:48.487 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 shutdown_connections 2023-09-12T15:57:48.487 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:57:48.485+0000 7f73b8160700 1 -- 172.21.15.159:0/1306139552 wait complete. 2023-09-12T15:57:48.927 DEBUG:teuthology.orchestra.run.smithi159:osd.1> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.1.service 2023-09-12T15:57:48.930 INFO:tasks.cephadm:Deploying osd.2 on smithi203 with /dev/vg_nvme/lv_4... 2023-09-12T15:57:48.930 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-12T15:57:48.959 INFO:journalctl@ceph.osd.1.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T15:57:49.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:49.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:49.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:49.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:49.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:49.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:49.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:49.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:49.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:49.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:57:49.494 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T15:57:49.915 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:57:49 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-1[107552]: 2023-09-12T15:57:49.628+0000 7fba0f53e640 -1 osd.1 0 log_to_monitors true 2023-09-12T15:57:49.916 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:49 smithi159 ceph-mon[102522]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:49.916 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:49 smithi159 ceph-mon[102522]: from='osd.1 [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-12T15:57:49.916 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:49 smithi159 ceph-mon[102522]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-12T15:57:49.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:49 smithi203 ceph-mon[103431]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:49.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:49 smithi203 ceph-mon[103431]: from='osd.1 [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-12T15:57:49.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:49 smithi203 ceph-mon[103431]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-12T15:57:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:49 smithi130 ceph-mon[98670]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:49 smithi130 ceph-mon[98670]: from='osd.1 [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-12T15:57:50.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:49 smithi130 ceph-mon[98670]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-12T15:57:51.138 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-12T15:57:51.138 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-12T15:57:51.139 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2023-09-12T15:57:51.139 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2023-09-12T15:57:51.139 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127311 s, 824 MB/s 2023-09-12T15:57:51.139 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2023-09-12T15:57:51.546 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi203:vg_nvme/lv_4 2023-09-12T15:57:51.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:51 smithi130 ceph-mon[98670]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-12T15:57:51.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:51 smithi130 ceph-mon[98670]: from='osd.1 [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:57:51.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:51 smithi130 ceph-mon[98670]: osdmap e11: 2 total, 1 up, 2 in 2023-09-12T15:57:51.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:51.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:51 smithi130 ceph-mon[98670]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:57:51.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:51 smithi130 ceph-mon[98670]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:51.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:51.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:51.727 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-1[107552]: 2023-09-12T15:57:51.452+0000 7fba04bb5700 -1 osd.1 0 waiting for initial osdmap 2023-09-12T15:57:51.727 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-1[107552]: 2023-09-12T15:57:51.455+0000 7fb9ff1a7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T15:57:51.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-mon[102522]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-12T15:57:51.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-mon[102522]: from='osd.1 [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:57:51.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-mon[102522]: osdmap e11: 2 total, 1 up, 2 in 2023-09-12T15:57:51.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:51.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-mon[102522]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:57:51.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-mon[102522]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:51.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:51.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:51.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:51 smithi203 ceph-mon[103431]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-12T15:57:51.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:51 smithi203 ceph-mon[103431]: from='osd.1 [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:57:51.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:51 smithi203 ceph-mon[103431]: osdmap e11: 2 total, 1 up, 2 in 2023-09-12T15:57:51.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:51.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:51 smithi203 ceph-mon[103431]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:57:51.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:51 smithi203 ceph-mon[103431]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:51.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:51.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:52.102 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T15:57:52.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi159 2023-09-12T15:57:52.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:57:52.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi159 to 17168M 2023-09-12T15:57:52.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T15:57:52.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:52.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: osdmap e12: 2 total, 1 up, 2 in 2023-09-12T15:57:52.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:52.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:52.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:52.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:52.728 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi159 2023-09-12T15:57:52.728 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:57:52.728 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi159 to 17168M 2023-09-12T15:57:52.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T15:57:52.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:52.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: osdmap e12: 2 total, 1 up, 2 in 2023-09-12T15:57:52.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:52.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:52.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:52.730 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:52.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi159 2023-09-12T15:57:52.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:57:52.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi159 to 17168M 2023-09-12T15:57:52.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T15:57:52.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:52.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: osdmap e12: 2 total, 1 up, 2 in 2023-09-12T15:57:52.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:52.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:52.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:57:52.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:57:53.442 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.441+0000 7f81bbafb700 1 -- 172.21.15.203:0/1533186143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81b4100790 msgr2=0x7f81b4100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:53.443 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.441+0000 7f81bbafb700 1 --2- 172.21.15.203:0/1533186143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81b4100790 0x7f81b4100bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f81b00099c0 tx=0x7f81b0009cd0 comp rx=0 tx=0).stop 2023-09-12T15:57:53.443 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.441+0000 7f81bbafb700 1 -- 172.21.15.203:0/1533186143 shutdown_connections 2023-09-12T15:57:53.443 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.441+0000 7f81bbafb700 1 --2- 172.21.15.203:0/1533186143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b4101130 0x7f81b41079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:53.443 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.441+0000 7f81bbafb700 1 --2- 172.21.15.203:0/1533186143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81b40ff590 0x7f81b40ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:53.444 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.441+0000 7f81bbafb700 1 --2- 172.21.15.203:0/1533186143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81b4100790 0x7f81b4100bf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:53.444 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.441+0000 7f81bbafb700 1 -- 172.21.15.203:0/1533186143 >> 172.21.15.203:0/1533186143 conn(0x7f81b40fab10 msgr2=0x7f81b40fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:57:53.444 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.441+0000 7f81bbafb700 1 -- 172.21.15.203:0/1533186143 shutdown_connections 2023-09-12T15:57:53.444 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.441+0000 7f81bbafb700 1 -- 172.21.15.203:0/1533186143 wait complete. 2023-09-12T15:57:53.444 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81bbafb700 1 Processor -- start 2023-09-12T15:57:53.444 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81bbafb700 1 -- start start 2023-09-12T15:57:53.445 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81bbafb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b40ff590 0x7f81b4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:53.445 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81bbafb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81b4100790 0x7f81b4196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:53.445 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81bbafb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81b4101130 0x7f81b419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:53.445 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81bbafb700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b419ded0 con 0x7f81b4100790 2023-09-12T15:57:53.445 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81bbafb700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b419e010 con 0x7f81b4101130 2023-09-12T15:57:53.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81bbafb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b419e1a0 con 0x7f81b40ff590 2023-09-12T15:57:53.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81b9897700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b40ff590 0x7f81b4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:53.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81b9897700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b40ff590 0x7f81b4196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:41408/0 (socket says 172.21.15.203:41408) 2023-09-12T15:57:53.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81b9897700 1 -- 172.21.15.203:0/118679919 learned_addr learned my addr 172.21.15.203:0/118679919 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-09-12T15:57:53.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81b9096700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81b4100790 0x7f81b4196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:53.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81ba098700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81b4101130 0x7f81b419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:53.447 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81b9897700 1 -- 172.21.15.203:0/118679919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81b4101130 msgr2=0x7f81b419d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:53.447 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81b9897700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81b4101130 0x7f81b419d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:53.447 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81b9897700 1 -- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81b4100790 msgr2=0x7f81b4196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:57:53.447 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81b9897700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81b4100790 0x7f81b4196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:57:53.447 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81b9897700 1 -- 172.21.15.203:0/118679919 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b00095f0 con 0x7f81b40ff590 2023-09-12T15:57:53.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.442+0000 7f81ba098700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81b4101130 0x7f81b419d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T15:57:53.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.443+0000 7f81b9897700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b40ff590 0x7f81b4196340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e9a0 tx=0x7f81a400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:53.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.443+0000 7f81aaffd700 1 -- 172.21.15.203:0/118679919 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4009790 con 0x7f81b40ff590 2023-09-12T15:57:53.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.443+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b419e480 con 0x7f81b40ff590 2023-09-12T15:57:53.449 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.443+0000 7f81aaffd700 1 -- 172.21.15.203:0/118679919 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81a40044e0 con 0x7f81b40ff590 2023-09-12T15:57:53.449 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.443+0000 7f81aaffd700 1 -- 172.21.15.203:0/118679919 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4003d70 con 0x7f81b40ff590 2023-09-12T15:57:53.450 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.443+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b419e9d0 con 0x7f81b40ff590 2023-09-12T15:57:53.450 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.444+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b410ada0 con 0x7f81b40ff590 2023-09-12T15:57:53.450 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.445+0000 7f81aaffd700 1 -- 172.21.15.203:0/118679919 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 15) v1 ==== 135204+0+0 (secure 0 0 0) 0x7f81a4003f10 con 0x7f81b40ff590 2023-09-12T15:57:53.450 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.445+0000 7f81aaffd700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f81a00a05b0 0x7f81a00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:57:53.450 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.446+0000 7f81aaffd700 1 -- 172.21.15.203:0/118679919 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f81a4014070 con 0x7f81b40ff590 2023-09-12T15:57:53.451 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.446+0000 7f81b9096700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f81a00a05b0 0x7f81a00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:57:53.451 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.446+0000 7f81b9096700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f81a00a05b0 0x7f81a00a2a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f81b000b580 tx=0x7f81b001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:57:53.451 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.450+0000 7f81aaffd700 1 -- 172.21.15.203:0/118679919 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f81a4095990 con 0x7f81b40ff590 2023-09-12T15:57:53.644 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:57:53.644+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f81b419ec50 con 0x7f81a00a05b0 2023-09-12T15:57:53.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:53 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T15:57:53.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:53 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T15:57:53.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:53.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:53 smithi130 ceph-mon[98670]: osd.1 [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] boot 2023-09-12T15:57:53.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:53 smithi130 ceph-mon[98670]: osdmap e13: 2 total, 2 up, 2 in 2023-09-12T15:57:53.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:53.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:53 smithi130 ceph-mon[98670]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:53.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:53 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T15:57:53.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:53 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T15:57:53.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:53.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:53 smithi159 ceph-mon[102522]: osd.1 [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] boot 2023-09-12T15:57:53.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:53 smithi159 ceph-mon[102522]: osdmap e13: 2 total, 2 up, 2 in 2023-09-12T15:57:53.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:53.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:53 smithi159 ceph-mon[102522]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:53.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:53 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T15:57:53.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:53 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T15:57:53.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:53.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:53 smithi203 ceph-mon[103431]: osd.1 [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] boot 2023-09-12T15:57:53.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:53 smithi203 ceph-mon[103431]: osdmap e13: 2 total, 2 up, 2 in 2023-09-12T15:57:53.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-12T15:57:53.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:53 smithi203 ceph-mon[103431]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-12T15:57:54.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:54 smithi130 ceph-mon[98670]: osdmap e14: 2 total, 2 up, 2 in 2023-09-12T15:57:54.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:54 smithi130 ceph-mon[98670]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:54.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:57:54.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:57:54.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:54.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:54 smithi159 ceph-mon[102522]: osdmap e14: 2 total, 2 up, 2 in 2023-09-12T15:57:54.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:54 smithi159 ceph-mon[102522]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:54.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:57:54.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:57:54.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:54.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:54 smithi203 ceph-mon[103431]: osdmap e14: 2 total, 2 up, 2 in 2023-09-12T15:57:54.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:54 smithi203 ceph-mon[103431]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:57:54.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:57:54.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:57:54.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:57:55.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:55 smithi130 ceph-mon[98670]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:57:55.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:55 smithi159 ceph-mon[102522]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:57:55.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:55 smithi203 ceph-mon[103431]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:57:56.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:56 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/2311761122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c101604-31dd-462a-88ee-7d55829e0d2d"}]: dispatch 2023-09-12T15:57:56.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:56 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c101604-31dd-462a-88ee-7d55829e0d2d"}]: dispatch 2023-09-12T15:57:56.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:56 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c101604-31dd-462a-88ee-7d55829e0d2d"}]': finished 2023-09-12T15:57:56.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:56 smithi130 ceph-mon[98670]: osdmap e15: 3 total, 2 up, 3 in 2023-09-12T15:57:56.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:56 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:57:56.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:56 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/2311761122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c101604-31dd-462a-88ee-7d55829e0d2d"}]: dispatch 2023-09-12T15:57:56.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:56 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c101604-31dd-462a-88ee-7d55829e0d2d"}]: dispatch 2023-09-12T15:57:56.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:56 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c101604-31dd-462a-88ee-7d55829e0d2d"}]': finished 2023-09-12T15:57:56.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:56 smithi159 ceph-mon[102522]: osdmap e15: 3 total, 2 up, 3 in 2023-09-12T15:57:56.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:56 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:57:56.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:56 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/2311761122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c101604-31dd-462a-88ee-7d55829e0d2d"}]: dispatch 2023-09-12T15:57:56.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:56 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c101604-31dd-462a-88ee-7d55829e0d2d"}]: dispatch 2023-09-12T15:57:56.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:56 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c101604-31dd-462a-88ee-7d55829e0d2d"}]': finished 2023-09-12T15:57:56.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:56 smithi203 ceph-mon[103431]: osdmap e15: 3 total, 2 up, 3 in 2023-09-12T15:57:56.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:56 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:57:57.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:57 smithi130 ceph-mon[98670]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:57:57.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:57 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/2209063485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:57:57.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:57 smithi159 ceph-mon[102522]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:57:57.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:57 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/2209063485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:57:57.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:57 smithi203 ceph-mon[103431]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:57:57.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:57 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/2209063485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:58:00.085 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:57:59 smithi203 ceph-mon[103431]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:00.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:57:59 smithi130 ceph-mon[98670]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:00.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:57:59 smithi159 ceph-mon[102522]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:01.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:01 smithi203 ceph-mon[103431]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:02.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:01 smithi130 ceph-mon[98670]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:02.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:01 smithi159 ceph-mon[102522]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:04.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:03 smithi203 ceph-mon[103431]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:04.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:03 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-12T15:58:04.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:03 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:04.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:03 smithi203 ceph-mon[103431]: Deploying daemon osd.2 on smithi203 2023-09-12T15:58:04.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:03 smithi130 ceph-mon[98670]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:04.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:03 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-12T15:58:04.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:03 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:04.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:03 smithi130 ceph-mon[98670]: Deploying daemon osd.2 on smithi203 2023-09-12T15:58:04.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:03 smithi159 ceph-mon[102522]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:04.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:03 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-12T15:58:04.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:03 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:04.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:03 smithi159 ceph-mon[102522]: Deploying daemon osd.2 on smithi203 2023-09-12T15:58:06.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:05 smithi130 ceph-mon[98670]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:06.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:05 smithi159 ceph-mon[102522]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:06.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:05 smithi203 ceph-mon[103431]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:07.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:07 smithi203 ceph-mon[103431]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:07.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:07.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:07.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:07.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:07 smithi130 ceph-mon[98670]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:07.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:07.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:07.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:07.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:07 smithi159 ceph-mon[102522]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:07.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:07.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:07.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:10.197 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:09 smithi203 ceph-mon[103431]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:10.197 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:10.197 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:10.198 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:10.198 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:10.198 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:10.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:09 smithi130 ceph-mon[98670]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:10.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:09 smithi159 ceph-mon[102522]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:10.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:10.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:10.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:10.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:10.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:10.253 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.253+0000 7f81aaffd700 1 -- 172.21.15.203:0/118679919 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f81b419ec50 con 0x7f81a00a05b0 2023-09-12T15:58:10.253 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 2 on host 'smithi203' 2023-09-12T15:58:10.256 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f81a00a05b0 msgr2=0x7f81a00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:10.256 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f81a00a05b0 0x7f81a00a2a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f81b000b580 tx=0x7f81b001a040 comp rx=0 tx=0).stop 2023-09-12T15:58:10.256 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b40ff590 msgr2=0x7f81b4196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:10.256 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b40ff590 0x7f81b4196340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e9a0 tx=0x7f81a400ed60 comp rx=0 tx=0).stop 2023-09-12T15:58:10.256 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 shutdown_connections 2023-09-12T15:58:10.257 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f81a00a05b0 0x7f81a00a2a70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:10.257 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81b4101130 0x7f81b419d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:10.257 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b40ff590 0x7f81b4196340 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:10.257 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 --2- 172.21.15.203:0/118679919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81b4100790 0x7f81b4196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:10.257 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 >> 172.21.15.203:0/118679919 conn(0x7f81b40fab10 msgr2=0x7f81b4068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:58:10.258 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 shutdown_connections 2023-09-12T15:58:10.258 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:10.255+0000 7f81bbafb700 1 -- 172.21.15.203:0/118679919 wait complete. 2023-09-12T15:58:10.696 DEBUG:teuthology.orchestra.run.smithi203:osd.2> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.2.service 2023-09-12T15:58:10.698 INFO:tasks.cephadm:Deploying osd.3 on smithi130 with /dev/vg_nvme/lv_3... 2023-09-12T15:58:10.699 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-12T15:58:10.729 INFO:journalctl@ceph.osd.2.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T15:58:11.266 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:58:11.442 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-2[109286]: 2023-09-12T15:58:11.332+0000 7facda539640 -1 osd.2 0 log_to_monitors true 2023-09-12T15:58:11.442 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:11.442 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:11.442 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:11.443 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:11 smithi203 ceph-mon[103431]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:11.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:11.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:11.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:11.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:11 smithi130 ceph-mon[98670]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:11.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:11.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:11.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:11.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:11 smithi159 ceph-mon[102522]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:12.263 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:12 smithi130 ceph-mon[98670]: from='osd.2 [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-12T15:58:12.263 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:12 smithi130 ceph-mon[98670]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-12T15:58:12.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:12 smithi203 ceph-mon[103431]: from='osd.2 [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-12T15:58:12.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:12 smithi203 ceph-mon[103431]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-12T15:58:12.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:12 smithi159 ceph-mon[102522]: from='osd.2 [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-12T15:58:12.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:12 smithi159 ceph-mon[102522]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-12T15:58:12.739 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-12T15:58:12.739 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-12T15:58:12.739 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2023-09-12T15:58:12.739 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2023-09-12T15:58:12.740 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110634 s, 948 MB/s 2023-09-12T15:58:12.740 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2023-09-12T15:58:13.106 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi130:vg_nvme/lv_3 2023-09-12T15:58:13.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:13 smithi130 ceph-mon[98670]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-12T15:58:13.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:13 smithi130 ceph-mon[98670]: osdmap e16: 3 total, 2 up, 3 in 2023-09-12T15:58:13.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:13.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:13 smithi130 ceph-mon[98670]: from='osd.2 [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:58:13.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:13 smithi130 ceph-mon[98670]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:58:13.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:13 smithi130 ceph-mon[98670]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:13.495 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-2[109286]: 2023-09-12T15:58:13.246+0000 7faccfbb0700 -1 osd.2 0 waiting for initial osdmap 2023-09-12T15:58:13.495 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-2[109286]: 2023-09-12T15:58:13.249+0000 7facc91a0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T15:58:13.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:13 smithi203 ceph-mon[103431]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-12T15:58:13.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:13 smithi203 ceph-mon[103431]: osdmap e16: 3 total, 2 up, 3 in 2023-09-12T15:58:13.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:13.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:13 smithi203 ceph-mon[103431]: from='osd.2 [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:58:13.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:13 smithi203 ceph-mon[103431]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:58:13.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:13 smithi203 ceph-mon[103431]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:13.665 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:58:13.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:13 smithi159 ceph-mon[102522]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-12T15:58:13.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:13 smithi159 ceph-mon[102522]: osdmap e16: 3 total, 2 up, 3 in 2023-09-12T15:58:13.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:13.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:13 smithi159 ceph-mon[102522]: from='osd.2 [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:58:13.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:13 smithi159 ceph-mon[102522]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:58:13.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:13 smithi159 ceph-mon[102522]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:14.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T15:58:14.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: osdmap e17: 3 total, 2 up, 3 in 2023-09-12T15:58:14.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:14.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:14.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi203 2023-09-12T15:58:14.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:14.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi203 to 17168M 2023-09-12T15:58:14.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:14.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:14.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T15:58:14.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: osdmap e17: 3 total, 2 up, 3 in 2023-09-12T15:58:14.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:14.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:14.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi203 2023-09-12T15:58:14.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:14.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi203 to 17168M 2023-09-12T15:58:14.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:14.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:14.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.734 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.735+0000 7f09fbb86700 1 -- 172.21.15.130:0/1283605314 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f41008b0 msgr2=0x7f09f4107160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:14.735 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.735+0000 7f09fbb86700 1 --2- 172.21.15.130:0/1283605314 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f41008b0 0x7f09f4107160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00b0a0 tx=0x7f09ec00b3b0 comp rx=0 tx=0).stop 2023-09-12T15:58:14.735 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.735+0000 7f09fbb86700 1 -- 172.21.15.130:0/1283605314 shutdown_connections 2023-09-12T15:58:14.735 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.735+0000 7f09fbb86700 1 --2- 172.21.15.130:0/1283605314 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f41008b0 0x7f09f4107160 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.735+0000 7f09fbb86700 1 --2- 172.21.15.130:0/1283605314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fed10 0x7f09f40ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.735+0000 7f09fbb86700 1 --2- 172.21.15.130:0/1283605314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09f40fff10 0x7f09f4100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.735+0000 7f09fbb86700 1 -- 172.21.15.130:0/1283605314 >> 172.21.15.130:0/1283605314 conn(0x7f09f40fa2b0 msgr2=0x7f09f40fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:58:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.735+0000 7f09fbb86700 1 -- 172.21.15.130:0/1283605314 shutdown_connections 2023-09-12T15:58:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.735+0000 7f09fbb86700 1 -- 172.21.15.130:0/1283605314 wait complete. 2023-09-12T15:58:14.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09fbb86700 1 Processor -- start 2023-09-12T15:58:14.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09fbb86700 1 -- start start 2023-09-12T15:58:14.740 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09fbb86700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fed10 0x7f09f4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:14.740 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09fbb86700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f40fff10 0x7f09f4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:14.740 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09fbb86700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09f41008b0 0x7f09f419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:14.740 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09fbb86700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f41a01b0 con 0x7f09f40fff10 2023-09-12T15:58:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09fbb86700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f41a02f0 con 0x7f09f41008b0 2023-09-12T15:58:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09fbb86700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f41a0480 con 0x7f09f40fed10 2023-09-12T15:58:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9121700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f40fff10 0x7f09f4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9121700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f40fff10 0x7f09f4198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52464/0 (socket says 172.21.15.130:52464) 2023-09-12T15:58:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9121700 1 -- 172.21.15.130:0/692767191 learned_addr learned my addr 172.21.15.130:0/692767191 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:58:14.742 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9922700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fed10 0x7f09f4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:14.742 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09fa123700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09f41008b0 0x7f09f419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:14.742 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9121700 1 -- 172.21.15.130:0/692767191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fed10 msgr2=0x7f09f4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:14.742 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9121700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fed10 0x7f09f4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:14.742 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9121700 1 -- 172.21.15.130:0/692767191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09f41008b0 msgr2=0x7f09f419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:14.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9121700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09f41008b0 0x7f09f419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:14.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9121700 1 -- 172.21.15.130:0/692767191 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec009d00 con 0x7f09f40fff10 2023-09-12T15:58:14.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09f9121700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f40fff10 0x7f09f4198b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09e400d9b0 tx=0x7f09e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:58:14.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.736+0000 7f09eaffd700 1 -- 172.21.15.130:0/692767191 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4004220 con 0x7f09f40fff10 2023-09-12T15:58:14.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.737+0000 7f09eaffd700 1 -- 172.21.15.130:0/692767191 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09e4007640 con 0x7f09f40fff10 2023-09-12T15:58:14.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.737+0000 7f09eaffd700 1 -- 172.21.15.130:0/692767191 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4005650 con 0x7f09f40fff10 2023-09-12T15:58:14.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.737+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f41a0760 con 0x7f09f40fff10 2023-09-12T15:58:14.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.737+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f41a0c60 con 0x7f09f40fff10 2023-09-12T15:58:14.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.738+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f410adb0 con 0x7f09f40fff10 2023-09-12T15:58:14.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.739+0000 7f09eaffd700 1 -- 172.21.15.130:0/692767191 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 15) v1 ==== 135204+0+0 (secure 0 0 0) 0x7f09e4004750 con 0x7f09f40fff10 2023-09-12T15:58:14.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.739+0000 7f09eaffd700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e00a0630 0x7f09e00a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:14.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.739+0000 7f09f9922700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e00a0630 0x7f09e00a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:14.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.739+0000 7f09eaffd700 1 -- 172.21.15.130:0/692767191 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f09e40ca310 con 0x7f09f40fff10 2023-09-12T15:58:14.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.740+0000 7f09f9922700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e00a0630 0x7f09e00a2af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f09f40ffd70 tx=0x7f09f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:58:14.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T15:58:14.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: osdmap e17: 3 total, 2 up, 3 in 2023-09-12T15:58:14.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:14.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:14.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi203 2023-09-12T15:58:14.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:14.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi203 to 17168M 2023-09-12T15:58:14.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:14.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:14.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:14.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.744+0000 7f09eaffd700 1 -- 172.21.15.130:0/692767191 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f09e4095470 con 0x7f09f40fff10 2023-09-12T15:58:14.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:14.948+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f09f4108050 con 0x7f09e00a0630 2023-09-12T15:58:15.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T15:58:15.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T15:58:15.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: osd.2 [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] boot 2023-09-12T15:58:15.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: osdmap e18: 3 total, 3 up, 3 in 2023-09-12T15:58:15.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:15.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:15.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-12T15:58:15.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:58:15.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:58:15.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:58:15.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:15.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T15:58:15.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T15:58:15.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: osd.2 [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] boot 2023-09-12T15:58:15.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: osdmap e18: 3 total, 3 up, 3 in 2023-09-12T15:58:15.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:15.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:15.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-12T15:58:15.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:58:15.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:58:15.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:58:15.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:15 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:15.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T15:58:15.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T15:58:15.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: osd.2 [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] boot 2023-09-12T15:58:15.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: osdmap e18: 3 total, 3 up, 3 in 2023-09-12T15:58:15.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-12T15:58:15.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-12T15:58:15.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-12T15:58:15.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:58:15.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:58:15.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:58:15.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:15 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:16.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-12T15:58:16.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:16 smithi130 ceph-mon[98670]: osdmap e19: 3 total, 3 up, 3 in 2023-09-12T15:58:16.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-12T15:58:16.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-12T15:58:16.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:16 smithi159 ceph-mon[102522]: osdmap e19: 3 total, 3 up, 3 in 2023-09-12T15:58:16.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-12T15:58:16.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-12T15:58:16.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:16 smithi203 ceph-mon[103431]: osdmap e19: 3 total, 3 up, 3 in 2023-09-12T15:58:16.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-12T15:58:16.886 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:58:16 smithi130 sudo[127346]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-12T15:58:16.886 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:58:16 smithi130 sudo[127346]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:16.887 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:58:16 smithi130 sudo[127346]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:16.887 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:58:16 smithi130 sudo[127346]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:17.224 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:58:16 smithi130 sudo[127379]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-12T15:58:17.225 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:58:16 smithi130 sudo[127379]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:17.225 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:58:16 smithi130 sudo[127379]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:17.225 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 15:58:16 smithi130 sudo[127379]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:17.409 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-12T15:58:17.410 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:17 smithi159 ceph-mon[102522]: osdmap e20: 3 total, 3 up, 3 in 2023-09-12T15:58:17.410 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:17 smithi159 ceph-mon[102522]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:17.410 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:17.411 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:58:17 smithi159 sudo[112253]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-12T15:58:17.411 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:58:17 smithi159 sudo[112253]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:17.411 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:58:17 smithi159 sudo[112253]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:17.411 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:58:17 smithi159 sudo[112253]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:17.681 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-12T15:58:17.681 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:17 smithi203 ceph-mon[103431]: osdmap e20: 3 total, 3 up, 3 in 2023-09-12T15:58:17.681 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:17 smithi203 ceph-mon[103431]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:17.682 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-12T15:58:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:17 smithi130 ceph-mon[98670]: osdmap e20: 3 total, 3 up, 3 in 2023-09-12T15:58:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:17 smithi130 ceph-mon[98670]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:17.726 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:58:17 smithi159 sudo[112256]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-12T15:58:17.726 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:58:17 smithi159 sudo[112256]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:17.726 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:58:17 smithi159 sudo[112256]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:17.726 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 15:58:17 smithi159 sudo[112256]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:17.932 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:17 smithi203 sudo[114563]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-12T15:58:17.933 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:17 smithi203 sudo[114563]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:17.933 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:17 smithi203 sudo[114563]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:17.933 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:17 smithi203 sudo[114563]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:18.245 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:17 smithi203 sudo[114566]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-12T15:58:18.246 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:17 smithi203 sudo[114566]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:18.246 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:17 smithi203 sudo[114566]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:18.246 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 15:58:17 smithi203 sudo[114566]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:18.456 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 sudo[127450]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-12T15:58:18.456 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 sudo[127450]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:18.456 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 sudo[127450]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:18.457 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 sudo[127450]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:18.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 sudo[127453]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-12T15:58:18.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 sudo[127453]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:18.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 sudo[127453]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:18.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 sudo[127453]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:18.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 ceph-mon[98670]: osdmap e21: 3 total, 3 up, 3 in 2023-09-12T15:58:18.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3886820598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7045b414-e695-4f08-9764-72c802812750"}]: dispatch 2023-09-12T15:58:18.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3886820598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7045b414-e695-4f08-9764-72c802812750"}]': finished 2023-09-12T15:58:18.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 ceph-mon[98670]: osdmap e22: 4 total, 3 up, 4 in 2023-09-12T15:58:18.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:18.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:18.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:18.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 ceph-mon[98670]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-12T15:58:18.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:18 smithi130 ceph-mon[98670]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-12T15:58:18.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:18 smithi159 ceph-mon[102522]: osdmap e21: 3 total, 3 up, 3 in 2023-09-12T15:58:18.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:18 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3886820598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7045b414-e695-4f08-9764-72c802812750"}]: dispatch 2023-09-12T15:58:18.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:18 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3886820598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7045b414-e695-4f08-9764-72c802812750"}]': finished 2023-09-12T15:58:18.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:18 smithi159 ceph-mon[102522]: osdmap e22: 4 total, 3 up, 4 in 2023-09-12T15:58:18.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:18.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:18.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:18.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:18 smithi159 ceph-mon[102522]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-12T15:58:18.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:18 smithi159 ceph-mon[102522]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-12T15:58:18.949 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 sudo[114569]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-12T15:58:18.950 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 sudo[114569]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:18.950 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 sudo[114569]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:18.950 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 sudo[114569]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:18.951 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 sudo[114572]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-12T15:58:18.951 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 sudo[114572]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:18.951 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 sudo[114572]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:18.951 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 sudo[114572]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:18.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:18.961+0000 7f09eaffd700 1 -- 172.21.15.130:0/692767191 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f09e4095840 con 0x7f09f40fff10 2023-09-12T15:58:19.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 ceph-mon[103431]: osdmap e21: 3 total, 3 up, 3 in 2023-09-12T15:58:19.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3886820598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7045b414-e695-4f08-9764-72c802812750"}]: dispatch 2023-09-12T15:58:19.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3886820598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7045b414-e695-4f08-9764-72c802812750"}]': finished 2023-09-12T15:58:19.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 ceph-mon[103431]: osdmap e22: 4 total, 3 up, 4 in 2023-09-12T15:58:19.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:19.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:19.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:19.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 ceph-mon[103431]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-12T15:58:19.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:18 smithi203 ceph-mon[103431]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-12T15:58:19.442 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 sudo[112259]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-12T15:58:19.442 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 sudo[112259]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:19.442 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 sudo[112259]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:19.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 sudo[112259]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:19.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:58:19.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4129775593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:58:19.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:58:19.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:58:19.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-12T15:58:19.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:58:19.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:58:19.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:58:19.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:19.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-12T15:58:19.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-12T15:58:19.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:58:19.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:58:19.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:58:19.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-09-12T15:58:19.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:19 smithi130 ceph-mon[98670]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-12T15:58:19.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 sudo[112262]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-12T15:58:19.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 sudo[112262]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-12T15:58:19.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 sudo[112262]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-12T15:58:19.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 sudo[112262]: pam_unix(sudo:session): session closed for user root 2023-09-12T15:58:19.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:58:19.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4129775593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:58:19.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:58:19.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:58:19.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-12T15:58:19.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:58:19.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:58:19.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:58:19.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:19.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-12T15:58:19.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-12T15:58:19.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:58:19.733 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:58:19.733 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:58:19.733 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-09-12T15:58:19.733 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:19 smithi159 ceph-mon[102522]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-12T15:58:19.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:58:19.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4129775593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:58:19.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:58:19.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:58:19.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-12T15:58:19.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:58:19.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:58:19.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:58:19.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:19.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-12T15:58:19.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-12T15:58:19.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-12T15:58:19.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-12T15:58:19.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-12T15:58:19.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-09-12T15:58:19.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:19 smithi203 ceph-mon[103431]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-12T15:58:21.845 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:21 smithi130 ceph-mon[98670]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:22.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:21 smithi159 ceph-mon[102522]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:22.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:21 smithi203 ceph-mon[103431]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:24.187 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:23 smithi130 ceph-mon[98670]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:24.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:23 smithi159 ceph-mon[102522]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:24.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:23 smithi203 ceph-mon[103431]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-12T15:58:25.137 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:24 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-12T15:58:25.137 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:24 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:25.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:24 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-12T15:58:25.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:24 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:25.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:24 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-12T15:58:25.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:24 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:25.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:25 smithi130 ceph-mon[98670]: Deploying daemon osd.3 on smithi130 2023-09-12T15:58:25.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:25 smithi130 ceph-mon[98670]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-09-12T15:58:26.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:25 smithi159 ceph-mon[102522]: Deploying daemon osd.3 on smithi130 2023-09-12T15:58:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:25 smithi159 ceph-mon[102522]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-09-12T15:58:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:25 smithi203 ceph-mon[103431]: Deploying daemon osd.3 on smithi130 2023-09-12T15:58:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:25 smithi203 ceph-mon[103431]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-09-12T15:58:27.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:27 smithi130 ceph-mon[98670]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-09-12T15:58:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:27 smithi159 ceph-mon[102522]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-09-12T15:58:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:27 smithi203 ceph-mon[103431]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-09-12T15:58:29.678 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:29 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:29.679 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:29 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:29.679 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:29 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:29.679 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:29 smithi130 ceph-mon[98670]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-12T15:58:29.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:29 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:29.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:29 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:29.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:29 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:29.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:29 smithi159 ceph-mon[102522]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-12T15:58:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:29 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:29 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:29 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:29 smithi203 ceph-mon[103431]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-12T15:58:31.110 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 3 on host 'smithi130' 2023-09-12T15:58:31.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.109+0000 7f09eaffd700 1 -- 172.21.15.130:0/692767191 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f09f4108050 con 0x7f09e00a0630 2023-09-12T15:58:31.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e00a0630 msgr2=0x7f09e00a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:31.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e00a0630 0x7f09e00a2af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f09f40ffd70 tx=0x7f09f0009260 comp rx=0 tx=0).stop 2023-09-12T15:58:31.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f40fff10 msgr2=0x7f09f4198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:31.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f40fff10 0x7f09f4198b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09e400d9b0 tx=0x7f09e400dcc0 comp rx=0 tx=0).stop 2023-09-12T15:58:31.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 shutdown_connections 2023-09-12T15:58:31.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e00a0630 0x7f09e00a2af0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:31.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09f41008b0 0x7f09f419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:31.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f40fed10 0x7f09f4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:31.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 --2- 172.21.15.130:0/692767191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f09f40fff10 0x7f09f4198b60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:31.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 >> 172.21.15.130:0/692767191 conn(0x7f09f40fa2b0 msgr2=0x7f09f4107940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:58:31.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 shutdown_connections 2023-09-12T15:58:31.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:58:31.112+0000 7f09fbb86700 1 -- 172.21.15.130:0/692767191 wait complete. 2023-09-12T15:58:31.561 DEBUG:teuthology.orchestra.run.smithi130:osd.3> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.3.service 2023-09-12T15:58:31.564 INFO:tasks.cephadm:Deploying osd.4 on smithi159 with /dev/vg_nvme/lv_3... 2023-09-12T15:58:31.564 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-12T15:58:31.596 INFO:journalctl@ceph.osd.3.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T15:58:31.967 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.968 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.968 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:31.968 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:31.968 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.969 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:31 smithi130 ceph-mon[98670]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-09-12T15:58:31.969 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.969 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.969 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:31.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:31.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:31.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:31 smithi159 ceph-mon[102522]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-09-12T15:58:31.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:31.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:31.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:31.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:31 smithi203 ceph-mon[103431]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-09-12T15:58:31.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:31.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:32.130 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T15:58:32.657 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 15:58:32 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T15:58:32.359+0000 7f9fe2a21640 -1 osd.3 0 log_to_monitors true 2023-09-12T15:58:32.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:32 smithi130 ceph-mon[98670]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-12T15:58:32.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:32 smithi159 ceph-mon[102522]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-12T15:58:32.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:32 smithi203 ceph-mon[103431]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-12T15:58:33.582 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-12T15:58:33.583 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-12T15:58:33.583 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-09-12T15:58:33.583 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-09-12T15:58:33.584 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111095 s, 944 MB/s 2023-09-12T15:58:33.584 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-09-12T15:58:33.932 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:33 smithi159 ceph-mon[102522]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-12T15:58:33.932 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:33 smithi159 ceph-mon[102522]: osdmap e23: 4 total, 3 up, 4 in 2023-09-12T15:58:33.933 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:33 smithi159 ceph-mon[102522]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T15:58:33.933 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:33.933 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:33 smithi159 ceph-mon[102522]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-12T15:58:33.943 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 15:58:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T15:58:33.678+0000 7f9fd8098700 -1 osd.3 0 waiting for initial osdmap 2023-09-12T15:58:33.943 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 15:58:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T15:58:33.683+0000 7f9fcfe85700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T15:58:33.944 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:33 smithi130 ceph-mon[98670]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-12T15:58:33.944 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:33 smithi130 ceph-mon[98670]: osdmap e23: 4 total, 3 up, 4 in 2023-09-12T15:58:33.944 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:33 smithi130 ceph-mon[98670]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T15:58:33.944 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:33.944 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:33 smithi130 ceph-mon[98670]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-12T15:58:33.968 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2023-09-12T15:58:33.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:33 smithi203 ceph-mon[103431]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-12T15:58:33.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:33 smithi203 ceph-mon[103431]: osdmap e23: 4 total, 3 up, 4 in 2023-09-12T15:58:33.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:33 smithi203 ceph-mon[103431]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T15:58:33.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:33.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:33 smithi203 ceph-mon[103431]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-12T15:58:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T15:58:34.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T15:58:34.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: osdmap e24: 4 total, 3 up, 4 in 2023-09-12T15:58:34.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:34.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:34.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi130 2023-09-12T15:58:34.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:34.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:34.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi130 to 8584M 2023-09-12T15:58:34.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:34.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:34.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:34 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T15:58:34.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: osdmap e24: 4 total, 3 up, 4 in 2023-09-12T15:58:34.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:34.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:34.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi130 2023-09-12T15:58:34.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:34.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:34.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi130 to 8584M 2023-09-12T15:58:34.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:34.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:34.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:34 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T15:58:34.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: osdmap e24: 4 total, 3 up, 4 in 2023-09-12T15:58:34.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:34.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:34.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi130 2023-09-12T15:58:34.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:34.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:34.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi130 to 8584M 2023-09-12T15:58:34.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:34.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:34.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:34.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:34 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:35.787 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 -- 172.21.15.159:0/485943651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74101130 msgr2=0x7f9a741079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:35.787 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 --2- 172.21.15.159:0/485943651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74101130 0x7f9a741079e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00b0d0 tx=0x7f9a6c00b3e0 comp rx=0 tx=0).stop 2023-09-12T15:58:35.787 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 -- 172.21.15.159:0/485943651 shutdown_connections 2023-09-12T15:58:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 --2- 172.21.15.159:0/485943651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74101130 0x7f9a741079e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 --2- 172.21.15.159:0/485943651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740ff590 0x7f9a740ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 --2- 172.21.15.159:0/485943651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a74100790 0x7f9a74100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 -- 172.21.15.159:0/485943651 >> 172.21.15.159:0/485943651 conn(0x7f9a740fab10 msgr2=0x7f9a740fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:58:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 -- 172.21.15.159:0/485943651 shutdown_connections 2023-09-12T15:58:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 -- 172.21.15.159:0/485943651 wait complete. 2023-09-12T15:58:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.787+0000 7f9a79d02700 1 Processor -- start 2023-09-12T15:58:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a79d02700 1 -- start start 2023-09-12T15:58:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a79d02700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740ff590 0x7f9a74068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a79d02700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74100790 0x7f9a7419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a79d02700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a74101130 0x7f9a7419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:35.792 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a79d02700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419df40 con 0x7f9a74101130 2023-09-12T15:58:35.792 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a79d02700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419e0b0 con 0x7f9a74100790 2023-09-12T15:58:35.792 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a79d02700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419e270 con 0x7f9a740ff590 2023-09-12T15:58:35.792 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a72ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74100790 0x7f9a7419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:35.792 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a72ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74100790 0x7f9a7419b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33128/0 (socket says 172.21.15.159:33128) 2023-09-12T15:58:35.793 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a72ffd700 1 -- 172.21.15.159:0/1536889233 learned_addr learned my addr 172.21.15.159:0/1536889233 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-09-12T15:58:35.793 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a73fff700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a74101130 0x7f9a7419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:35.793 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a737fe700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740ff590 0x7f9a74068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:35.793 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a72ffd700 1 -- 172.21.15.159:0/1536889233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740ff590 msgr2=0x7f9a74068c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:35.793 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a72ffd700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740ff590 0x7f9a74068c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:35.793 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a72ffd700 1 -- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a74101130 msgr2=0x7f9a7419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:35.794 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a72ffd700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a74101130 0x7f9a7419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:35.794 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a72ffd700 1 -- 172.21.15.159:0/1536889233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c009d00 con 0x7f9a74100790 2023-09-12T15:58:35.794 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a73fff700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a74101130 0x7f9a7419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T15:58:35.794 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.788+0000 7f9a72ffd700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74100790 0x7f9a7419b150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400e9a0 tx=0x7f9a6400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:58:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.789+0000 7f9a70ff9700 1 -- 172.21.15.159:0/1536889233 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64009790 con 0x7f9a74100790 2023-09-12T15:58:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.789+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7419e550 con 0x7f9a74100790 2023-09-12T15:58:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.789+0000 7f9a70ff9700 1 -- 172.21.15.159:0/1536889233 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a6400cbd0 con 0x7f9a74100790 2023-09-12T15:58:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.789+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7419eaa0 con 0x7f9a74100790 2023-09-12T15:58:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.789+0000 7f9a70ff9700 1 -- 172.21.15.159:0/1536889233 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64018680 con 0x7f9a74100790 2023-09-12T15:58:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.790+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7410ada0 con 0x7f9a74100790 2023-09-12T15:58:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.791+0000 7f9a70ff9700 1 -- 172.21.15.159:0/1536889233 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f9a6400cd40 con 0x7f9a74100790 2023-09-12T15:58:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.791+0000 7f9a70ff9700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9a600a0640 0x7f9a600a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.791+0000 7f9a70ff9700 1 -- 172.21.15.159:0/1536889233 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f9a64014070 con 0x7f9a74100790 2023-09-12T15:58:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.791+0000 7f9a737fe700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9a600a0640 0x7f9a600a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.792+0000 7f9a737fe700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9a600a0640 0x7f9a600a2b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c005950 tx=0x7f9a5c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:58:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.796+0000 7f9a70ff9700 1 -- 172.21.15.159:0/1536889233 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9a64051e10 con 0x7f9a74100790 2023-09-12T15:58:35.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:35 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T15:58:35.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:35 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T15:58:35.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:35 smithi130 ceph-mon[98670]: osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] boot 2023-09-12T15:58:35.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:35 smithi130 ceph-mon[98670]: osdmap e25: 4 total, 4 up, 4 in 2023-09-12T15:58:35.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:35.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:35 smithi130 ceph-mon[98670]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:35.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:35 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T15:58:35.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:35 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T15:58:35.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:35 smithi159 ceph-mon[102522]: osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] boot 2023-09-12T15:58:35.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:35 smithi159 ceph-mon[102522]: osdmap e25: 4 total, 4 up, 4 in 2023-09-12T15:58:35.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:35.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:35 smithi159 ceph-mon[102522]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:35.990 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:35.991+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a74060a40 con 0x7f9a600a0640 2023-09-12T15:58:35.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:35 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T15:58:35.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:35 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T15:58:35.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:35 smithi203 ceph-mon[103431]: osd.3 [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] boot 2023-09-12T15:58:35.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:35 smithi203 ceph-mon[103431]: osdmap e25: 4 total, 4 up, 4 in 2023-09-12T15:58:35.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-12T15:58:35.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:35 smithi203 ceph-mon[103431]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:36.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:36 smithi130 ceph-mon[98670]: osdmap e26: 4 total, 4 up, 4 in 2023-09-12T15:58:36.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:36 smithi130 ceph-mon[98670]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:58:36.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:36 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:58:36.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:36 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:58:36.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:36 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:36.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:36 smithi159 ceph-mon[102522]: osdmap e26: 4 total, 4 up, 4 in 2023-09-12T15:58:36.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:36 smithi159 ceph-mon[102522]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:58:36.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:36 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:58:36.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:36 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:58:36.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:36 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:36.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:36 smithi203 ceph-mon[103431]: osdmap e26: 4 total, 4 up, 4 in 2023-09-12T15:58:36.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:36 smithi203 ceph-mon[103431]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:58:36.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:36 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:58:36.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:36 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:58:36.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:36 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:37.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:37 smithi130 ceph-mon[98670]: osdmap e27: 4 total, 4 up, 4 in 2023-09-12T15:58:37.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:37 smithi130 ceph-mon[98670]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:37.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:37 smithi159 ceph-mon[102522]: osdmap e27: 4 total, 4 up, 4 in 2023-09-12T15:58:37.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:37 smithi159 ceph-mon[102522]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:37.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:37 smithi203 ceph-mon[103431]: osdmap e27: 4 total, 4 up, 4 in 2023-09-12T15:58:37.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:37 smithi203 ceph-mon[103431]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:38.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:38 smithi130 ceph-mon[98670]: osdmap e28: 4 total, 4 up, 4 in 2023-09-12T15:58:38.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:38 smithi130 ceph-mon[98670]: from='client.? 172.21.15.159:0/1841418697' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e61d2d4-2479-45ab-a033-28ae1aeb8202"}]: dispatch 2023-09-12T15:58:38.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:38 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e61d2d4-2479-45ab-a033-28ae1aeb8202"}]: dispatch 2023-09-12T15:58:38.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:38 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e61d2d4-2479-45ab-a033-28ae1aeb8202"}]': finished 2023-09-12T15:58:38.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:38 smithi130 ceph-mon[98670]: osdmap e29: 5 total, 4 up, 5 in 2023-09-12T15:58:38.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:38 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:38.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:38 smithi159 ceph-mon[102522]: osdmap e28: 4 total, 4 up, 4 in 2023-09-12T15:58:38.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:38 smithi159 ceph-mon[102522]: from='client.? 172.21.15.159:0/1841418697' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e61d2d4-2479-45ab-a033-28ae1aeb8202"}]: dispatch 2023-09-12T15:58:38.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:38 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e61d2d4-2479-45ab-a033-28ae1aeb8202"}]: dispatch 2023-09-12T15:58:38.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:38 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e61d2d4-2479-45ab-a033-28ae1aeb8202"}]': finished 2023-09-12T15:58:38.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:38 smithi159 ceph-mon[102522]: osdmap e29: 5 total, 4 up, 5 in 2023-09-12T15:58:38.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:38 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:38.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:38 smithi203 ceph-mon[103431]: osdmap e28: 4 total, 4 up, 4 in 2023-09-12T15:58:38.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:38 smithi203 ceph-mon[103431]: from='client.? 172.21.15.159:0/1841418697' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e61d2d4-2479-45ab-a033-28ae1aeb8202"}]: dispatch 2023-09-12T15:58:38.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:38 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e61d2d4-2479-45ab-a033-28ae1aeb8202"}]: dispatch 2023-09-12T15:58:38.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:38 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e61d2d4-2479-45ab-a033-28ae1aeb8202"}]': finished 2023-09-12T15:58:38.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:38 smithi203 ceph-mon[103431]: osdmap e29: 5 total, 4 up, 5 in 2023-09-12T15:58:38.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:38 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:39.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:39 smithi130 ceph-mon[98670]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-12T15:58:39.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:39 smithi130 ceph-mon[98670]: from='client.? 172.21.15.159:0/3573467851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:58:39.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:39 smithi159 ceph-mon[102522]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-12T15:58:39.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:39 smithi159 ceph-mon[102522]: from='client.? 172.21.15.159:0/3573467851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:58:39.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:39 smithi203 ceph-mon[103431]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-12T15:58:39.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:39 smithi203 ceph-mon[103431]: from='client.? 172.21.15.159:0/3573467851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:58:42.064 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:41 smithi159 ceph-mon[102522]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-09-12T15:58:42.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:41 smithi130 ceph-mon[98670]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-09-12T15:58:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:41 smithi203 ceph-mon[103431]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-09-12T15:58:43.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:43 smithi159 ceph-mon[102522]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-12T15:58:44.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:43 smithi130 ceph-mon[98670]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-12T15:58:44.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:43 smithi203 ceph-mon[103431]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-12T15:58:45.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:45 smithi159 ceph-mon[102522]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-12T15:58:45.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-12T15:58:45.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:45.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:45 smithi159 ceph-mon[102522]: Deploying daemon osd.4 on smithi159 2023-09-12T15:58:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:45 smithi130 ceph-mon[98670]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-12T15:58:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-12T15:58:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:45 smithi130 ceph-mon[98670]: Deploying daemon osd.4 on smithi159 2023-09-12T15:58:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:45 smithi203 ceph-mon[103431]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-12T15:58:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-12T15:58:46.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:46.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:45 smithi203 ceph-mon[103431]: Deploying daemon osd.4 on smithi159 2023-09-12T15:58:47.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:47 smithi130 ceph-mon[98670]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-12T15:58:47.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:47 smithi159 ceph-mon[102522]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-12T15:58:47.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:47 smithi203 ceph-mon[103431]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-12T15:58:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:49 smithi130 ceph-mon[98670]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-09-12T15:58:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:50.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:49 smithi159 ceph-mon[102522]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-09-12T15:58:50.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:50.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:50.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:49 smithi203 ceph-mon[103431]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-09-12T15:58:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:50.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:50.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:51.697 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 4 on host 'smithi159' 2023-09-12T15:58:51.698 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.697+0000 7f9a70ff9700 1 -- 172.21.15.159:0/1536889233 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9a74060a40 con 0x7f9a600a0640 2023-09-12T15:58:51.700 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9a600a0640 msgr2=0x7f9a600a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:51.700 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9a600a0640 0x7f9a600a2b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c005950 tx=0x7f9a5c0058e0 comp rx=0 tx=0).stop 2023-09-12T15:58:51.700 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74100790 msgr2=0x7f9a7419b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:51.700 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74100790 0x7f9a7419b150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400e9a0 tx=0x7f9a6400ed60 comp rx=0 tx=0).stop 2023-09-12T15:58:51.700 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 shutdown_connections 2023-09-12T15:58:51.701 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9a600a0640 0x7f9a600a2b00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:51.701 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a74101130 0x7f9a7419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:51.701 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740ff590 0x7f9a74068c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:51.701 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 --2- 172.21.15.159:0/1536889233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a74100790 0x7f9a7419b150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:51.701 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 >> 172.21.15.159:0/1536889233 conn(0x7f9a740fab10 msgr2=0x7f9a740fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:58:51.702 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 shutdown_connections 2023-09-12T15:58:51.702 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:58:51.700+0000 7f9a79d02700 1 -- 172.21.15.159:0/1536889233 wait complete. 2023-09-12T15:58:51.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:51 smithi159 ceph-mon[102522]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:51.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:51.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:51.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:51.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:51.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:51.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:51.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:51.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:52.110 DEBUG:teuthology.orchestra.run.smithi159:osd.4> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.4.service 2023-09-12T15:58:52.113 INFO:tasks.cephadm:Deploying osd.5 on smithi203 with /dev/vg_nvme/lv_3... 2023-09-12T15:58:52.113 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-12T15:58:52.149 INFO:journalctl@ceph.osd.4.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T15:58:52.149 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:51 smithi203 ceph-mon[103431]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:52.150 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.150 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.150 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:52.150 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:52.150 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:51 smithi130 ceph-mon[98670]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:58:52.679 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T15:58:53.225 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 15:58:52 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4[115429]: 2023-09-12T15:58:52.919+0000 7ff9a5dd2640 -1 osd.4 0 log_to_monitors true 2023-09-12T15:58:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:53 smithi130 ceph-mon[98670]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:53 smithi130 ceph-mon[98670]: from='osd.4 [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-12T15:58:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:53 smithi130 ceph-mon[98670]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-12T15:58:54.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:53 smithi159 ceph-mon[102522]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:54.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:53 smithi159 ceph-mon[102522]: from='osd.4 [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-12T15:58:54.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:53 smithi159 ceph-mon[102522]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-12T15:58:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:53 smithi203 ceph-mon[103431]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:53 smithi203 ceph-mon[103431]: from='osd.4 [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-12T15:58:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:53 smithi203 ceph-mon[103431]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-12T15:58:54.327 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-12T15:58:54.327 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-12T15:58:54.328 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2023-09-12T15:58:54.328 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2023-09-12T15:58:54.328 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0108906 s, 963 MB/s 2023-09-12T15:58:54.328 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2023-09-12T15:58:54.736 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi203:vg_nvme/lv_3 2023-09-12T15:58:54.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-12T15:58:54.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: osdmap e30: 5 total, 4 up, 5 in 2023-09-12T15:58:54.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:54.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='osd.4 [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:58:54.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:58:54.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:54.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:54.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:54.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:54.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:54.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:54.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:54.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-12T15:58:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: osdmap e30: 5 total, 4 up, 5 in 2023-09-12T15:58:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:55.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='osd.4 [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:58:55.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:58:55.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:55.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:55.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:55.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:55.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:55.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:55.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:55.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:55.230 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4[115429]: 2023-09-12T15:58:54.787+0000 7ff99b449700 -1 osd.4 0 waiting for initial osdmap 2023-09-12T15:58:55.230 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4[115429]: 2023-09-12T15:58:54.794+0000 7ff993a37700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T15:58:55.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-12T15:58:55.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: osdmap e30: 5 total, 4 up, 5 in 2023-09-12T15:58:55.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:55.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='osd.4 [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:58:55.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:58:55.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:55.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:55.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:55.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:58:55.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:55.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:55.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:58:55.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:58:55.334 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T15:58:55.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:55 smithi203 ceph-mon[103431]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:55.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:55 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi159 2023-09-12T15:58:55.985 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:55 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi159 to 8584M 2023-09-12T15:58:55.985 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:55 smithi203 ceph-mon[103431]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T15:58:55.985 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:55 smithi203 ceph-mon[103431]: osdmap e31: 5 total, 4 up, 5 in 2023-09-12T15:58:55.985 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:55.985 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:55 smithi130 ceph-mon[98670]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:55 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi159 2023-09-12T15:58:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:55 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi159 to 8584M 2023-09-12T15:58:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:55 smithi130 ceph-mon[98670]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T15:58:56.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:55 smithi130 ceph-mon[98670]: osdmap e31: 5 total, 4 up, 5 in 2023-09-12T15:58:56.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:56.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:56.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:55 smithi159 ceph-mon[102522]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:56.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:55 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi159 2023-09-12T15:58:56.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:55 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi159 to 8584M 2023-09-12T15:58:56.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:55 smithi159 ceph-mon[102522]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T15:58:56.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:55 smithi159 ceph-mon[102522]: osdmap e31: 5 total, 4 up, 5 in 2023-09-12T15:58:56.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:56.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:56.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.650+0000 7f237bae8700 1 -- 172.21.15.203:0/829634734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23741008b0 msgr2=0x7f2374107160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:56.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.650+0000 7f237bae8700 1 --2- 172.21.15.203:0/829634734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23741008b0 0x7f2374107160 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f236c00b0a0 tx=0x7f236c00b3b0 comp rx=0 tx=0).stop 2023-09-12T15:58:56.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 -- 172.21.15.203:0/829634734 shutdown_connections 2023-09-12T15:58:56.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 --2- 172.21.15.203:0/829634734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23741008b0 0x7f2374107160 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:56.654 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 --2- 172.21.15.203:0/829634734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23740fed10 0x7f23740ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:56.654 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 --2- 172.21.15.203:0/829634734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23740fff10 0x7f2374100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:56.654 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 -- 172.21.15.203:0/829634734 >> 172.21.15.203:0/829634734 conn(0x7f23740fa2b0 msgr2=0x7f23740fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:58:56.654 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 -- 172.21.15.203:0/829634734 shutdown_connections 2023-09-12T15:58:56.654 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 -- 172.21.15.203:0/829634734 wait complete. 2023-09-12T15:58:56.655 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 Processor -- start 2023-09-12T15:58:56.655 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 -- start start 2023-09-12T15:58:56.655 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23740fed10 0x7f2374196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:56.655 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23740fff10 0x7f2374196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:56.655 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23741008b0 0x7f237419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:56.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237419dfb0 con 0x7f23741008b0 2023-09-12T15:58:56.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237419e0f0 con 0x7f23740fff10 2023-09-12T15:58:56.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.651+0000 7f237bae8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237419e280 con 0x7f23740fed10 2023-09-12T15:58:56.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379884700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23740fed10 0x7f2374196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:56.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379884700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23740fed10 0x7f2374196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:41484/0 (socket says 172.21.15.203:41484) 2023-09-12T15:58:56.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379884700 1 -- 172.21.15.203:0/4149383939 learned_addr learned my addr 172.21.15.203:0/4149383939 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-09-12T15:58:56.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f237a085700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23741008b0 0x7f237419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:56.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379083700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23740fff10 0x7f2374196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:56.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379884700 1 -- 172.21.15.203:0/4149383939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23740fff10 msgr2=0x7f2374196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:56.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379884700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23740fff10 0x7f2374196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:56.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379884700 1 -- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23741008b0 msgr2=0x7f237419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:58:56.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379884700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23741008b0 0x7f237419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:58:56.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379884700 1 -- 172.21.15.203:0/4149383939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c009d00 con 0x7f23740fed10 2023-09-12T15:58:56.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f2379884700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23740fed10 0x7f2374196420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f237000d9b0 tx=0x7f237000dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:58:56.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f236affd700 1 -- 172.21.15.203:0/4149383939 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237000be80 con 0x7f23740fed10 2023-09-12T15:58:56.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237419e560 con 0x7f23740fed10 2023-09-12T15:58:56.659 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f236affd700 1 -- 172.21.15.203:0/4149383939 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23700044e0 con 0x7f23740fed10 2023-09-12T15:58:56.659 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f236affd700 1 -- 172.21.15.203:0/4149383939 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370003b40 con 0x7f23740fed10 2023-09-12T15:58:56.659 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.652+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237419ea60 con 0x7f23740fed10 2023-09-12T15:58:56.660 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.653+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2374109c90 con 0x7f23740fed10 2023-09-12T15:58:56.661 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.655+0000 7f236affd700 1 -- 172.21.15.203:0/4149383939 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f2370009940 con 0x7f23740fed10 2023-09-12T15:58:56.661 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.655+0000 7f236affd700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23600a0640 0x7f23600a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:58:56.661 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.655+0000 7f236affd700 1 -- 172.21.15.203:0/4149383939 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f23700cacf0 con 0x7f23740fed10 2023-09-12T15:58:56.661 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.656+0000 7f2379083700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23600a0640 0x7f23600a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:58:56.661 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.656+0000 7f2379083700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23600a0640 0x7f23600a2b00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2364005950 tx=0x7f23640058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:58:56.662 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.659+0000 7f236affd700 1 -- 172.21.15.203:0/4149383939 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f23700957f0 con 0x7f23740fed10 2023-09-12T15:58:56.856 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:58:56.855+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2374060a40 con 0x7f23600a0640 2023-09-12T15:58:57.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:56 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T15:58:57.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:56 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T15:58:57.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:56 smithi130 ceph-mon[98670]: osd.4 [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] boot 2023-09-12T15:58:57.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:56 smithi130 ceph-mon[98670]: osdmap e32: 5 total, 5 up, 5 in 2023-09-12T15:58:57.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:56 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:57.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:56 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T15:58:57.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:56 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T15:58:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:56 smithi159 ceph-mon[102522]: osd.4 [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] boot 2023-09-12T15:58:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:56 smithi159 ceph-mon[102522]: osdmap e32: 5 total, 5 up, 5 in 2023-09-12T15:58:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:56 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:57.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:56 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T15:58:57.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:56 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T15:58:57.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:56 smithi203 ceph-mon[103431]: osd.4 [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] boot 2023-09-12T15:58:57.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:56 smithi203 ceph-mon[103431]: osdmap e32: 5 total, 5 up, 5 in 2023-09-12T15:58:57.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:56 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-12T15:58:57.967 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:57 smithi203 ceph-mon[103431]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:57.967 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:57 smithi203 ceph-mon[103431]: osdmap e33: 5 total, 5 up, 5 in 2023-09-12T15:58:57.968 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:57 smithi203 ceph-mon[103431]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:58:57.968 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:58:57.968 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:58:57.968 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:58.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:57 smithi130 ceph-mon[98670]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:58.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:57 smithi130 ceph-mon[98670]: osdmap e33: 5 total, 5 up, 5 in 2023-09-12T15:58:58.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:57 smithi130 ceph-mon[98670]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:58:58.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:58:58.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:58:58.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:58:58.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:57 smithi159 ceph-mon[102522]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-12T15:58:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:57 smithi159 ceph-mon[102522]: osdmap e33: 5 total, 5 up, 5 in 2023-09-12T15:58:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:57 smithi159 ceph-mon[102522]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:58:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:58:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:58:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:00.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:59 smithi130 ceph-mon[98670]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:00.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:59 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/1314794920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fc371aa-08c6-49f0-8f7f-0ce25705e987"}]: dispatch 2023-09-12T15:59:00.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:59 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fc371aa-08c6-49f0-8f7f-0ce25705e987"}]: dispatch 2023-09-12T15:59:00.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:59 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fc371aa-08c6-49f0-8f7f-0ce25705e987"}]': finished 2023-09-12T15:59:00.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:59 smithi130 ceph-mon[98670]: osdmap e34: 6 total, 5 up, 6 in 2023-09-12T15:59:00.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:58:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:00.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:59 smithi159 ceph-mon[102522]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:00.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:59 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/1314794920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fc371aa-08c6-49f0-8f7f-0ce25705e987"}]: dispatch 2023-09-12T15:59:00.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:59 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fc371aa-08c6-49f0-8f7f-0ce25705e987"}]: dispatch 2023-09-12T15:59:00.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:59 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fc371aa-08c6-49f0-8f7f-0ce25705e987"}]': finished 2023-09-12T15:59:00.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:59 smithi159 ceph-mon[102522]: osdmap e34: 6 total, 5 up, 6 in 2023-09-12T15:59:00.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:58:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:00.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:59 smithi203 ceph-mon[103431]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:00.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:59 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/1314794920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fc371aa-08c6-49f0-8f7f-0ce25705e987"}]: dispatch 2023-09-12T15:59:00.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:59 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fc371aa-08c6-49f0-8f7f-0ce25705e987"}]: dispatch 2023-09-12T15:59:00.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:59 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fc371aa-08c6-49f0-8f7f-0ce25705e987"}]': finished 2023-09-12T15:59:00.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:59 smithi203 ceph-mon[103431]: osdmap e34: 6 total, 5 up, 6 in 2023-09-12T15:59:00.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:58:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:01.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:00 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/2349423878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:59:01.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:00 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/2349423878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:59:01.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:00 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/2349423878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:59:02.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:01 smithi130 ceph-mon[98670]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:02.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:01 smithi159 ceph-mon[102522]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:02.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:01 smithi203 ceph-mon[103431]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:04.089 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:03 smithi203 ceph-mon[103431]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:04.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:03 smithi130 ceph-mon[98670]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:04.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:03 smithi159 ceph-mon[102522]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:06.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:05 smithi130 ceph-mon[98670]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:06.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:05 smithi159 ceph-mon[102522]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:06.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:05 smithi203 ceph-mon[103431]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:07.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:06 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-12T15:59:07.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:06 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:07.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:06 smithi203 ceph-mon[103431]: Deploying daemon osd.5 on smithi203 2023-09-12T15:59:07.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:06 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-12T15:59:07.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:06 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:07.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:06 smithi130 ceph-mon[98670]: Deploying daemon osd.5 on smithi203 2023-09-12T15:59:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:06 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-12T15:59:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:06 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:06 smithi159 ceph-mon[102522]: Deploying daemon osd.5 on smithi203 2023-09-12T15:59:08.174 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:07 smithi203 ceph-mon[103431]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:08.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:07 smithi130 ceph-mon[98670]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:08.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:07 smithi159 ceph-mon[102522]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:10.170 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:09 smithi203 ceph-mon[103431]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:09 smithi130 ceph-mon[98670]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:10.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:09 smithi159 ceph-mon[102522]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:11.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:11.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:11.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:11 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:11.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:11 smithi203 ceph-mon[103431]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:11.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:11.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:11.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:11 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:11.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:11 smithi130 ceph-mon[98670]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:11.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:11.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:11.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:11 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:11.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:11 smithi159 ceph-mon[102522]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:13.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.466+0000 7f236affd700 1 -- 172.21.15.203:0/4149383939 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2374060a40 con 0x7f23600a0640 2023-09-12T15:59:13.466 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 5 on host 'smithi203' 2023-09-12T15:59:13.468 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23600a0640 msgr2=0x7f23600a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:13.468 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23600a0640 0x7f23600a2b00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2364005950 tx=0x7f23640058e0 comp rx=0 tx=0).stop 2023-09-12T15:59:13.469 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23740fed10 msgr2=0x7f2374196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:13.469 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23740fed10 0x7f2374196420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f237000d9b0 tx=0x7f237000dd70 comp rx=0 tx=0).stop 2023-09-12T15:59:13.469 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 shutdown_connections 2023-09-12T15:59:13.469 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23600a0640 0x7f23600a2b00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:13.469 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23741008b0 0x7f237419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:13.470 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23740fed10 0x7f2374196420 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:13.470 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 --2- 172.21.15.203:0/4149383939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23740fff10 0x7f2374196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:13.470 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 >> 172.21.15.203:0/4149383939 conn(0x7f23740fa2b0 msgr2=0x7f23740fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:59:13.470 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 shutdown_connections 2023-09-12T15:59:13.470 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:13.469+0000 7f237bae8700 1 -- 172.21.15.203:0/4149383939 wait complete. 2023-09-12T15:59:13.924 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:13 smithi203 ceph-mon[103431]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:13.924 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:13.924 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:13.925 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:13.925 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:13.925 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:13.925 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:13.925 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:13.926 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:13.927 DEBUG:teuthology.orchestra.run.smithi203:osd.5> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.5.service 2023-09-12T15:59:13.929 INFO:tasks.cephadm:Deploying osd.6 on smithi130 with /dev/vg_nvme/lv_2... 2023-09-12T15:59:13.930 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-12T15:59:13.971 INFO:journalctl@ceph.osd.5.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T15:59:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:13 smithi130 ceph-mon[98670]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:13 smithi159 ceph-mon[102522]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:14.509 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:59:14.657 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 15:59:14 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T15:59:14.464+0000 7f2cff817640 -1 osd.5 0 log_to_monitors true 2023-09-12T15:59:14.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:14 smithi130 ceph-mon[98670]: from='osd.5 [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-12T15:59:14.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:14 smithi130 ceph-mon[98670]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-12T15:59:14.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:14 smithi203 ceph-mon[103431]: from='osd.5 [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-12T15:59:14.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:14 smithi203 ceph-mon[103431]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-12T15:59:15.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:14 smithi159 ceph-mon[102522]: from='osd.5 [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-12T15:59:15.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:14 smithi159 ceph-mon[102522]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-12T15:59:15.994 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-12T15:59:15.994 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-12T15:59:15.994 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2023-09-12T15:59:15.994 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2023-09-12T15:59:15.995 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126584 s, 828 MB/s 2023-09-12T15:59:15.995 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2023-09-12T15:59:16.129 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:15 smithi130 ceph-mon[98670]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:16.129 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:15 smithi130 ceph-mon[98670]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-12T15:59:16.129 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:15 smithi130 ceph-mon[98670]: osdmap e35: 6 total, 5 up, 6 in 2023-09-12T15:59:16.130 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:16.130 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:15 smithi130 ceph-mon[98670]: from='osd.5 [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:59:16.130 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:15 smithi130 ceph-mon[98670]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:59:16.163 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 15:59:15 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T15:59:15.874+0000 7f2cf4e8e700 -1 osd.5 0 waiting for initial osdmap 2023-09-12T15:59:16.163 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 15:59:15 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T15:59:15.881+0000 7f2cecc7b700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T15:59:16.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:15 smithi203 ceph-mon[103431]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:16.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:15 smithi203 ceph-mon[103431]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-12T15:59:16.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:15 smithi203 ceph-mon[103431]: osdmap e35: 6 total, 5 up, 6 in 2023-09-12T15:59:16.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:15 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:16.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:15 smithi203 ceph-mon[103431]: from='osd.5 [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:59:16.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:15 smithi203 ceph-mon[103431]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:59:16.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:15 smithi159 ceph-mon[102522]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:16.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:15 smithi159 ceph-mon[102522]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-12T15:59:16.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:15 smithi159 ceph-mon[102522]: osdmap e35: 6 total, 5 up, 6 in 2023-09-12T15:59:16.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:15 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:16.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:15 smithi159 ceph-mon[102522]: from='osd.5 [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:59:16.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:15 smithi159 ceph-mon[102522]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T15:59:16.351 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi130:vg_nvme/lv_2 2023-09-12T15:59:16.911 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T15:59:17.134 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T15:59:17.135 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: osdmap e36: 6 total, 5 up, 6 in 2023-09-12T15:59:17.135 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:17.135 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:17.135 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: osd.5 [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] boot 2023-09-12T15:59:17.136 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: osdmap e37: 6 total, 6 up, 6 in 2023-09-12T15:59:17.136 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:17.136 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.136 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.136 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:17.136 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:17.137 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.137 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:17.137 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:17.137 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T15:59:17.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: osdmap e36: 6 total, 5 up, 6 in 2023-09-12T15:59:17.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:17.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:17.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: osd.5 [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] boot 2023-09-12T15:59:17.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: osdmap e37: 6 total, 6 up, 6 in 2023-09-12T15:59:17.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:17.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:17.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:17.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:17.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:17.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T15:59:17.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: osdmap e36: 6 total, 5 up, 6 in 2023-09-12T15:59:17.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:17.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:17.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: osd.5 [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] boot 2023-09-12T15:59:17.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: osdmap e37: 6 total, 6 up, 6 in 2023-09-12T15:59:17.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-12T15:59:17.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:17.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:17.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:17.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:17.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:17.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:18.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.105+0000 7f7fcd852700 1 -- 172.21.15.130:0/4254527945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8102230 msgr2=0x7f7fc8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:18.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.105+0000 7f7fcd852700 1 --2- 172.21.15.130:0/4254527945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8102230 0x7f7fc8108ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000b0d0 tx=0x7f7fc000b3e0 comp rx=0 tx=0).stop 2023-09-12T15:59:18.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.105+0000 7f7fcd852700 1 -- 172.21.15.130:0/4254527945 shutdown_connections 2023-09-12T15:59:18.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.105+0000 7f7fcd852700 1 --2- 172.21.15.130:0/4254527945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8102230 0x7f7fc8108ae0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:18.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.105+0000 7f7fcd852700 1 --2- 172.21.15.130:0/4254527945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc8100690 0x7f7fc8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:18.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.105+0000 7f7fcd852700 1 --2- 172.21.15.130:0/4254527945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fc8101890 0x7f7fc8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:18.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.105+0000 7f7fcd852700 1 -- 172.21.15.130:0/4254527945 >> 172.21.15.130:0/4254527945 conn(0x7f7fc80fbbf0 msgr2=0x7f7fc80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:59:18.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.105+0000 7f7fcd852700 1 -- 172.21.15.130:0/4254527945 shutdown_connections 2023-09-12T15:59:18.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.105+0000 7f7fcd852700 1 -- 172.21.15.130:0/4254527945 wait complete. 2023-09-12T15:59:18.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fcd852700 1 Processor -- start 2023-09-12T15:59:18.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fcd852700 1 -- start start 2023-09-12T15:59:18.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fcd852700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc8100690 0x7f7fc81964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:18.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fcd852700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8101890 0x7f7fc8196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:18.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fcd852700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fc8102230 0x7f7fc819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:18.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fcd852700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc819e050 con 0x7f7fc8101890 2023-09-12T15:59:18.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fcd852700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc819e190 con 0x7f7fc8100690 2023-09-12T15:59:18.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fcd852700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc819e320 con 0x7f7fc8102230 2023-09-12T15:59:18.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fc67fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8101890 0x7f7fc8196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:18.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fc67fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8101890 0x7f7fc8196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52592/0 (socket says 172.21.15.130:52592) 2023-09-12T15:59:18.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fc67fc700 1 -- 172.21.15.130:0/816602885 learned_addr learned my addr 172.21.15.130:0/816602885 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T15:59:18.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fc77fe700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fc8102230 0x7f7fc819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:18.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fc6ffd700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc8100690 0x7f7fc81964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:18.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.106+0000 7f7fc67fc700 1 -- 172.21.15.130:0/816602885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fc8102230 msgr2=0x7f7fc819da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:18.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fc67fc700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fc8102230 0x7f7fc819da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:18.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fc67fc700 1 -- 172.21.15.130:0/816602885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc8100690 msgr2=0x7f7fc81964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:18.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fc67fc700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc8100690 0x7f7fc81964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:18.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fc67fc700 1 -- 172.21.15.130:0/816602885 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc0009d00 con 0x7f7fc8101890 2023-09-12T15:59:18.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fc6ffd700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc8100690 0x7f7fc81964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T15:59:18.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fc67fc700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8101890 0x7f7fc8196a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800e9f0 tx=0x7f7fb800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:59:18.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fcc850700 1 -- 172.21.15.130:0/816602885 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80097b0 con 0x7f7fc8101890 2023-09-12T15:59:18.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fcc850700 1 -- 172.21.15.130:0/816602885 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fb8004cf0 con 0x7f7fc8101890 2023-09-12T15:59:18.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc819e600 con 0x7f7fc8101890 2023-09-12T15:59:18.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fcc850700 1 -- 172.21.15.130:0/816602885 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80055d0 con 0x7f7fc8101890 2023-09-12T15:59:18.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.107+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc819eab0 con 0x7f7fc8101890 2023-09-12T15:59:18.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.108+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc8107c90 con 0x7f7fc8101890 2023-09-12T15:59:18.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.109+0000 7f7fcc850700 1 -- 172.21.15.130:0/816602885 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f7fb800cc60 con 0x7f7fc8101890 2023-09-12T15:59:18.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.110+0000 7f7fcc850700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7fb00a0640 0x7f7fb00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:18.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.110+0000 7f7fc6ffd700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7fb00a0640 0x7f7fb00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:18.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.110+0000 7f7fcc850700 1 -- 172.21.15.130:0/816602885 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f7fb8014070 con 0x7f7fc8101890 2023-09-12T15:59:18.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.110+0000 7f7fc6ffd700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7fb00a0640 0x7f7fb00a2b00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005fd0 tx=0x7f7fbc005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:59:18.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.114+0000 7f7fcc850700 1 -- 172.21.15.130:0/816602885 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7fb8095910 con 0x7f7fc8101890 2023-09-12T15:59:18.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:17 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T15:59:18.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:17 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T15:59:18.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:17 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi203 2023-09-12T15:59:18.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:17 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi203 to 8584M 2023-09-12T15:59:18.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:17 smithi130 ceph-mon[98670]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:18.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:17 smithi130 ceph-mon[98670]: osdmap e38: 6 total, 6 up, 6 in 2023-09-12T15:59:18.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:17 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T15:59:18.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:17 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T15:59:18.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:17 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi203 2023-09-12T15:59:18.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:17 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi203 to 8584M 2023-09-12T15:59:18.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:17 smithi159 ceph-mon[102522]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:18.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:17 smithi159 ceph-mon[102522]: osdmap e38: 6 total, 6 up, 6 in 2023-09-12T15:59:18.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:17 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T15:59:18.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:17 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T15:59:18.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:17 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi203 2023-09-12T15:59:18.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:17 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi203 to 8584M 2023-09-12T15:59:18.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:17 smithi203 ceph-mon[103431]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-12T15:59:18.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:17 smithi203 ceph-mon[103431]: osdmap e38: 6 total, 6 up, 6 in 2023-09-12T15:59:18.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:18.313+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fc802cfa0 con 0x7f7fb00a0640 2023-09-12T15:59:19.140 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:59:19.140 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:59:19.140 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:19.140 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:18 smithi130 ceph-mon[98670]: osdmap e39: 6 total, 6 up, 6 in 2023-09-12T15:59:19.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:59:19.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:59:19.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:19.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:18 smithi159 ceph-mon[102522]: osdmap e39: 6 total, 6 up, 6 in 2023-09-12T15:59:19.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:59:19.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:59:19.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:19.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:18 smithi203 ceph-mon[103431]: osdmap e39: 6 total, 6 up, 6 in 2023-09-12T15:59:20.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:19 smithi130 ceph-mon[98670]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:59:20.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:19 smithi130 ceph-mon[98670]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:20.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:19 smithi159 ceph-mon[102522]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:59:20.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:19 smithi159 ceph-mon[102522]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:20.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:19 smithi203 ceph-mon[103431]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:59:20.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:19 smithi203 ceph-mon[103431]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:21.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:20 smithi130 ceph-mon[98670]: osdmap e40: 6 total, 6 up, 6 in 2023-09-12T15:59:21.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:20 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/723836058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cb4de31-02a3-4822-944d-14a9fe33f79c"}]: dispatch 2023-09-12T15:59:21.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:20 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/723836058' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cb4de31-02a3-4822-944d-14a9fe33f79c"}]': finished 2023-09-12T15:59:21.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:20 smithi130 ceph-mon[98670]: osdmap e41: 7 total, 6 up, 7 in 2023-09-12T15:59:21.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:21.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:20 smithi159 ceph-mon[102522]: osdmap e40: 6 total, 6 up, 6 in 2023-09-12T15:59:21.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:20 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/723836058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cb4de31-02a3-4822-944d-14a9fe33f79c"}]: dispatch 2023-09-12T15:59:21.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:20 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/723836058' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cb4de31-02a3-4822-944d-14a9fe33f79c"}]': finished 2023-09-12T15:59:21.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:20 smithi159 ceph-mon[102522]: osdmap e41: 7 total, 6 up, 7 in 2023-09-12T15:59:21.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:21.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:20 smithi203 ceph-mon[103431]: osdmap e40: 6 total, 6 up, 6 in 2023-09-12T15:59:21.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:20 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/723836058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cb4de31-02a3-4822-944d-14a9fe33f79c"}]: dispatch 2023-09-12T15:59:21.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:20 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/723836058' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cb4de31-02a3-4822-944d-14a9fe33f79c"}]': finished 2023-09-12T15:59:21.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:20 smithi203 ceph-mon[103431]: osdmap e41: 7 total, 6 up, 7 in 2023-09-12T15:59:21.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:22.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:21 smithi130 ceph-mon[98670]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:22.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:21 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/77913032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:59:22.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:21 smithi159 ceph-mon[102522]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:22.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:21 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/77913032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:59:22.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:21 smithi203 ceph-mon[103431]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:22.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:21 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/77913032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:59:24.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:23 smithi130 ceph-mon[98670]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:24.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:23 smithi159 ceph-mon[102522]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:24.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:23 smithi203 ceph-mon[103431]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:26.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:25 smithi130 ceph-mon[98670]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-09-12T15:59:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:25 smithi159 ceph-mon[102522]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-09-12T15:59:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:25 smithi203 ceph-mon[103431]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-09-12T15:59:27.874 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:27 smithi130 ceph-mon[98670]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-12T15:59:27.874 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-12T15:59:27.875 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:27.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:27 smithi159 ceph-mon[102522]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-12T15:59:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-12T15:59:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:27 smithi203 ceph-mon[103431]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-12T15:59:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-12T15:59:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:28.899 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:28 smithi130 ceph-mon[98670]: Deploying daemon osd.6 on smithi130 2023-09-12T15:59:28.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:28 smithi159 ceph-mon[102522]: Deploying daemon osd.6 on smithi130 2023-09-12T15:59:28.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:28 smithi203 ceph-mon[103431]: Deploying daemon osd.6 on smithi130 2023-09-12T15:59:29.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:29 smithi130 ceph-mon[98670]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-09-12T15:59:29.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:29 smithi159 ceph-mon[102522]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-09-12T15:59:29.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:29 smithi203 ceph-mon[103431]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-09-12T15:59:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:31 smithi130 ceph-mon[98670]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-09-12T15:59:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:32.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:31 smithi159 ceph-mon[102522]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-09-12T15:59:32.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:32.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:32.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:31 smithi203 ceph-mon[103431]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-09-12T15:59:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:32.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.084 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:33 smithi130 ceph-mon[98670]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-12T15:59:34.085 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.085 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.085 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:34.085 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:34.086 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.110 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 6 on host 'smithi130' 2023-09-12T15:59:34.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.110+0000 7f7fcc850700 1 -- 172.21.15.130:0/816602885 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7fc802cfa0 con 0x7f7fb00a0640 2023-09-12T15:59:34.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7fb00a0640 msgr2=0x7f7fb00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:34.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7fb00a0640 0x7f7fb00a2b00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005fd0 tx=0x7f7fbc005f00 comp rx=0 tx=0).stop 2023-09-12T15:59:34.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8101890 msgr2=0x7f7fc8196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:34.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8101890 0x7f7fc8196a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800e9f0 tx=0x7f7fb800ed00 comp rx=0 tx=0).stop 2023-09-12T15:59:34.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 shutdown_connections 2023-09-12T15:59:34.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7fb00a0640 0x7f7fb00a2b00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:34.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fc8102230 0x7f7fc819da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:34.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc8100690 0x7f7fc81964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:34.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 --2- 172.21.15.130:0/816602885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7fc8101890 0x7f7fc8196a00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:34.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 >> 172.21.15.130:0/816602885 conn(0x7f7fc80fbbf0 msgr2=0x7f7fc8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:59:34.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 shutdown_connections 2023-09-12T15:59:34.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T15:59:34.112+0000 7f7fcd852700 1 -- 172.21.15.130:0/816602885 wait complete. 2023-09-12T15:59:34.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:33 smithi159 ceph-mon[102522]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-12T15:59:34.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:34.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:34.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:33 smithi203 ceph-mon[103431]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-12T15:59:34.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:34.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:34.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:34.506 DEBUG:teuthology.orchestra.run.smithi130:osd.6> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.6.service 2023-09-12T15:59:34.509 INFO:tasks.cephadm:Deploying osd.7 on smithi159 with /dev/vg_nvme/lv_2... 2023-09-12T15:59:34.509 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-12T15:59:34.539 INFO:journalctl@ceph.osd.6.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T15:59:35.086 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T15:59:35.302 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:35.302 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:35.303 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:35.303 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:35 smithi130 ceph-mon[98670]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-12T15:59:35.303 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 15:59:35 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T15:59:35.190+0000 7fd08940e640 -1 osd.6 0 log_to_monitors true 2023-09-12T15:59:35.343 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:35.344 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:35.344 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:35.344 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:35 smithi159 ceph-mon[102522]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-12T15:59:35.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:35.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:35.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:35.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:35 smithi203 ceph-mon[103431]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-12T15:59:36.137 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:36 smithi159 ceph-mon[102522]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-12T15:59:36.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:36 smithi130 ceph-mon[98670]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-12T15:59:36.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:36 smithi203 ceph-mon[103431]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-12T15:59:36.575 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-12T15:59:36.575 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-12T15:59:36.575 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-09-12T15:59:36.575 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-09-12T15:59:36.575 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01111 s, 944 MB/s 2023-09-12T15:59:36.576 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-09-12T15:59:36.943 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2023-09-12T15:59:37.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-12T15:59:37.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: osdmap e42: 7 total, 6 up, 7 in 2023-09-12T15:59:37.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T15:59:37.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:37.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:37.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:37.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:37.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:37.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:37.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:37.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T15:59:37.113+0000 7fd07ea85700 -1 osd.6 0 waiting for initial osdmap 2023-09-12T15:59:37.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T15:59:37.125+0000 7fd079077700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T15:59:37.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-12T15:59:37.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: osdmap e42: 7 total, 6 up, 7 in 2023-09-12T15:59:37.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T15:59:37.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:37.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:37.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:37.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:37.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:37.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:37.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:37.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-12T15:59:37.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: osdmap e42: 7 total, 6 up, 7 in 2023-09-12T15:59:37.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T15:59:37.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:37.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:37.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:37.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:37.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:37.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:37.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:37.505 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T15:59:38.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi130 2023-09-12T15:59:38.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi130 to 5722M 2023-09-12T15:59:38.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:38.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:38.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:38.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T15:59:38.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: osdmap e43: 7 total, 6 up, 7 in 2023-09-12T15:59:38.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:38.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:38.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:38 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:38.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi130 2023-09-12T15:59:38.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi130 to 5722M 2023-09-12T15:59:38.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T15:59:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: osdmap e43: 7 total, 6 up, 7 in 2023-09-12T15:59:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:38.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:38.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:38 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:38.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi130 2023-09-12T15:59:38.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi130 to 5722M 2023-09-12T15:59:38.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:38.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:38.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:38.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: from='osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T15:59:38.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: osdmap e43: 7 total, 6 up, 7 in 2023-09-12T15:59:38.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:38.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:38.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:38 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:38.744 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 -- 172.21.15.159:0/1072431316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 msgr2=0x7ff2b4100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:38.745 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/1072431316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 0x7ff2b4100f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009990 tx=0x7ff2b0009ca0 comp rx=0 tx=0).stop 2023-09-12T15:59:38.745 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 -- 172.21.15.159:0/1072431316 shutdown_connections 2023-09-12T15:59:38.745 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/1072431316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2b4100160 0x7ff2b4107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:38.745 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/1072431316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 0x7ff2b4100f60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:38.745 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/1072431316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b40ff7a0 0x7ff2b40ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:38.746 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 -- 172.21.15.159:0/1072431316 >> 172.21.15.159:0/1072431316 conn(0x7ff2b40fa2b0 msgr2=0x7ff2b40fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:59:38.746 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 -- 172.21.15.159:0/1072431316 shutdown_connections 2023-09-12T15:59:38.746 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 -- 172.21.15.159:0/1072431316 wait complete. 2023-09-12T15:59:38.746 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 Processor -- start 2023-09-12T15:59:38.746 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.743+0000 7ff2bbe43700 1 -- start start 2023-09-12T15:59:38.747 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2bbe43700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2b40ff7a0 0x7ff2b4195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:38.747 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2bbe43700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b4100160 0x7ff2b4195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:38.747 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2bbe43700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 0x7ff2b419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:38.747 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2bbe43700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419cd90 con 0x7ff2b40ff7a0 2023-09-12T15:59:38.747 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2bbe43700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419ced0 con 0x7ff2b4100b40 2023-09-12T15:59:38.747 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2bbe43700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419d060 con 0x7ff2b4100160 2023-09-12T15:59:38.748 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2ba3e0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 0x7ff2b419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:38.748 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2ba3e0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 0x7ff2b419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33232/0 (socket says 172.21.15.159:33232) 2023-09-12T15:59:38.748 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2ba3e0700 1 -- 172.21.15.159:0/3847018375 learned_addr learned my addr 172.21.15.159:0/3847018375 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-09-12T15:59:38.748 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2b9bdf700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2b40ff7a0 0x7ff2b4195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:38.748 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2b93de700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b4100160 0x7ff2b4195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:38.749 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2ba3e0700 1 -- 172.21.15.159:0/3847018375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b4100160 msgr2=0x7ff2b4195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:38.749 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2ba3e0700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b4100160 0x7ff2b4195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:38.749 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2ba3e0700 1 -- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2b40ff7a0 msgr2=0x7ff2b4195200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:38.749 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2ba3e0700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2b40ff7a0 0x7ff2b4195200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:38.749 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2ba3e0700 1 -- 172.21.15.159:0/3847018375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00095f0 con 0x7ff2b4100b40 2023-09-12T15:59:38.749 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2b93de700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b4100160 0x7ff2b4195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T15:59:38.750 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2ba3e0700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 0x7ff2b419c7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac007d70 tx=0x7ff2ac00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:59:38.750 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2a6ffd700 1 -- 172.21.15.159:0/3847018375 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac00ad40 con 0x7ff2b4100b40 2023-09-12T15:59:38.750 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b419d340 con 0x7ff2b4100b40 2023-09-12T15:59:38.750 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2a6ffd700 1 -- 172.21.15.159:0/3847018375 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ac0128e0 con 0x7ff2b4100b40 2023-09-12T15:59:38.750 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2a6ffd700 1 -- 172.21.15.159:0/3847018375 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac0116b0 con 0x7ff2b4100b40 2023-09-12T15:59:38.751 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.744+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b419d840 con 0x7ff2b4100b40 2023-09-12T15:59:38.752 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.745+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b410add0 con 0x7ff2b4100b40 2023-09-12T15:59:38.752 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.746+0000 7ff2a6ffd700 1 -- 172.21.15.159:0/3847018375 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7ff2ac012a50 con 0x7ff2b4100b40 2023-09-12T15:59:38.752 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.747+0000 7ff2a6ffd700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2a00a0640 0x7ff2a00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:38.752 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.747+0000 7ff2a6ffd700 1 -- 172.21.15.159:0/3847018375 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4965+0+0 (secure 0 0 0) 0x7ff2ac0cb0e0 con 0x7ff2b4100b40 2023-09-12T15:59:38.753 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.747+0000 7ff2b9bdf700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2a00a0640 0x7ff2a00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:38.753 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.748+0000 7ff2b9bdf700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2a00a0640 0x7ff2a00a2b00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000b5c0 tx=0x7ff2b0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:59:38.753 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.751+0000 7ff2a6ffd700 1 -- 172.21.15.159:0/3847018375 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff2ac095920 con 0x7ff2b4100b40 2023-09-12T15:59:38.947 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:38.947+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2b4060a40 con 0x7ff2a00a0640 2023-09-12T15:59:39.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T15:59:39.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T15:59:39.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:39.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] boot 2023-09-12T15:59:39.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: osdmap e44: 7 total, 7 up, 7 in 2023-09-12T15:59:39.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:39.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:39.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:59:39.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:59:39.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:59:39.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:39.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T15:59:39.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T15:59:39.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:39.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] boot 2023-09-12T15:59:39.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: osdmap e44: 7 total, 7 up, 7 in 2023-09-12T15:59:39.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:39.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:39.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:59:39.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:59:39.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:59:39.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:39.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T15:59:39.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T15:59:39.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:39.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: osd.6 [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] boot 2023-09-12T15:59:39.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: osdmap e44: 7 total, 7 up, 7 in 2023-09-12T15:59:39.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-12T15:59:39.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-12T15:59:39.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T15:59:39.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T15:59:39.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T15:59:39.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:40.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:40 smithi130 ceph-mon[98670]: osdmap e45: 7 total, 7 up, 7 in 2023-09-12T15:59:40.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:40 smithi159 ceph-mon[102522]: osdmap e45: 7 total, 7 up, 7 in 2023-09-12T15:59:40.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:40 smithi203 ceph-mon[103431]: osdmap e45: 7 total, 7 up, 7 in 2023-09-12T15:59:41.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:41 smithi130 ceph-mon[98670]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:41.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:41 smithi159 ceph-mon[102522]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:41.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:41 smithi203 ceph-mon[103431]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:42 smithi130 ceph-mon[98670]: from='client.? 172.21.15.159:0/3723831804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c5305d-d340-496a-8967-461ee63ee0c9"}]: dispatch 2023-09-12T15:59:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:42 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c5305d-d340-496a-8967-461ee63ee0c9"}]: dispatch 2023-09-12T15:59:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:42 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8c5305d-d340-496a-8967-461ee63ee0c9"}]': finished 2023-09-12T15:59:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:42 smithi130 ceph-mon[98670]: osdmap e46: 8 total, 7 up, 8 in 2023-09-12T15:59:42.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:42.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:42 smithi130 ceph-mon[98670]: from='client.? 172.21.15.159:0/2188882160' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:59:42.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:42 smithi159 ceph-mon[102522]: from='client.? 172.21.15.159:0/3723831804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c5305d-d340-496a-8967-461ee63ee0c9"}]: dispatch 2023-09-12T15:59:42.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:42 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c5305d-d340-496a-8967-461ee63ee0c9"}]: dispatch 2023-09-12T15:59:42.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:42 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8c5305d-d340-496a-8967-461ee63ee0c9"}]': finished 2023-09-12T15:59:42.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:42 smithi159 ceph-mon[102522]: osdmap e46: 8 total, 7 up, 8 in 2023-09-12T15:59:42.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:42.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:42 smithi159 ceph-mon[102522]: from='client.? 172.21.15.159:0/2188882160' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:59:42.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:42 smithi203 ceph-mon[103431]: from='client.? 172.21.15.159:0/3723831804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c5305d-d340-496a-8967-461ee63ee0c9"}]: dispatch 2023-09-12T15:59:42.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:42 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c5305d-d340-496a-8967-461ee63ee0c9"}]: dispatch 2023-09-12T15:59:42.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:42 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8c5305d-d340-496a-8967-461ee63ee0c9"}]': finished 2023-09-12T15:59:42.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:42 smithi203 ceph-mon[103431]: osdmap e46: 8 total, 7 up, 8 in 2023-09-12T15:59:42.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:42.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:42 smithi203 ceph-mon[103431]: from='client.? 172.21.15.159:0/2188882160' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T15:59:43.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:43 smithi130 ceph-mon[98670]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:43.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:43 smithi159 ceph-mon[102522]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:43.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:43 smithi203 ceph-mon[103431]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:46.064 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:45 smithi159 ceph-mon[102522]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:46.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:45 smithi130 ceph-mon[98670]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:45 smithi203 ceph-mon[103431]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:47.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:47 smithi130 ceph-mon[98670]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:47.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:47 smithi159 ceph-mon[102522]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:47.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:47 smithi203 ceph-mon[103431]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:48.658 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-12T15:59:48.659 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:48.659 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:48 smithi159 ceph-mon[102522]: Deploying daemon osd.7 on smithi159 2023-09-12T15:59:48.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-12T15:59:48.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:48.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:48 smithi130 ceph-mon[98670]: Deploying daemon osd.7 on smithi159 2023-09-12T15:59:48.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-12T15:59:48.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:48.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:48 smithi203 ceph-mon[103431]: Deploying daemon osd.7 on smithi159 2023-09-12T15:59:49.842 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:49 smithi159 ceph-mon[102522]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:49.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:49 smithi130 ceph-mon[98670]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:49.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:49 smithi203 ceph-mon[103431]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:51.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:51 smithi159 ceph-mon[102522]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:51 smithi130 ceph-mon[98670]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:51 smithi203 ceph-mon[103431]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:52.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:52.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:52.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:53.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:53.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:53.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:53.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:53.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:53.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:53.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:53 smithi159 ceph-mon[102522]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:53 smithi130 ceph-mon[98670]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:54.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:53 smithi203 ceph-mon[103431]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:54.553 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 7 on host 'smithi159' 2023-09-12T15:59:54.554 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.553+0000 7ff2a6ffd700 1 -- 172.21.15.159:0/3847018375 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff2b4060a40 con 0x7ff2a00a0640 2023-09-12T15:59:54.556 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2a00a0640 msgr2=0x7ff2a00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:54.556 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2a00a0640 0x7ff2a00a2b00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000b5c0 tx=0x7ff2b0005fb0 comp rx=0 tx=0).stop 2023-09-12T15:59:54.556 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 msgr2=0x7ff2b419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:54.557 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 0x7ff2b419c7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac007d70 tx=0x7ff2ac00a340 comp rx=0 tx=0).stop 2023-09-12T15:59:54.557 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 shutdown_connections 2023-09-12T15:59:54.557 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2a00a0640 0x7ff2a00a2b00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:54.557 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b4100b40 0x7ff2b419c7c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:54.557 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2b40ff7a0 0x7ff2b4195200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:54.558 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 --2- 172.21.15.159:0/3847018375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b4100160 0x7ff2b4195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:54.558 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 >> 172.21.15.159:0/3847018375 conn(0x7ff2b40fa2b0 msgr2=0x7ff2b40fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:59:54.558 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 shutdown_connections 2023-09-12T15:59:54.558 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T15:59:54.556+0000 7ff2bbe43700 1 -- 172.21.15.159:0/3847018375 wait complete. 2023-09-12T15:59:54.992 DEBUG:teuthology.orchestra.run.smithi159:osd.7> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.7.service 2023-09-12T15:59:54.994 INFO:tasks.cephadm:Deploying osd.8 on smithi203 with /dev/vg_nvme/lv_2... 2023-09-12T15:59:54.994 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-12T15:59:55.028 INFO:journalctl@ceph.osd.7.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T15:59:55.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:55.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:55.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:55.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:55 smithi130 ceph-mon[98670]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:55.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:55.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:55.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:55.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-mon[102522]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:55.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:55.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:55.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:55.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T15:59:55.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:55 smithi203 ceph-mon[103431]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:55.565 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T15:59:55.807 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 15:59:55 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T15:59:55.764+0000 7fd102629640 -1 osd.7 0 log_to_monitors true 2023-09-12T15:59:56.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:56 smithi130 ceph-mon[98670]: from='osd.7 [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-12T15:59:56.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:56 smithi130 ceph-mon[98670]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-12T15:59:56.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:56 smithi159 ceph-mon[102522]: from='osd.7 [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-12T15:59:56.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:56 smithi159 ceph-mon[102522]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-12T15:59:56.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:56 smithi203 ceph-mon[103431]: from='osd.7 [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-12T15:59:56.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:56 smithi203 ceph-mon[103431]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-12T15:59:57.231 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-12T15:59:57.232 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-12T15:59:57.232 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2023-09-12T15:59:57.232 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2023-09-12T15:59:57.233 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109982 s, 953 MB/s 2023-09-12T15:59:57.233 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2023-09-12T15:59:57.710 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi203:vg_nvme/lv_2 2023-09-12T15:59:57.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:57 smithi130 ceph-mon[98670]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-12T15:59:57.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:57 smithi130 ceph-mon[98670]: osdmap e47: 8 total, 7 up, 8 in 2023-09-12T15:59:57.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:57 smithi130 ceph-mon[98670]: from='osd.7 [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:59:57.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:57.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:57 smithi130 ceph-mon[98670]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:59:57.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:57 smithi130 ceph-mon[98670]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:57.976 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 15:59:57 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T15:59:57.560+0000 7fd0f7ca0700 -1 osd.7 0 waiting for initial osdmap 2023-09-12T15:59:57.976 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 15:59:57 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T15:59:57.570+0000 7fd0f3294700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T15:59:57.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:57 smithi159 ceph-mon[102522]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-12T15:59:57.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:57 smithi159 ceph-mon[102522]: osdmap e47: 8 total, 7 up, 8 in 2023-09-12T15:59:57.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:57 smithi159 ceph-mon[102522]: from='osd.7 [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:59:57.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:57.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:57 smithi159 ceph-mon[102522]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:59:57.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:57 smithi159 ceph-mon[102522]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:57.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:57 smithi203 ceph-mon[103431]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-12T15:59:57.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:57 smithi203 ceph-mon[103431]: osdmap e47: 8 total, 7 up, 8 in 2023-09-12T15:59:57.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:57 smithi203 ceph-mon[103431]: from='osd.7 [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:59:57.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:57.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:57 smithi203 ceph-mon[103431]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T15:59:57.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:57 smithi203 ceph-mon[103431]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:58.263 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T15:59:58.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T15:59:58.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: osdmap e48: 8 total, 7 up, 8 in 2023-09-12T15:59:58.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:58.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:58.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi159 2023-09-12T15:59:58.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:58.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:58.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:58.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi159 to 5722M 2023-09-12T15:59:58.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:58.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:58.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T15:59:58.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: osdmap e48: 8 total, 7 up, 8 in 2023-09-12T15:59:58.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:58.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:58.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi159 2023-09-12T15:59:58.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:58.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:58.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:58.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi159 to 5722M 2023-09-12T15:59:58.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:58.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:58.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.982 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T15:59:58.982 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: osdmap e48: 8 total, 7 up, 8 in 2023-09-12T15:59:58.982 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:58.983 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:58.983 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi159 2023-09-12T15:59:58.983 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.983 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.983 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:58.983 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:58.984 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T15:59:58.984 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi159 to 5722M 2023-09-12T15:59:58.984 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:58.984 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T15:59:58.984 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T15:59:58.985 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T15:59:59.594 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 -- 172.21.15.203:0/2361774724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 msgr2=0x7f9c60103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:59.594 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/2361774724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 0x7f9c60103510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009990 tx=0x7f9c5c009ca0 comp rx=0 tx=0).stop 2023-09-12T15:59:59.595 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 -- 172.21.15.203:0/2361774724 shutdown_connections 2023-09-12T15:59:59.595 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/2361774724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c60106380 0x7f9c601087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:59.595 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/2361774724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 0x7f9c60103510 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:59.595 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/2361774724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c60103a50 0x7f9c60105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:59.596 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 -- 172.21.15.203:0/2361774724 >> 172.21.15.203:0/2361774724 conn(0x7f9c600fab10 msgr2=0x7f9c600fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T15:59:59.596 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 -- 172.21.15.203:0/2361774724 shutdown_connections 2023-09-12T15:59:59.596 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 -- 172.21.15.203:0/2361774724 wait complete. 2023-09-12T15:59:59.596 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 Processor -- start 2023-09-12T15:59:59.596 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.592+0000 7f9c67a9c700 1 -- start start 2023-09-12T15:59:59.596 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c67a9c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 0x7f9c60196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:59.597 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c67a9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c60103a50 0x7f9c601968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:59.597 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c67a9c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c60106380 0x7f9c6019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:59.597 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c67a9c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6019def0 con 0x7f9c60106380 2023-09-12T15:59:59.597 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c67a9c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6019e030 con 0x7f9c60103a50 2023-09-12T15:59:59.597 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c67a9c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6019e1c0 con 0x7f9c60101120 2023-09-12T15:59:59.598 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65838700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 0x7f9c60196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:59.598 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65838700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 0x7f9c60196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:41600/0 (socket says 172.21.15.203:41600) 2023-09-12T15:59:59.598 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65838700 1 -- 172.21.15.203:0/1603040167 learned_addr learned my addr 172.21.15.203:0/1603040167 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-09-12T15:59:59.598 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c66039700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c60106380 0x7f9c6019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:59.598 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65037700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c60103a50 0x7f9c601968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:59.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65838700 1 -- 172.21.15.203:0/1603040167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c60103a50 msgr2=0x7f9c601968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:59.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65838700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c60103a50 0x7f9c601968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:59.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65838700 1 -- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c60106380 msgr2=0x7f9c6019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T15:59:59.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65838700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c60106380 0x7f9c6019d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T15:59:59.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65838700 1 -- 172.21.15.203:0/1603040167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0095f0 con 0x7f9c60101120 2023-09-12T15:59:59.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c65838700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 0x7f9c60196360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c005910 tx=0x7f9c5c00f780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:59:59.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c56ffd700 1 -- 172.21.15.203:0/1603040167 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c01d070 con 0x7f9c60101120 2023-09-12T15:59:59.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c56ffd700 1 -- 172.21.15.203:0/1603040167 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c5c00fc60 con 0x7f9c60101120 2023-09-12T15:59:59.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c56ffd700 1 -- 172.21.15.203:0/1603040167 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c017650 con 0x7f9c60101120 2023-09-12T15:59:59.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6019e440 con 0x7f9c60101120 2023-09-12T15:59:59.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.593+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6019e8e0 con 0x7f9c60101120 2023-09-12T15:59:59.602 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.594+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c600fc690 con 0x7f9c60101120 2023-09-12T15:59:59.602 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.595+0000 7f9c56ffd700 1 -- 172.21.15.203:0/1603040167 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f9c5c022470 con 0x7f9c60101120 2023-09-12T15:59:59.602 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.596+0000 7f9c56ffd700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c4c0a06c0 0x7f9c4c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T15:59:59.602 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.596+0000 7f9c56ffd700 1 -- 172.21.15.203:0/1603040167 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f9c5c0cc3f0 con 0x7f9c60101120 2023-09-12T15:59:59.602 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.596+0000 7f9c65037700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c4c0a06c0 0x7f9c4c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T15:59:59.603 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.596+0000 7f9c65037700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c4c0a06c0 0x7f9c4c0a2b80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9c50005fd0 tx=0x7f9c5000a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T15:59:59.603 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.600+0000 7f9c56ffd700 1 -- 172.21.15.203:0/1603040167 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9c5c05ce10 con 0x7f9c60101120 2023-09-12T15:59:59.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:59 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T15:59:59.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:59 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T15:59:59.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:59 smithi203 ceph-mon[103431]: osd.7 [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] boot 2023-09-12T15:59:59.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:59 smithi203 ceph-mon[103431]: osdmap e49: 8 total, 8 up, 8 in 2023-09-12T15:59:59.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:59.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:59 smithi203 ceph-mon[103431]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:59.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 15:59:59 smithi203 ceph-mon[103431]: osdmap e50: 8 total, 8 up, 8 in 2023-09-12T15:59:59.800 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T15:59:59.800+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c6002cfa0 con 0x7f9c4c0a06c0 2023-09-12T15:59:59.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:59 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T15:59:59.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:59 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T15:59:59.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:59 smithi130 ceph-mon[98670]: osd.7 [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] boot 2023-09-12T15:59:59.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:59 smithi130 ceph-mon[98670]: osdmap e49: 8 total, 8 up, 8 in 2023-09-12T15:59:59.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:59.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:59 smithi130 ceph-mon[98670]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:59.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 15:59:59 smithi130 ceph-mon[98670]: osdmap e50: 8 total, 8 up, 8 in 2023-09-12T15:59:59.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:59 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T15:59:59.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:59 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T15:59:59.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:59 smithi159 ceph-mon[102522]: osd.7 [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] boot 2023-09-12T15:59:59.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:59 smithi159 ceph-mon[102522]: osdmap e49: 8 total, 8 up, 8 in 2023-09-12T15:59:59.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-12T15:59:59.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:59 smithi159 ceph-mon[102522]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-12T15:59:59.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 15:59:59 smithi159 ceph-mon[102522]: osdmap e50: 8 total, 8 up, 8 in 2023-09-12T16:00:00.940 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:00 smithi203 ceph-mon[103431]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:00:00.940 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:00:00.941 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:00:00.941 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:00.941 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:00 smithi203 ceph-mon[103431]: overall HEALTH_OK 2023-09-12T16:00:00.941 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:00 smithi203 ceph-mon[103431]: osdmap e51: 8 total, 8 up, 8 in 2023-09-12T16:00:00.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:00 smithi130 ceph-mon[98670]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:00:00.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:00:00.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:00:00.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:00.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:00 smithi130 ceph-mon[98670]: overall HEALTH_OK 2023-09-12T16:00:00.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:00 smithi130 ceph-mon[98670]: osdmap e51: 8 total, 8 up, 8 in 2023-09-12T16:00:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:00 smithi159 ceph-mon[102522]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:00:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:00:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:00:00.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:00.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:00 smithi159 ceph-mon[102522]: overall HEALTH_OK 2023-09-12T16:00:00.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:00 smithi159 ceph-mon[102522]: osdmap e51: 8 total, 8 up, 8 in 2023-09-12T16:00:01.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:01 smithi130 ceph-mon[98670]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:01.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:01 smithi130 ceph-mon[98670]: osdmap e52: 8 total, 8 up, 8 in 2023-09-12T16:00:01.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:01 smithi159 ceph-mon[102522]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:01.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:01 smithi159 ceph-mon[102522]: osdmap e52: 8 total, 8 up, 8 in 2023-09-12T16:00:01.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:01 smithi203 ceph-mon[103431]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:01.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:01 smithi203 ceph-mon[103431]: osdmap e52: 8 total, 8 up, 8 in 2023-09-12T16:00:02.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:02 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/3111003657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52e84740-7c08-4017-b5d4-3a20f0ecad18"}]: dispatch 2023-09-12T16:00:02.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:02 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52e84740-7c08-4017-b5d4-3a20f0ecad18"}]: dispatch 2023-09-12T16:00:02.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:02 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52e84740-7c08-4017-b5d4-3a20f0ecad18"}]': finished 2023-09-12T16:00:02.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:02 smithi130 ceph-mon[98670]: osdmap e53: 9 total, 8 up, 9 in 2023-09-12T16:00:02.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:02 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:02.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:02 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/3111003657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52e84740-7c08-4017-b5d4-3a20f0ecad18"}]: dispatch 2023-09-12T16:00:02.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:02 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52e84740-7c08-4017-b5d4-3a20f0ecad18"}]: dispatch 2023-09-12T16:00:02.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:02 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52e84740-7c08-4017-b5d4-3a20f0ecad18"}]': finished 2023-09-12T16:00:02.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:02 smithi159 ceph-mon[102522]: osdmap e53: 9 total, 8 up, 9 in 2023-09-12T16:00:02.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:02 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:02.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:02 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/3111003657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52e84740-7c08-4017-b5d4-3a20f0ecad18"}]: dispatch 2023-09-12T16:00:02.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:02 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52e84740-7c08-4017-b5d4-3a20f0ecad18"}]: dispatch 2023-09-12T16:00:02.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:02 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52e84740-7c08-4017-b5d4-3a20f0ecad18"}]': finished 2023-09-12T16:00:02.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:02 smithi203 ceph-mon[103431]: osdmap e53: 9 total, 8 up, 9 in 2023-09-12T16:00:02.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:02 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:03.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:03 smithi130 ceph-mon[98670]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:03.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:03 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/1226721072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:00:03.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:03 smithi159 ceph-mon[102522]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:03.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:03 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/1226721072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:00:03.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:03 smithi203 ceph-mon[103431]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:03.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:03 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/1226721072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:00:06.114 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:05 smithi203 ceph-mon[103431]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:06.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:05 smithi130 ceph-mon[98670]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:06.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:05 smithi159 ceph-mon[102522]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:07.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:07 smithi203 ceph-mon[103431]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:07.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:07 smithi130 ceph-mon[98670]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:07.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:07 smithi159 ceph-mon[102522]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:09.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:09 smithi203 ceph-mon[103431]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:09.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-12T16:00:09.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:09.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:09 smithi203 ceph-mon[103431]: Deploying daemon osd.8 on smithi203 2023-09-12T16:00:10.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:09 smithi130 ceph-mon[98670]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:10.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-12T16:00:10.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:10.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:09 smithi130 ceph-mon[98670]: Deploying daemon osd.8 on smithi203 2023-09-12T16:00:10.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:09 smithi159 ceph-mon[102522]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:10.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-12T16:00:10.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:10.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:09 smithi159 ceph-mon[102522]: Deploying daemon osd.8 on smithi203 2023-09-12T16:00:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:11 smithi203 ceph-mon[103431]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:12.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:11 smithi130 ceph-mon[98670]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:12.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:11 smithi159 ceph-mon[102522]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:13.958 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:13 smithi203 ceph-mon[103431]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:13.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:13.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:13.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:14.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:13 smithi130 ceph-mon[98670]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:14.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:13 smithi159 ceph-mon[102522]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:15.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:15 smithi203 ceph-mon[103431]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:16.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:15 smithi130 ceph-mon[98670]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:16.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:15 smithi159 ceph-mon[102522]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:16.511 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.510+0000 7f9c56ffd700 1 -- 172.21.15.203:0/1603040167 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9c6002cfa0 con 0x7f9c4c0a06c0 2023-09-12T16:00:16.511 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 8 on host 'smithi203' 2023-09-12T16:00:16.513 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c4c0a06c0 msgr2=0x7f9c4c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:16.513 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c4c0a06c0 0x7f9c4c0a2b80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9c50005fd0 tx=0x7f9c5000a560 comp rx=0 tx=0).stop 2023-09-12T16:00:16.513 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 msgr2=0x7f9c60196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:16.514 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 0x7f9c60196360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c005910 tx=0x7f9c5c00f780 comp rx=0 tx=0).stop 2023-09-12T16:00:16.514 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 shutdown_connections 2023-09-12T16:00:16.514 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c4c0a06c0 0x7f9c4c0a2b80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:16.514 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c60106380 0x7f9c6019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:16.514 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c60101120 0x7f9c60196360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:16.515 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 --2- 172.21.15.203:0/1603040167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c60103a50 0x7f9c601968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:16.515 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 >> 172.21.15.203:0/1603040167 conn(0x7f9c600fab10 msgr2=0x7f9c60101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:00:16.515 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 shutdown_connections 2023-09-12T16:00:16.515 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:00:16.512+0000 7f9c67a9c700 1 -- 172.21.15.203:0/1603040167 wait complete. 2023-09-12T16:00:16.956 DEBUG:teuthology.orchestra.run.smithi203:osd.8> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.8.service 2023-09-12T16:00:16.959 INFO:tasks.cephadm:Deploying osd.9 on smithi130 with /dev/vg_nvme/lv_1... 2023-09-12T16:00:16.959 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-12T16:00:16.991 INFO:journalctl@ceph.osd.8.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T16:00:17.531 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:00:17.724 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:17 smithi130 ceph-mon[98670]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:17.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:17 smithi159 ceph-mon[102522]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:17.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:17.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:17 smithi203 ceph-mon[103431]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:17.747 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:00:17 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:00:17.483+0000 7fd38563d640 -1 osd.8 0 log_to_monitors true 2023-09-12T16:00:18.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:18 smithi203 ceph-mon[103431]: from='osd.8 [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-12T16:00:18.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:18 smithi203 ceph-mon[103431]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-12T16:00:18.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:18 smithi130 ceph-mon[98670]: from='osd.8 [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-12T16:00:18.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:18 smithi130 ceph-mon[98670]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-12T16:00:18.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:18 smithi159 ceph-mon[102522]: from='osd.8 [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-12T16:00:18.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:18 smithi159 ceph-mon[102522]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-12T16:00:18.976 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-12T16:00:18.976 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-12T16:00:18.976 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2023-09-12T16:00:18.976 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2023-09-12T16:00:18.977 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0143132 s, 733 MB/s 2023-09-12T16:00:18.977 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2023-09-12T16:00:19.291 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi130:vg_nvme/lv_1 2023-09-12T16:00:19.724 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:19 smithi130 ceph-mon[98670]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-12T16:00:19.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:19 smithi130 ceph-mon[98670]: osdmap e54: 9 total, 8 up, 9 in 2023-09-12T16:00:19.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:19.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:19 smithi130 ceph-mon[98670]: from='osd.8 [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:00:19.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:19 smithi130 ceph-mon[98670]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:00:19.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:19 smithi130 ceph-mon[98670]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:19.745 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:00:19 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:00:19.488+0000 7fd37acb4700 -1 osd.8 0 waiting for initial osdmap 2023-09-12T16:00:19.746 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:00:19 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:00:19.498+0000 7fd376aa9700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T16:00:19.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:19 smithi203 ceph-mon[103431]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-12T16:00:19.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:19 smithi203 ceph-mon[103431]: osdmap e54: 9 total, 8 up, 9 in 2023-09-12T16:00:19.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:19.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:19 smithi203 ceph-mon[103431]: from='osd.8 [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:00:19.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:19 smithi203 ceph-mon[103431]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:00:19.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:19 smithi203 ceph-mon[103431]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:19.854 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:00:19.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:19 smithi159 ceph-mon[102522]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-12T16:00:19.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:19 smithi159 ceph-mon[102522]: osdmap e54: 9 total, 8 up, 9 in 2023-09-12T16:00:19.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:19.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:19 smithi159 ceph-mon[102522]: from='osd.8 [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:00:19.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:19 smithi159 ceph-mon[102522]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:00:19.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:19 smithi159 ceph-mon[102522]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-12T16:00:20.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T16:00:20.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: osdmap e55: 9 total, 8 up, 9 in 2023-09-12T16:00:20.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:20.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:20.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi203 2023-09-12T16:00:20.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:20.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:20.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:20.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi203 to 5722M 2023-09-12T16:00:20.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:20.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:20.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:20.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:20.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:20.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: osd.8 [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] boot 2023-09-12T16:00:20.732 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: osdmap e56: 9 total, 9 up, 9 in 2023-09-12T16:00:20.732 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:20.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T16:00:20.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: osdmap e55: 9 total, 8 up, 9 in 2023-09-12T16:00:20.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:20.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:20.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi203 2023-09-12T16:00:20.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:20.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:20.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:20.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi203 to 5722M 2023-09-12T16:00:20.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:20.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:20.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:20.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:20.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:20.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: osd.8 [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] boot 2023-09-12T16:00:20.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: osdmap e56: 9 total, 9 up, 9 in 2023-09-12T16:00:20.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:20.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T16:00:20.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: osdmap e55: 9 total, 8 up, 9 in 2023-09-12T16:00:20.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:20.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:20.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi203 2023-09-12T16:00:20.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:20.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:20.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:20.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi203 to 5722M 2023-09-12T16:00:20.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:20.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:20.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:20.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:20.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:20.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:20.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: osd.8 [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] boot 2023-09-12T16:00:20.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: osdmap e56: 9 total, 9 up, 9 in 2023-09-12T16:00:20.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-12T16:00:21.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.167+0000 7fabd5e24700 1 -- 172.21.15.130:0/615013161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0100790 msgr2=0x7fabd0100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:21.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.167+0000 7fabd5e24700 1 --2- 172.21.15.130:0/615013161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0100790 0x7fabd0100bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fabc00099c0 tx=0x7fabc0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:00:21.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.167+0000 7fabd5e24700 1 -- 172.21.15.130:0/615013161 shutdown_connections 2023-09-12T16:00:21.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.167+0000 7fabd5e24700 1 --2- 172.21.15.130:0/615013161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabd0101130 0x7fabd01079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:21.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.167+0000 7fabd5e24700 1 --2- 172.21.15.130:0/615013161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabd00ff590 0x7fabd00ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:21.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.167+0000 7fabd5e24700 1 --2- 172.21.15.130:0/615013161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0100790 0x7fabd0100bf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:21.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.167+0000 7fabd5e24700 1 -- 172.21.15.130:0/615013161 >> 172.21.15.130:0/615013161 conn(0x7fabd00fab10 msgr2=0x7fabd00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:00:21.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.167+0000 7fabd5e24700 1 -- 172.21.15.130:0/615013161 shutdown_connections 2023-09-12T16:00:21.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.167+0000 7fabd5e24700 1 -- 172.21.15.130:0/615013161 wait complete. 2023-09-12T16:00:21.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabd5e24700 1 Processor -- start 2023-09-12T16:00:21.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabd5e24700 1 -- start start 2023-09-12T16:00:21.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabd5e24700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabd00ff590 0x7fabd0196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:00:21.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabd5e24700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabd0100790 0x7fabd0196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:00:21.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabd5e24700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0101130 0x7fabd019d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:00:21.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabd5e24700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019deb0 con 0x7fabd0101130 2023-09-12T16:00:21.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabd5e24700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019dff0 con 0x7fabd00ff590 2023-09-12T16:00:21.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabd5e24700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019e180 con 0x7fabd0100790 2023-09-12T16:00:21.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabcffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0101130 0x7fabd019d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:00:21.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabcffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0101130 0x7fabd019d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52728/0 (socket says 172.21.15.130:52728) 2023-09-12T16:00:21.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.168+0000 7fabcffff700 1 -- 172.21.15.130:0/4018248875 learned_addr learned my addr 172.21.15.130:0/4018248875 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:00:21.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabcf7fe700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabd00ff590 0x7fabd0196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:00:21.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabceffd700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabd0100790 0x7fabd0196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:00:21.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabcffff700 1 -- 172.21.15.130:0/4018248875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabd0100790 msgr2=0x7fabd0196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:21.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabcffff700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabd0100790 0x7fabd0196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:21.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabcffff700 1 -- 172.21.15.130:0/4018248875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabd00ff590 msgr2=0x7fabd0196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:21.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabcffff700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabd00ff590 0x7fabd0196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:21.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabcffff700 1 -- 172.21.15.130:0/4018248875 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc00095f0 con 0x7fabd0101130 2023-09-12T16:00:21.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabceffd700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabd0100790 0x7fabd0196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:00:21.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabcf7fe700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabd00ff590 0x7fabd0196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:00:21.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabcffff700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0101130 0x7fabd019d8e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabc8007da0 tx=0x7fabc800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:00:21.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabccff9700 1 -- 172.21.15.130:0/4018248875 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc800ad00 con 0x7fabd0101130 2023-09-12T16:00:21.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabccff9700 1 -- 172.21.15.130:0/4018248875 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabc800ae60 con 0x7fabd0101130 2023-09-12T16:00:21.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd019e460 con 0x7fabd0101130 2023-09-12T16:00:21.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabccff9700 1 -- 172.21.15.130:0/4018248875 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8011780 con 0x7fabd0101130 2023-09-12T16:00:21.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.169+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd019e960 con 0x7fabd0101130 2023-09-12T16:00:21.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.170+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd010ae30 con 0x7fabd0101130 2023-09-12T16:00:21.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.172+0000 7fabccff9700 1 -- 172.21.15.130:0/4018248875 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fabc8012a70 con 0x7fabd0101130 2023-09-12T16:00:21.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.172+0000 7fabccff9700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fabbc0a06c0 0x7fabbc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:00:21.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.172+0000 7fabcf7fe700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fabbc0a06c0 0x7fabbc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:00:21.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.172+0000 7fabccff9700 1 -- 172.21.15.130:0/4018248875 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fabc80cb8e0 con 0x7fabd0101130 2023-09-12T16:00:21.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.173+0000 7fabcf7fe700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fabbc0a06c0 0x7fabbc0a2b80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fabd01005f0 tx=0x7fabb8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:00:21.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.175+0000 7fabccff9700 1 -- 172.21.15.130:0/4018248875 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fabc8095da0 con 0x7fabd0101130 2023-09-12T16:00:21.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:21.374+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fabd0068850 con 0x7fabbc0a06c0 2023-09-12T16:00:21.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:21 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T16:00:21.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:21 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T16:00:21.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:21 smithi203 ceph-mon[103431]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:21.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:21 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:00:21.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:21 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:00:21.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:21 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:21.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:21 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T16:00:21.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:21 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T16:00:21.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:21 smithi130 ceph-mon[98670]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:21.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:00:21.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:00:21.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:21.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:21 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T16:00:21.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:21 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T16:00:21.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:21 smithi159 ceph-mon[102522]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:21.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:21 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:00:21.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:21 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:00:21.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:21 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:22.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:22 smithi130 ceph-mon[98670]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:00:22.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:22 smithi130 ceph-mon[98670]: osdmap e57: 9 total, 9 up, 9 in 2023-09-12T16:00:22.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:22 smithi203 ceph-mon[103431]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:00:22.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:22 smithi203 ceph-mon[103431]: osdmap e57: 9 total, 9 up, 9 in 2023-09-12T16:00:22.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:22 smithi159 ceph-mon[102522]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:00:22.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:22 smithi159 ceph-mon[102522]: osdmap e57: 9 total, 9 up, 9 in 2023-09-12T16:00:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:23 smithi130 ceph-mon[98670]: osdmap e58: 9 total, 9 up, 9 in 2023-09-12T16:00:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:23 smithi130 ceph-mon[98670]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:23.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:23 smithi159 ceph-mon[102522]: osdmap e58: 9 total, 9 up, 9 in 2023-09-12T16:00:23.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:23 smithi159 ceph-mon[102522]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:23.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:23 smithi203 ceph-mon[103431]: osdmap e58: 9 total, 9 up, 9 in 2023-09-12T16:00:23.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:23 smithi203 ceph-mon[103431]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:24.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:24 smithi130 ceph-mon[98670]: osdmap e59: 9 total, 9 up, 9 in 2023-09-12T16:00:24.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:24 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2109050802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c95841a-2cba-411e-b7e6-f958256adc5c"}]: dispatch 2023-09-12T16:00:24.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:24 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2109050802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c95841a-2cba-411e-b7e6-f958256adc5c"}]': finished 2023-09-12T16:00:24.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:24 smithi130 ceph-mon[98670]: osdmap e60: 10 total, 9 up, 10 in 2023-09-12T16:00:24.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:24 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:24.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:24 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3117863533' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:00:24.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:24 smithi159 ceph-mon[102522]: osdmap e59: 9 total, 9 up, 9 in 2023-09-12T16:00:24.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:24 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2109050802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c95841a-2cba-411e-b7e6-f958256adc5c"}]: dispatch 2023-09-12T16:00:24.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:24 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2109050802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c95841a-2cba-411e-b7e6-f958256adc5c"}]': finished 2023-09-12T16:00:24.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:24 smithi159 ceph-mon[102522]: osdmap e60: 10 total, 9 up, 10 in 2023-09-12T16:00:24.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:24 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:24.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:24 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3117863533' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:00:24.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:24 smithi203 ceph-mon[103431]: osdmap e59: 9 total, 9 up, 9 in 2023-09-12T16:00:24.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:24 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2109050802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c95841a-2cba-411e-b7e6-f958256adc5c"}]: dispatch 2023-09-12T16:00:24.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:24 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2109050802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c95841a-2cba-411e-b7e6-f958256adc5c"}]': finished 2023-09-12T16:00:24.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:24 smithi203 ceph-mon[103431]: osdmap e60: 10 total, 9 up, 10 in 2023-09-12T16:00:24.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:24 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:24.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:24 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3117863533' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:00:25.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:25 smithi130 ceph-mon[98670]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-12T16:00:25.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:25 smithi159 ceph-mon[102522]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-12T16:00:25.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:25 smithi203 ceph-mon[103431]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-12T16:00:27.634 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:27 smithi130 ceph-mon[98670]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-09-12T16:00:27.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:27 smithi159 ceph-mon[102522]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-09-12T16:00:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:27 smithi203 ceph-mon[103431]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-09-12T16:00:29.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:29 smithi130 ceph-mon[98670]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-09-12T16:00:30.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:29 smithi159 ceph-mon[102522]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-09-12T16:00:30.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:29 smithi203 ceph-mon[103431]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-09-12T16:00:30.953 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:30 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-12T16:00:30.954 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:30 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:31.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:30 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-12T16:00:31.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:30 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:31.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:30 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-12T16:00:31.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:30 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:32.171 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:31 smithi130 ceph-mon[98670]: Deploying daemon osd.9 on smithi130 2023-09-12T16:00:32.171 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:31 smithi130 ceph-mon[98670]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-12T16:00:32.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:31 smithi159 ceph-mon[102522]: Deploying daemon osd.9 on smithi130 2023-09-12T16:00:32.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:31 smithi159 ceph-mon[102522]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-12T16:00:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:31 smithi203 ceph-mon[103431]: Deploying daemon osd.9 on smithi130 2023-09-12T16:00:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:31 smithi203 ceph-mon[103431]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-12T16:00:34.149 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:33 smithi130 ceph-mon[98670]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-09-12T16:00:34.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:33 smithi159 ceph-mon[102522]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-09-12T16:00:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:33 smithi203 ceph-mon[103431]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-09-12T16:00:35.350 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:35.350 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:35.350 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:35.351 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:35 smithi130 ceph-mon[98670]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-09-12T16:00:35.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:35.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:35.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:35.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:35 smithi159 ceph-mon[102522]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-09-12T16:00:35.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:35.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:35.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:35.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:35 smithi203 ceph-mon[103431]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-09-12T16:00:36.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.777+0000 7fabccff9700 1 -- 172.21.15.130:0/4018248875 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fabd0068850 con 0x7fabbc0a06c0 2023-09-12T16:00:36.779 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 9 on host 'smithi130' 2023-09-12T16:00:36.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.779+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fabbc0a06c0 msgr2=0x7fabbc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:36.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.779+0000 7fabd5e24700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fabbc0a06c0 0x7fabbc0a2b80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fabd01005f0 tx=0x7fabb8008040 comp rx=0 tx=0).stop 2023-09-12T16:00:36.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.779+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0101130 msgr2=0x7fabd019d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:36.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.779+0000 7fabd5e24700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0101130 0x7fabd019d8e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabc8007da0 tx=0x7fabc800a340 comp rx=0 tx=0).stop 2023-09-12T16:00:36.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.780+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 shutdown_connections 2023-09-12T16:00:36.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.780+0000 7fabd5e24700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fabbc0a06c0 0x7fabbc0a2b80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:36.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.780+0000 7fabd5e24700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabd0101130 0x7fabd019d8e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:36.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.780+0000 7fabd5e24700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabd00ff590 0x7fabd0196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:36.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.780+0000 7fabd5e24700 1 --2- 172.21.15.130:0/4018248875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabd0100790 0x7fabd0196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:36.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.780+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 >> 172.21.15.130:0/4018248875 conn(0x7fabd00fab10 msgr2=0x7fabd0068140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:00:36.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.780+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 shutdown_connections 2023-09-12T16:00:36.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:00:36.780+0000 7fabd5e24700 1 -- 172.21.15.130:0/4018248875 wait complete. 2023-09-12T16:00:37.130 DEBUG:teuthology.orchestra.run.smithi130:osd.9> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.9.service 2023-09-12T16:00:37.133 INFO:tasks.cephadm:Deploying osd.10 on smithi159 with /dev/vg_nvme/lv_1... 2023-09-12T16:00:37.133 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-12T16:00:37.162 INFO:journalctl@ceph.osd.9.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T16:00:37.699 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T16:00:37.941 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:37 smithi130 ceph-mon[98670]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:37.942 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:37.942 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:37.942 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:37.942 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:38.108 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:37 smithi159 ceph-mon[102522]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:38.109 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:38.109 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:38.109 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:38.109 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:38.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:37 smithi203 ceph-mon[103431]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:38.486 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:00:38 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:00:38.277+0000 7fc592f69640 -1 osd.9 0 log_to_monitors true 2023-09-12T16:00:39.181 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-12T16:00:39.181 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-12T16:00:39.182 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-09-12T16:00:39.182 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-09-12T16:00:39.182 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110628 s, 948 MB/s 2023-09-12T16:00:39.183 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-09-12T16:00:39.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:38 smithi159 ceph-mon[102522]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-12T16:00:39.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:38 smithi130 ceph-mon[98670]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-12T16:00:39.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:38 smithi203 ceph-mon[103431]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-12T16:00:39.555 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2023-09-12T16:00:39.975 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:00:39.769+0000 7fc5885e0700 -1 osd.9 0 waiting for initial osdmap 2023-09-12T16:00:39.975 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:00:39.780+0000 7fc581bd0700 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T16:00:39.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:39.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-12T16:00:39.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: osdmap e61: 10 total, 9 up, 10 in 2023-09-12T16:00:39.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T16:00:39.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:39.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:39.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:39.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:39.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:39.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:39.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:39.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:39.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:39.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:39.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:39.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:39.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-12T16:00:39.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: osdmap e61: 10 total, 9 up, 10 in 2023-09-12T16:00:39.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T16:00:39.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:39.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:39.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:39.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:39.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:39.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:39.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:39.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:39.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:39.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:39.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:40.112 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T16:00:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:40.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-12T16:00:40.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: osdmap e61: 10 total, 9 up, 10 in 2023-09-12T16:00:40.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2023-09-12T16:00:40.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:40.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:40.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:40.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:40.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:40.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:40.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:00:40.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:40.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:40.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:40.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:40.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi130 2023-09-12T16:00:40.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi130 to 4292M 2023-09-12T16:00:40.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T16:00:40.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: osdmap e62: 10 total, 9 up, 10 in 2023-09-12T16:00:40.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:40.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:40.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:40.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:40.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:40.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:41.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi130 2023-09-12T16:00:41.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi130 to 4292M 2023-09-12T16:00:41.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T16:00:41.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: osdmap e62: 10 total, 9 up, 10 in 2023-09-12T16:00:41.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:41.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:41.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:41.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:41.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:41.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:41.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi130 2023-09-12T16:00:41.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi130 to 4292M 2023-09-12T16:00:41.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: from='osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2023-09-12T16:00:41.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: osdmap e62: 10 total, 9 up, 10 in 2023-09-12T16:00:41.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:41.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:41.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:41.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:41.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:41.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:41.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/33180176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4101160 msgr2=0x7f4bf4107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:41.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/33180176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4101160 0x7f4bf4107a10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00b0d0 tx=0x7f4bec00b3e0 comp rx=0 tx=0).stop 2023-09-12T16:00:41.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/33180176 shutdown_connections 2023-09-12T16:00:41.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/33180176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4101160 0x7f4bf4107a10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:41.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/33180176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4bf40ff5c0 0x7f4bf40ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:41.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/33180176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf41007c0 0x7f4bf4100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:41.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/33180176 >> 172.21.15.159:0/33180176 conn(0x7f4bf40fab80 msgr2=0x7f4bf40fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:00:41.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/33180176 shutdown_connections 2023-09-12T16:00:41.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/33180176 wait complete. 2023-09-12T16:00:41.311 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 Processor -- start 2023-09-12T16:00:41.311 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.309+0000 7f4bfb8b8700 1 -- start start 2023-09-12T16:00:41.313 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bfb8b8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf40ff5c0 0x7f4bf41037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:00:41.313 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bfb8b8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4bf41007c0 0x7f4bf4101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:00:41.313 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bfb8b8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf4101160 0x7f4bf4102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:00:41.314 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bfb8b8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf4102af0 con 0x7f4bf41007c0 2023-09-12T16:00:41.314 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bfb8b8700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf4102c60 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.314 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bfb8b8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf41a04e0 con 0x7f4bf4101160 2023-09-12T16:00:41.314 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9654700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf40ff5c0 0x7f4bf41037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:00:41.314 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9654700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf40ff5c0 0x7f4bf41037e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33384/0 (socket says 172.21.15.159:33384) 2023-09-12T16:00:41.315 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9654700 1 -- 172.21.15.159:0/1715713379 learned_addr learned my addr 172.21.15.159:0/1715713379 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-09-12T16:00:41.315 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf8e53700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4bf41007c0 0x7f4bf4101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:00:41.315 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9e55700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf4101160 0x7f4bf4102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:00:41.315 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9654700 1 -- 172.21.15.159:0/1715713379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf4101160 msgr2=0x7f4bf4102370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:41.315 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9654700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf4101160 0x7f4bf4102370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:41.316 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9654700 1 -- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4bf41007c0 msgr2=0x7f4bf4101e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:41.316 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9654700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4bf41007c0 0x7f4bf4101e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:41.316 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9654700 1 -- 172.21.15.159:0/1715713379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec009d00 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.316 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9e55700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf4101160 0x7f4bf4102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:00:41.316 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bf9654700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf40ff5c0 0x7f4bf41037e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000d980 tx=0x7f4bf000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:00:41.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4be67fc700 1 -- 172.21.15.159:0/1715713379 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0004020 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4be67fc700 1 -- 172.21.15.159:0/1715713379 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bf0009730 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf41a07c0 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4be67fc700 1 -- 172.21.15.159:0/1715713379 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0003a80 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.310+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf41a0cc0 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.311+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf4107630 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.318 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.312+0000 7f4be67fc700 1 -- 172.21.15.159:0/1715713379 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f4bf00098a0 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.318 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.313+0000 7f4be67fc700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4be00a0640 0x7f4be00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:00:41.318 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.313+0000 7f4be67fc700 1 -- 172.21.15.159:0/1715713379 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f4bf00cb0c0 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.318 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.313+0000 7f4bf8e53700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4be00a0640 0x7f4be00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:00:41.318 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.314+0000 7f4bf8e53700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4be00a0640 0x7f4be00a2b00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005950 tx=0x7f4be800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:00:41.319 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.317+0000 7f4be67fc700 1 -- 172.21.15.159:0/1715713379 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4bf00953f0 con 0x7f4bf40ff5c0 2023-09-12T16:00:41.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:41.515+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bf402cf80 con 0x7f4be00a0640 2023-09-12T16:00:42.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:41 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T16:00:42.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:41 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T16:00:42.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:41 smithi130 ceph-mon[98670]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:42.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:41 smithi130 ceph-mon[98670]: osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] boot 2023-09-12T16:00:42.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:41 smithi130 ceph-mon[98670]: osdmap e63: 10 total, 10 up, 10 in 2023-09-12T16:00:42.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:42.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:00:42.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:00:42.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:42.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:41 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T16:00:42.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:41 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T16:00:42.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:41 smithi159 ceph-mon[102522]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:42.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:41 smithi159 ceph-mon[102522]: osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] boot 2023-09-12T16:00:42.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:41 smithi159 ceph-mon[102522]: osdmap e63: 10 total, 10 up, 10 in 2023-09-12T16:00:42.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:42.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:00:42.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:00:42.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:41 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T16:00:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:41 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T16:00:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:41 smithi203 ceph-mon[103431]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-12T16:00:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:41 smithi203 ceph-mon[103431]: osd.9 [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] boot 2023-09-12T16:00:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:41 smithi203 ceph-mon[103431]: osdmap e63: 10 total, 10 up, 10 in 2023-09-12T16:00:42.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-12T16:00:42.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:00:42.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:00:42.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:42.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:42 smithi159 ceph-mon[102522]: from='client.24422 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:00:42.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:42 smithi159 ceph-mon[102522]: osdmap e64: 10 total, 10 up, 10 in 2023-09-12T16:00:43.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:42 smithi130 ceph-mon[98670]: from='client.24422 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:00:43.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:42 smithi130 ceph-mon[98670]: osdmap e64: 10 total, 10 up, 10 in 2023-09-12T16:00:43.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:42 smithi203 ceph-mon[103431]: from='client.24422 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:00:43.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:42 smithi203 ceph-mon[103431]: osdmap e64: 10 total, 10 up, 10 in 2023-09-12T16:00:44.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:43 smithi130 ceph-mon[98670]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:44.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:43 smithi159 ceph-mon[102522]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:44.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:43 smithi203 ceph-mon[103431]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:45.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:44 smithi130 ceph-mon[98670]: from='client.? 172.21.15.159:0/3611135153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "742fd1e6-2b06-438c-8d3d-d7249d475f3e"}]: dispatch 2023-09-12T16:00:45.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:44 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "742fd1e6-2b06-438c-8d3d-d7249d475f3e"}]: dispatch 2023-09-12T16:00:45.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:44 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "742fd1e6-2b06-438c-8d3d-d7249d475f3e"}]': finished 2023-09-12T16:00:45.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:44 smithi130 ceph-mon[98670]: osdmap e65: 11 total, 10 up, 11 in 2023-09-12T16:00:45.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:44 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:00:45.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:44 smithi130 ceph-mon[98670]: from='client.? 172.21.15.159:0/2124771682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:00:45.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:44 smithi159 ceph-mon[102522]: from='client.? 172.21.15.159:0/3611135153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "742fd1e6-2b06-438c-8d3d-d7249d475f3e"}]: dispatch 2023-09-12T16:00:45.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:44 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "742fd1e6-2b06-438c-8d3d-d7249d475f3e"}]: dispatch 2023-09-12T16:00:45.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:44 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "742fd1e6-2b06-438c-8d3d-d7249d475f3e"}]': finished 2023-09-12T16:00:45.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:44 smithi159 ceph-mon[102522]: osdmap e65: 11 total, 10 up, 11 in 2023-09-12T16:00:45.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:44 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:00:45.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:44 smithi159 ceph-mon[102522]: from='client.? 172.21.15.159:0/2124771682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:00:45.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:44 smithi203 ceph-mon[103431]: from='client.? 172.21.15.159:0/3611135153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "742fd1e6-2b06-438c-8d3d-d7249d475f3e"}]: dispatch 2023-09-12T16:00:45.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:44 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "742fd1e6-2b06-438c-8d3d-d7249d475f3e"}]: dispatch 2023-09-12T16:00:45.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:44 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "742fd1e6-2b06-438c-8d3d-d7249d475f3e"}]': finished 2023-09-12T16:00:45.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:44 smithi203 ceph-mon[103431]: osdmap e65: 11 total, 10 up, 11 in 2023-09-12T16:00:45.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:44 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:00:45.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:44 smithi203 ceph-mon[103431]: from='client.? 172.21.15.159:0/2124771682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:00:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:45 smithi130 ceph-mon[98670]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:46.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:45 smithi159 ceph-mon[102522]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:45 smithi203 ceph-mon[103431]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:47.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:47 smithi130 ceph-mon[98670]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:47.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:47 smithi159 ceph-mon[102522]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:47.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:47 smithi203 ceph-mon[103431]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:49.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:49 smithi159 ceph-mon[102522]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:49 smithi130 ceph-mon[98670]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:49 smithi203 ceph-mon[103431]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:51.054 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-12T16:00:51.055 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:51.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-12T16:00:51.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:51.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-12T16:00:51.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:51 smithi130 ceph-mon[98670]: Deploying daemon osd.10 on smithi159 2023-09-12T16:00:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:51 smithi130 ceph-mon[98670]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:52.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:51 smithi159 ceph-mon[102522]: Deploying daemon osd.10 on smithi159 2023-09-12T16:00:52.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:51 smithi159 ceph-mon[102522]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:51 smithi203 ceph-mon[103431]: Deploying daemon osd.10 on smithi159 2023-09-12T16:00:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:51 smithi203 ceph-mon[103431]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:53 smithi130 ceph-mon[98670]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:54.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:53 smithi159 ceph-mon[102522]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:54.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:53 smithi203 ceph-mon[103431]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:55.814 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:55.815 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:55.815 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:55 smithi159 ceph-mon[102522]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:55.815 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:55.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:55.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:55.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:55 smithi130 ceph-mon[98670]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:55.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:55.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:55.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:55.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:55 smithi203 ceph-mon[103431]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:55.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:57.424 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 10 on host 'smithi159' 2023-09-12T16:00:57.425 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.422+0000 7f4be67fc700 1 -- 172.21.15.159:0/1715713379 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4bf402cf80 con 0x7f4be00a0640 2023-09-12T16:00:57.426 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4be00a0640 msgr2=0x7f4be00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:57.426 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4be00a0640 0x7f4be00a2b00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005950 tx=0x7f4be800a300 comp rx=0 tx=0).stop 2023-09-12T16:00:57.426 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf40ff5c0 msgr2=0x7f4bf41037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:00:57.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf40ff5c0 0x7f4bf41037e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000d980 tx=0x7f4bf000dc90 comp rx=0 tx=0).stop 2023-09-12T16:00:57.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 shutdown_connections 2023-09-12T16:00:57.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4be00a0640 0x7f4be00a2b00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:57.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf4101160 0x7f4bf4102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:57.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf40ff5c0 0x7f4bf41037e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:57.428 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 --2- 172.21.15.159:0/1715713379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4bf41007c0 0x7f4bf4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:00:57.428 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 >> 172.21.15.159:0/1715713379 conn(0x7f4bf40fab80 msgr2=0x7f4bf40fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:00:57.428 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 shutdown_connections 2023-09-12T16:00:57.428 INFO:teuthology.orchestra.run.smithi159.stderr:2023-09-12T16:00:57.425+0000 7f4bfb8b8700 1 -- 172.21.15.159:0/1715713379 wait complete. 2023-09-12T16:00:57.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:57 smithi159 ceph-mon[102522]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:57.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:57.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:57.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:57.907 DEBUG:teuthology.orchestra.run.smithi159:osd.10> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.10.service 2023-09-12T16:00:57.912 INFO:tasks.cephadm:Deploying osd.11 on smithi203 with /dev/vg_nvme/lv_1... 2023-09-12T16:00:57.912 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-12T16:00:57.943 INFO:journalctl@ceph.osd.10.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T16:00:57.945 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:57 smithi203 ceph-mon[103431]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:57.946 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.946 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.947 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:57.947 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:57.947 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.947 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.948 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.948 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:57.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:57 smithi130 ceph-mon[98670]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:57.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:00:57.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:00:57.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:00:57.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:00:58.479 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T16:00:58.936 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:58 smithi159 ceph-mon[102522]: from='osd.10 [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-12T16:00:58.936 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:58 smithi159 ceph-mon[102522]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-12T16:00:58.937 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:00:58 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:00:58.698+0000 7f2aec7fe640 -1 osd.10 0 log_to_monitors true 2023-09-12T16:00:58.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:58 smithi203 ceph-mon[103431]: from='osd.10 [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-12T16:00:58.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:58 smithi203 ceph-mon[103431]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-12T16:00:59.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:58 smithi130 ceph-mon[98670]: from='osd.10 [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-12T16:00:59.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:58 smithi130 ceph-mon[98670]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-12T16:00:59.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:59 smithi203 ceph-mon[103431]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:00:59.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:59 smithi203 ceph-mon[103431]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-12T16:00:59.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:59 smithi203 ceph-mon[103431]: from='osd.10 [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T16:00:59.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:59 smithi203 ceph-mon[103431]: osdmap e66: 11 total, 10 up, 11 in 2023-09-12T16:00:59.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:00:59.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:59 smithi203 ceph-mon[103431]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T16:00:59.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:59 smithi203 ceph-mon[103431]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T16:00:59.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:59 smithi203 ceph-mon[103431]: osdmap e67: 11 total, 10 up, 11 in 2023-09-12T16:00:59.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:00:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:00.150 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-12T16:01:00.150 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-12T16:01:00.151 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2023-09-12T16:01:00.151 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2023-09-12T16:01:00.151 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112966 s, 928 MB/s 2023-09-12T16:01:00.152 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2023-09-12T16:01:00.207 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:00:59.762+0000 7f2ae1e75700 -1 osd.10 0 waiting for initial osdmap 2023-09-12T16:01:00.207 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:00:59.774+0000 7f2addc6a700 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T16:01:00.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-mon[102522]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:01:00.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-mon[102522]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-12T16:01:00.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-mon[102522]: from='osd.10 [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T16:01:00.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-mon[102522]: osdmap e66: 11 total, 10 up, 11 in 2023-09-12T16:01:00.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:00.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-mon[102522]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T16:01:00.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-mon[102522]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T16:01:00.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-mon[102522]: osdmap e67: 11 total, 10 up, 11 in 2023-09-12T16:01:00.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:00:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:00.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:59 smithi130 ceph-mon[98670]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:01:00.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:59 smithi130 ceph-mon[98670]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-12T16:01:00.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:59 smithi130 ceph-mon[98670]: from='osd.10 [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T16:01:00.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:59 smithi130 ceph-mon[98670]: osdmap e66: 11 total, 10 up, 11 in 2023-09-12T16:01:00.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:00.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:59 smithi130 ceph-mon[98670]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-09-12T16:01:00.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:59 smithi130 ceph-mon[98670]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-09-12T16:01:00.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:59 smithi130 ceph-mon[98670]: osdmap e67: 11 total, 10 up, 11 in 2023-09-12T16:01:00.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:00:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:00.542 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi203:vg_nvme/lv_1 2023-09-12T16:01:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:00.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:00.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:00.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:00.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:00.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:00.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:00.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: osd.10 [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] boot 2023-09-12T16:01:00.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: osdmap e68: 11 total, 11 up, 11 in 2023-09-12T16:01:00.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:00.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:00.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:00.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:00.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:00.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:00.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:00.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:00.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:00.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:00.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:00.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:00.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: osd.10 [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] boot 2023-09-12T16:01:00.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: osdmap e68: 11 total, 11 up, 11 in 2023-09-12T16:01:00.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:01.140 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T16:01:01.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:01.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:01.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:01.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:01.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:01.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:01.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: osd.10 [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] boot 2023-09-12T16:01:01.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: osdmap e68: 11 total, 11 up, 11 in 2023-09-12T16:01:01.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-12T16:01:01.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:01 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T16:01:01.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:01 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T16:01:01.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:01 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi159 2023-09-12T16:01:01.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:01 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi159 to 4292M 2023-09-12T16:01:01.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:01 smithi203 ceph-mon[103431]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:01:01.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:01 smithi203 ceph-mon[103431]: osdmap e69: 11 total, 11 up, 11 in 2023-09-12T16:01:02.026 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:01 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T16:01:02.027 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:01 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T16:01:02.027 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:01 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi159 2023-09-12T16:01:02.027 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:01 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi159 to 4292M 2023-09-12T16:01:02.027 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:01 smithi130 ceph-mon[98670]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:01:02.028 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:01 smithi130 ceph-mon[98670]: osdmap e69: 11 total, 11 up, 11 in 2023-09-12T16:01:02.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:01 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T16:01:02.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:01 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T16:01:02.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:01 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi159 2023-09-12T16:01:02.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:01 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi159 to 4292M 2023-09-12T16:01:02.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:01 smithi159 ceph-mon[102522]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-12T16:01:02.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:01 smithi159 ceph-mon[102522]: osdmap e69: 11 total, 11 up, 11 in 2023-09-12T16:01:02.420 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 -- 172.21.15.203:0/2328839951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26281018c0 msgr2=0x7f2628101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:02.420 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2328839951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26281018c0 0x7f2628101d20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f261c0098f0 tx=0x7f261c009c00 comp rx=0 tx=0).stop 2023-09-12T16:01:02.421 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 -- 172.21.15.203:0/2328839951 shutdown_connections 2023-09-12T16:01:02.421 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2328839951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2628102260 0x7f2628108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:02.421 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2328839951 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26281006c0 0x7f2628100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:02.421 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2328839951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26281018c0 0x7f2628101d20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:02.421 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 -- 172.21.15.203:0/2328839951 >> 172.21.15.203:0/2328839951 conn(0x7f26280fbc80 msgr2=0x7f26280fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:02.422 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 -- 172.21.15.203:0/2328839951 shutdown_connections 2023-09-12T16:01:02.422 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 -- 172.21.15.203:0/2328839951 wait complete. 2023-09-12T16:01:02.422 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 Processor -- start 2023-09-12T16:01:02.422 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.419+0000 7f262ea4f700 1 -- start start 2023-09-12T16:01:02.423 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262ea4f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26281006c0 0x7f26281963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:02.423 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262ea4f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26281018c0 0x7f26281968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:02.423 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262ea4f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2628102260 0x7f262819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:02.423 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262ea4f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262819df30 con 0x7f26281006c0 2023-09-12T16:01:02.423 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262ea4f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262819e070 con 0x7f26281018c0 2023-09-12T16:01:02.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262ea4f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262819e200 con 0x7f2628102260 2023-09-12T16:01:02.426 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262cfec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2628102260 0x7f262819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:02.426 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262cfec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2628102260 0x7f262819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:41760/0 (socket says 172.21.15.203:41760) 2023-09-12T16:01:02.426 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262cfec700 1 -- 172.21.15.203:0/2109663155 learned_addr learned my addr 172.21.15.203:0/2109663155 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-09-12T16:01:02.426 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f2627fff700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26281006c0 0x7f26281963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:02.427 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f26277fe700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26281018c0 0x7f26281968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:02.427 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262cfec700 1 -- 172.21.15.203:0/2109663155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26281018c0 msgr2=0x7f26281968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:02.427 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262cfec700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26281018c0 0x7f26281968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:02.427 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262cfec700 1 -- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26281006c0 msgr2=0x7f26281963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:02.427 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262cfec700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26281006c0 0x7f26281963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:02.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262cfec700 1 -- 172.21.15.203:0/2109663155 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2620009d00 con 0x7f2628102260 2023-09-12T16:01:02.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f26277fe700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26281018c0 0x7f26281968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:01:02.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262cfec700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2628102260 0x7f262819d960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f262000e3f0 tx=0x7f262000e7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:02.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f26257fa700 1 -- 172.21.15.203:0/2109663155 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200092d0 con 0x7f2628102260 2023-09-12T16:01:02.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c009520 con 0x7f2628102260 2023-09-12T16:01:02.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f26257fa700 1 -- 172.21.15.203:0/2109663155 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f262000f040 con 0x7f2628102260 2023-09-12T16:01:02.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f26257fa700 1 -- 172.21.15.203:0/2109663155 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620007de0 con 0x7f2628102260 2023-09-12T16:01:02.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.420+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262819e800 con 0x7f2628102260 2023-09-12T16:01:02.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.422+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262810bec0 con 0x7f2628102260 2023-09-12T16:01:02.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.422+0000 7f26257fa700 1 -- 172.21.15.203:0/2109663155 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f2620003a00 con 0x7f2628102260 2023-09-12T16:01:02.430 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.423+0000 7f26257fa700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26100a0640 0x7f26100a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:02.430 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.423+0000 7f26257fa700 1 -- 172.21.15.203:0/2109663155 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f26200cb350 con 0x7f2628102260 2023-09-12T16:01:02.430 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.423+0000 7f2627fff700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26100a0640 0x7f26100a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:02.430 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.424+0000 7f2627fff700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26100a0640 0x7f26100a2b00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2628101720 tx=0x7f261800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:02.431 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.427+0000 7f26257fa700 1 -- 172.21.15.203:0/2109663155 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2620095540 con 0x7f2628102260 2023-09-12T16:01:02.629 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:02.628+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f262819ea20 con 0x7f26100a0640 2023-09-12T16:01:03.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:03 smithi203 ceph-mon[103431]: osdmap e70: 11 total, 11 up, 11 in 2023-09-12T16:01:03.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:03 smithi203 ceph-mon[103431]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:01:03.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:03 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:01:03.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:03 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:01:03.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:03 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:03.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:03 smithi203 ceph-mon[103431]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:03.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:03 smithi130 ceph-mon[98670]: osdmap e70: 11 total, 11 up, 11 in 2023-09-12T16:01:03.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:03 smithi130 ceph-mon[98670]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:01:03.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:03 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:01:03.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:03 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:01:03.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:03 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:03.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:03 smithi130 ceph-mon[98670]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:03.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:03 smithi159 ceph-mon[102522]: osdmap e70: 11 total, 11 up, 11 in 2023-09-12T16:01:03.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:03 smithi159 ceph-mon[102522]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:01:03.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:03 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-12T16:01:03.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:03 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-12T16:01:03.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:03 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:03.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:03 smithi159 ceph-mon[102522]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:04.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:04 smithi203 ceph-mon[103431]: osdmap e71: 11 total, 11 up, 11 in 2023-09-12T16:01:04.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:04 smithi130 ceph-mon[98670]: osdmap e71: 11 total, 11 up, 11 in 2023-09-12T16:01:04.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:04 smithi159 ceph-mon[102522]: osdmap e71: 11 total, 11 up, 11 in 2023-09-12T16:01:05.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:05 smithi130 ceph-mon[98670]: osdmap e72: 11 total, 11 up, 11 in 2023-09-12T16:01:05.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:05 smithi130 ceph-mon[98670]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 145 KiB/s, 0 objects/s recovering 2023-09-12T16:01:05.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:05 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/730754753' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32e5d8c-4556-4d0e-83ce-9f35138be3f4"}]: dispatch 2023-09-12T16:01:05.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:05 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32e5d8c-4556-4d0e-83ce-9f35138be3f4"}]: dispatch 2023-09-12T16:01:05.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:05 smithi130 ceph-mon[98670]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b32e5d8c-4556-4d0e-83ce-9f35138be3f4"}]': finished 2023-09-12T16:01:05.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:05 smithi130 ceph-mon[98670]: osdmap e73: 12 total, 11 up, 12 in 2023-09-12T16:01:05.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:05 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:05.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:05 smithi159 ceph-mon[102522]: osdmap e72: 11 total, 11 up, 11 in 2023-09-12T16:01:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:05 smithi159 ceph-mon[102522]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 145 KiB/s, 0 objects/s recovering 2023-09-12T16:01:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:05 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/730754753' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32e5d8c-4556-4d0e-83ce-9f35138be3f4"}]: dispatch 2023-09-12T16:01:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:05 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32e5d8c-4556-4d0e-83ce-9f35138be3f4"}]: dispatch 2023-09-12T16:01:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:05 smithi159 ceph-mon[102522]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b32e5d8c-4556-4d0e-83ce-9f35138be3f4"}]': finished 2023-09-12T16:01:05.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:05 smithi159 ceph-mon[102522]: osdmap e73: 12 total, 11 up, 12 in 2023-09-12T16:01:05.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:05 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:05.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:05 smithi203 ceph-mon[103431]: osdmap e72: 11 total, 11 up, 11 in 2023-09-12T16:01:05.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:05 smithi203 ceph-mon[103431]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 145 KiB/s, 0 objects/s recovering 2023-09-12T16:01:05.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:05 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/730754753' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32e5d8c-4556-4d0e-83ce-9f35138be3f4"}]: dispatch 2023-09-12T16:01:05.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:05 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32e5d8c-4556-4d0e-83ce-9f35138be3f4"}]: dispatch 2023-09-12T16:01:05.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:05 smithi203 ceph-mon[103431]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b32e5d8c-4556-4d0e-83ce-9f35138be3f4"}]': finished 2023-09-12T16:01:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:05 smithi203 ceph-mon[103431]: osdmap e73: 12 total, 11 up, 12 in 2023-09-12T16:01:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:05 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:06.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:06 smithi130 ceph-mon[98670]: from='client.? 172.21.15.203:0/4117757528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:01:06.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:06 smithi159 ceph-mon[102522]: from='client.? 172.21.15.203:0/4117757528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:01:06.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:06 smithi203 ceph-mon[103431]: from='client.? 172.21.15.203:0/4117757528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-12T16:01:07.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:07 smithi130 ceph-mon[98670]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2023-09-12T16:01:07.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:07 smithi159 ceph-mon[102522]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2023-09-12T16:01:07.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:07 smithi203 ceph-mon[103431]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2023-09-12T16:01:10.085 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:09 smithi203 ceph-mon[103431]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-12T16:01:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:09 smithi130 ceph-mon[98670]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-12T16:01:10.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:09 smithi159 ceph-mon[102522]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-12T16:01:12.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:11 smithi130 ceph-mon[98670]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-09-12T16:01:12.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:11 smithi159 ceph-mon[102522]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-09-12T16:01:12.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:11 smithi203 ceph-mon[103431]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-09-12T16:01:12.929 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-12T16:01:12.929 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:12 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:12.929 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:12 smithi203 ceph-mon[103431]: Deploying daemon osd.11 on smithi203 2023-09-12T16:01:13.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-12T16:01:13.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:12 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:13.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:12 smithi130 ceph-mon[98670]: Deploying daemon osd.11 on smithi203 2023-09-12T16:01:13.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-12T16:01:13.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:12 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:13.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:12 smithi159 ceph-mon[102522]: Deploying daemon osd.11 on smithi203 2023-09-12T16:01:14.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:13 smithi130 ceph-mon[98670]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:14.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:13 smithi159 ceph-mon[102522]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:14.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:13 smithi203 ceph-mon[103431]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:15.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:15 smithi203 ceph-mon[103431]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:16.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:15 smithi130 ceph-mon[98670]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:16.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:15 smithi159 ceph-mon[102522]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:17.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:17.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:17.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:01:17.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:17 smithi130 ceph-mon[98670]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:17.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:17.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:17.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:01:17.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:17 smithi159 ceph-mon[102522]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:17.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:17.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:17.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:01:17.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:17 smithi203 ceph-mon[103431]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:19.846 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 11 on host 'smithi203' 2023-09-12T16:01:19.847 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.846+0000 7f26257fa700 1 -- 172.21.15.203:0/2109663155 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f262819ea20 con 0x7f26100a0640 2023-09-12T16:01:19.849 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26100a0640 msgr2=0x7f26100a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:19.849 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26100a0640 0x7f26100a2b00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2628101720 tx=0x7f261800b5b0 comp rx=0 tx=0).stop 2023-09-12T16:01:19.850 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2628102260 msgr2=0x7f262819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:19.850 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2628102260 0x7f262819d960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f262000e3f0 tx=0x7f262000e7b0 comp rx=0 tx=0).stop 2023-09-12T16:01:19.850 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 shutdown_connections 2023-09-12T16:01:19.850 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26100a0640 0x7f26100a2b00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:19.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2628102260 0x7f262819d960 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:19.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26281006c0 0x7f26281963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:19.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 --2- 172.21.15.203:0/2109663155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26281018c0 0x7f26281968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:19.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 >> 172.21.15.203:0/2109663155 conn(0x7f26280fbc80 msgr2=0x7f26280fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:19.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.849+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 shutdown_connections 2023-09-12T16:01:19.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:19.850+0000 7f262ea4f700 1 -- 172.21.15.203:0/2109663155 wait complete. 2023-09-12T16:01:19.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:19 smithi203 ceph-mon[103431]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:19.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:19.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:19.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:19.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:19.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:20.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:19 smithi130 ceph-mon[98670]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:20.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:20.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:20.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:20.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:20.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:20.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:19 smithi159 ceph-mon[102522]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:20.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:20.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:20.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:20.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:20.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:20.323 DEBUG:teuthology.orchestra.run.smithi203:osd.11> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.11.service 2023-09-12T16:01:20.327 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-09-12T16:01:20.327 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd stat -f json 2023-09-12T16:01:20.359 INFO:journalctl@ceph.osd.11.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T16:01:20.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:20.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:01:20.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:20.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:20 smithi203 ceph-mon[103431]: from='osd.11 [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-12T16:01:20.886 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:20 smithi203 ceph-mon[103431]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-12T16:01:20.886 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:01:20 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:01:20.589+0000 7f8f51905640 -1 osd.11 0 log_to_monitors true 2023-09-12T16:01:20.891 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:01:21.074 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:21.075 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:01:21.075 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:21.075 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:20 smithi130 ceph-mon[98670]: from='osd.11 [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-12T16:01:21.075 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:20 smithi130 ceph-mon[98670]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-12T16:01:21.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:21.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:01:21.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:21.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:20 smithi159 ceph-mon[102522]: from='osd.11 [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-12T16:01:21.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:20 smithi159 ceph-mon[102522]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-12T16:01:21.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:21 smithi130 ceph-mon[98670]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:21.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:21 smithi130 ceph-mon[98670]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-12T16:01:21.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:21 smithi130 ceph-mon[98670]: from='osd.11 [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:01:21.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:21 smithi130 ceph-mon[98670]: osdmap e74: 12 total, 11 up, 12 in 2023-09-12T16:01:21.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:21 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:21.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:21 smithi130 ceph-mon[98670]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:01:21.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:01:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:01:21.818+0000 7f8f46f7c700 -1 osd.11 0 waiting for initial osdmap 2023-09-12T16:01:21.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:01:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:01:21.834+0000 7f8f3fd6b700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T16:01:21.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:21 smithi203 ceph-mon[103431]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:21.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:21 smithi203 ceph-mon[103431]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-12T16:01:21.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:21 smithi203 ceph-mon[103431]: from='osd.11 [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:01:21.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:21 smithi203 ceph-mon[103431]: osdmap e74: 12 total, 11 up, 12 in 2023-09-12T16:01:21.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:21 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:21.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:21 smithi203 ceph-mon[103431]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:01:22.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.053+0000 7f17614e2700 1 -- 172.21.15.130:0/637792530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c100c70 msgr2=0x7f175c101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:22.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.053+0000 7f17614e2700 1 --2- 172.21.15.130:0/637792530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c100c70 0x7f175c101090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f174c009990 tx=0x7f174c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.053+0000 7f17614e2700 1 -- 172.21.15.130:0/637792530 shutdown_connections 2023-09-12T16:01:22.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.053+0000 7f17614e2700 1 --2- 172.21.15.130:0/637792530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c100200 0x7f175c107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.053+0000 7f17614e2700 1 --2- 172.21.15.130:0/637792530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c100c70 0x7f175c101090 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.053+0000 7f17614e2700 1 --2- 172.21.15.130:0/637792530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f175c0ff840 0x7f175c0ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.053+0000 7f17614e2700 1 -- 172.21.15.130:0/637792530 >> 172.21.15.130:0/637792530 conn(0x7f175c0fa310 msgr2=0x7f175c0fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:22.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.053+0000 7f17614e2700 1 -- 172.21.15.130:0/637792530 shutdown_connections 2023-09-12T16:01:22.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.053+0000 7f17614e2700 1 -- 172.21.15.130:0/637792530 wait complete. 2023-09-12T16:01:22.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f17614e2700 1 Processor -- start 2023-09-12T16:01:22.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f17614e2700 1 -- start start 2023-09-12T16:01:22.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f17614e2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c0ff840 0x7f175c0ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:22.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f17614e2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f175c100200 0x7f175c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:22.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f17614e2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c100c70 0x7f175c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:22.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f17614e2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c19dee0 con 0x7f175c0ff840 2023-09-12T16:01:22.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f17614e2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c19e050 con 0x7f175c100c70 2023-09-12T16:01:22.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f17614e2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c19e210 con 0x7f175c100200 2023-09-12T16:01:22.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f175affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c0ff840 0x7f175c0ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:22.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f175affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c0ff840 0x7f175c0ff2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52888/0 (socket says 172.21.15.130:52888) 2023-09-12T16:01:22.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f175affd700 1 -- 172.21.15.130:0/3255689227 learned_addr learned my addr 172.21.15.130:0/3255689227 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:22.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.054+0000 7f175a7fc700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f175c100200 0x7f175c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:22.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f175b7fe700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c100c70 0x7f175c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:22.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f175affd700 1 -- 172.21.15.130:0/3255689227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f175c100200 msgr2=0x7f175c19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:22.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f175affd700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f175c100200 0x7f175c19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f175affd700 1 -- 172.21.15.130:0/3255689227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c100c70 msgr2=0x7f175c19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:22.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f175affd700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c100c70 0x7f175c19d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f175affd700 1 -- 172.21.15.130:0/3255689227 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c0095f0 con 0x7f175c0ff840 2023-09-12T16:01:22.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f175affd700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c0ff840 0x7f175c0ff2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f174c005910 tx=0x7f174c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:22.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f1743fff700 1 -- 172.21.15.130:0/3255689227 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c01d070 con 0x7f175c0ff840 2023-09-12T16:01:22.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f1743fff700 1 -- 172.21.15.130:0/3255689227 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f174c022470 con 0x7f175c0ff840 2023-09-12T16:01:22.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f1743fff700 1 -- 172.21.15.130:0/3255689227 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c017650 con 0x7f175c0ff840 2023-09-12T16:01:22.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c19e490 con 0x7f175c0ff840 2023-09-12T16:01:22.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.055+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c19e850 con 0x7f175c0ff840 2023-09-12T16:01:22.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.056+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c10ae80 con 0x7f175c0ff840 2023-09-12T16:01:22.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.057+0000 7f1743fff700 1 -- 172.21.15.130:0/3255689227 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f174c00fad0 con 0x7f175c0ff840 2023-09-12T16:01:22.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.058+0000 7f1743fff700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f17440a0640 0x7f17440a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:22.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.058+0000 7f175a7fc700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f17440a0640 0x7f17440a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:22.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.058+0000 7f1743fff700 1 -- 172.21.15.130:0/3255689227 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f174c0ccd30 con 0x7f175c0ff840 2023-09-12T16:01:22.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.058+0000 7f175a7fc700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f17440a0640 0x7f17440a2b00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1750005950 tx=0x7f175000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:22.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.061+0000 7f1743fff700 1 -- 172.21.15.130:0/3255689227 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f174c096eb0 con 0x7f175c0ff840 2023-09-12T16:01:22.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:21 smithi159 ceph-mon[102522]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:22.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:21 smithi159 ceph-mon[102522]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-12T16:01:22.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:21 smithi159 ceph-mon[102522]: from='osd.11 [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:01:22.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:21 smithi159 ceph-mon[102522]: osdmap e74: 12 total, 11 up, 12 in 2023-09-12T16:01:22.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:21 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:22.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:21 smithi159 ceph-mon[102522]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-09-12T16:01:22.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.249+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f175c106550 con 0x7f175c0ff840 2023-09-12T16:01:22.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.249+0000 7f1743fff700 1 -- 172.21.15.130:0/3255689227 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f174c09a4d0 con 0x7f175c0ff840 2023-09-12T16:01:22.250 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:22.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f17440a0640 msgr2=0x7f17440a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:22.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f17440a0640 0x7f17440a2b00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1750005950 tx=0x7f175000a300 comp rx=0 tx=0).stop 2023-09-12T16:01:22.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c0ff840 msgr2=0x7f175c0ff2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:22.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c0ff840 0x7f175c0ff2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f174c005910 tx=0x7f174c00f690 comp rx=0 tx=0).stop 2023-09-12T16:01:22.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 shutdown_connections 2023-09-12T16:01:22.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f17440a0640 0x7f17440a2b00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c100c70 0x7f175c19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f175c0ff840 0x7f175c0ff2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 --2- 172.21.15.130:0/3255689227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f175c100200 0x7f175c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:22.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 >> 172.21.15.130:0/3255689227 conn(0x7f175c0fa310 msgr2=0x7f175c0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:22.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 shutdown_connections 2023-09-12T16:01:22.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:22.251+0000 7f17614e2700 1 -- 172.21.15.130:0/3255689227 wait complete. 2023-09-12T16:01:22.646 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1694534460,"num_in_osds":12,"osd_in_since":1694534465,"num_remapped_pgs":0} 2023-09-12T16:01:22.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:22 smithi130 ceph-mon[98670]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T16:01:22.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:22 smithi130 ceph-mon[98670]: osdmap e75: 12 total, 11 up, 12 in 2023-09-12T16:01:22.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:22 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:22.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:22 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:22.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:22 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3255689227' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-12T16:01:23.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:22 smithi159 ceph-mon[102522]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T16:01:23.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:22 smithi159 ceph-mon[102522]: osdmap e75: 12 total, 11 up, 12 in 2023-09-12T16:01:23.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:22 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:23.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:22 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:23.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:22 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3255689227' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-12T16:01:23.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:22 smithi203 ceph-mon[103431]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-09-12T16:01:23.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:22 smithi203 ceph-mon[103431]: osdmap e75: 12 total, 11 up, 12 in 2023-09-12T16:01:23.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:22 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:23.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:22 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:23.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:22 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3255689227' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-12T16:01:23.648 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd stat -f json 2023-09-12T16:01:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: purged_snaps scrub starts 2023-09-12T16:01:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: purged_snaps scrub ok 2023-09-12T16:01:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:23.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: osd.11 [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] boot 2023-09-12T16:01:23.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: osdmap e76: 12 total, 12 up, 12 in 2023-09-12T16:01:23.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:23.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: Detected new or changed devices on smithi203 2023-09-12T16:01:23.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:23.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:23.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:23.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:23.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:23.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:23.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi203 to 4292M 2023-09-12T16:01:23.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:23.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:23.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:23.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:23 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:24.216 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:01:24.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: purged_snaps scrub starts 2023-09-12T16:01:24.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: purged_snaps scrub ok 2023-09-12T16:01:24.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:24.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:24.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: osd.11 [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] boot 2023-09-12T16:01:24.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: osdmap e76: 12 total, 12 up, 12 in 2023-09-12T16:01:24.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:24.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: Detected new or changed devices on smithi203 2023-09-12T16:01:24.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:24.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:24.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:24.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:24.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:24.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:24.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi203 to 4292M 2023-09-12T16:01:24.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:24.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:24.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:24.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:23 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:24.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: purged_snaps scrub starts 2023-09-12T16:01:24.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: purged_snaps scrub ok 2023-09-12T16:01:24.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-12T16:01:24.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:24.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: osd.11 [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] boot 2023-09-12T16:01:24.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: osdmap e76: 12 total, 12 up, 12 in 2023-09-12T16:01:24.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-12T16:01:24.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: Detected new or changed devices on smithi203 2023-09-12T16:01:24.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:24.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:24.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:24.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:24.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:24.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:01:24.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi203 to 4292M 2023-09-12T16:01:24.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:24.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:24.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:24.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:23 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:24.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:24 smithi130 ceph-mon[98670]: osdmap e77: 12 total, 12 up, 12 in 2023-09-12T16:01:25.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:24 smithi159 ceph-mon[102522]: osdmap e77: 12 total, 12 up, 12 in 2023-09-12T16:01:25.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:24 smithi203 ceph-mon[103431]: osdmap e77: 12 total, 12 up, 12 in 2023-09-12T16:01:25.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.419+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/283196260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74101130 msgr2=0x7f2d741079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:25.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.419+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/283196260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74101130 0x7f2d741079e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00b0d0 tx=0x7f2d6c00b3e0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.419+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/283196260 shutdown_connections 2023-09-12T16:01:25.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.419+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/283196260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74101130 0x7f2d741079e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.419+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/283196260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d740ff590 0x7f2d740ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.419+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/283196260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d74100790 0x7f2d74100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.419+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/283196260 >> 172.21.15.130:0/283196260 conn(0x7f2d740fab10 msgr2=0x7f2d740fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:25.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.419+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/283196260 shutdown_connections 2023-09-12T16:01:25.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.419+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/283196260 wait complete. 2023-09-12T16:01:25.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7c4fb700 1 Processor -- start 2023-09-12T16:01:25.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7c4fb700 1 -- start start 2023-09-12T16:01:25.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7c4fb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d740ff590 0x7f2d74195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:25.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7c4fb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74100790 0x7f2d741957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:25.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7c4fb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d74101130 0x7f2d7419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:25.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7c4fb700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419ce20 con 0x7f2d74100790 2023-09-12T16:01:25.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7c4fb700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419cf60 con 0x7f2d740ff590 2023-09-12T16:01:25.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7c4fb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419d0f0 con 0x7f2d74101130 2023-09-12T16:01:25.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d79a96700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74100790 0x7f2d741957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:25.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d79a96700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74100790 0x7f2d741957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52906/0 (socket says 172.21.15.130:52906) 2023-09-12T16:01:25.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d79a96700 1 -- 172.21.15.130:0/1510226231 learned_addr learned my addr 172.21.15.130:0/1510226231 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:25.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7a297700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d740ff590 0x7f2d74195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:25.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d7aa98700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d74101130 0x7f2d7419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:25.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d79a96700 1 -- 172.21.15.130:0/1510226231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d74101130 msgr2=0x7f2d7419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:25.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d79a96700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d74101130 0x7f2d7419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d79a96700 1 -- 172.21.15.130:0/1510226231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d740ff590 msgr2=0x7f2d74195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:25.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d79a96700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d740ff590 0x7f2d74195290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.420+0000 7f2d79a96700 1 -- 172.21.15.130:0/1510226231 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c009d00 con 0x7f2d74100790 2023-09-12T16:01:25.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.421+0000 7f2d79a96700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74100790 0x7f2d741957d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000c950 tx=0x7f2d7000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:25.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.421+0000 7f2d6b7fe700 1 -- 172.21.15.130:0/1510226231 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000cea0 con 0x7f2d74100790 2023-09-12T16:01:25.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.421+0000 7f2d6b7fe700 1 -- 172.21.15.130:0/1510226231 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d70004830 con 0x7f2d74100790 2023-09-12T16:01:25.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.421+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7419d3d0 con 0x7f2d74100790 2023-09-12T16:01:25.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.421+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7419d920 con 0x7f2d74100790 2023-09-12T16:01:25.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.421+0000 7f2d6b7fe700 1 -- 172.21.15.130:0/1510226231 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d700054e0 con 0x7f2d74100790 2023-09-12T16:01:25.432 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.422+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d58005320 con 0x7f2d74100790 2023-09-12T16:01:25.432 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.423+0000 7f2d6b7fe700 1 -- 172.21.15.130:0/1510226231 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f2d70007ac0 con 0x7f2d74100790 2023-09-12T16:01:25.432 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.423+0000 7f2d6b7fe700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d600a0740 0x7f2d600a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:25.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.423+0000 7f2d7a297700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d600a0740 0x7f2d600a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:25.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.424+0000 7f2d6b7fe700 1 -- 172.21.15.130:0/1510226231 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2d700cb730 con 0x7f2d74100790 2023-09-12T16:01:25.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.424+0000 7f2d7a297700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d600a0740 0x7f2d600a2c00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2d741005f0 tx=0x7f2d6400a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:25.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.427+0000 7f2d6b7fe700 1 -- 172.21.15.130:0/1510226231 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2d700956e0 con 0x7f2d74100790 2023-09-12T16:01:25.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.616+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f2d58005190 con 0x7f2d74100790 2023-09-12T16:01:25.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.616+0000 7f2d6b7fe700 1 -- 172.21.15.130:0/1510226231 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f2d7001d0a0 con 0x7f2d74100790 2023-09-12T16:01:25.617 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:25.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d600a0740 msgr2=0x7f2d600a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:25.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d600a0740 0x7f2d600a2c00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2d741005f0 tx=0x7f2d6400a3b0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74100790 msgr2=0x7f2d741957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:25.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74100790 0x7f2d741957d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000c950 tx=0x7f2d7000cc60 comp rx=0 tx=0).stop 2023-09-12T16:01:25.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 shutdown_connections 2023-09-12T16:01:25.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d600a0740 0x7f2d600a2c00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d74101130 0x7f2d7419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d740ff590 0x7f2d74195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 --2- 172.21.15.130:0/1510226231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d74100790 0x7f2d741957d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:25.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 >> 172.21.15.130:0/1510226231 conn(0x7f2d740fab10 msgr2=0x7f2d74103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:25.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 shutdown_connections 2023-09-12T16:01:25.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:25.618+0000 7f2d7c4fb700 1 -- 172.21.15.130:0/1510226231 wait complete. 2023-09-12T16:01:25.970 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1694534482,"num_in_osds":12,"osd_in_since":1694534465,"num_remapped_pgs":0} 2023-09-12T16:01:25.971 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-12T16:01:26.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:25 smithi130 ceph-mon[98670]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:26.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:25 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1510226231' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-12T16:01:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:25 smithi159 ceph-mon[102522]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:25 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1510226231' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-12T16:01:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:25 smithi203 ceph-mon[103431]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:25 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1510226231' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-12T16:01:26.526 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:01:27.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 -- 172.21.15.130:0/4007316273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc105610 msgr2=0x7f6adc1059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:27.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 --2- 172.21.15.130:0/4007316273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc105610 0x7f6adc1059f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80099c0 tx=0x7f6ad8009cd0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 -- 172.21.15.130:0/4007316273 shutdown_connections 2023-09-12T16:01:27.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 --2- 172.21.15.130:0/4007316273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6adc0fff30 0x7f6adc10c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 --2- 172.21.15.130:0/4007316273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc105610 0x7f6adc1059f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 --2- 172.21.15.130:0/4007316273 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc0ff570 0x7f6adc0ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 -- 172.21.15.130:0/4007316273 >> 172.21.15.130:0/4007316273 conn(0x7f6adc076680 msgr2=0x7f6adc076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:27.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 -- 172.21.15.130:0/4007316273 shutdown_connections 2023-09-12T16:01:27.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 -- 172.21.15.130:0/4007316273 wait complete. 2023-09-12T16:01:27.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 Processor -- start 2023-09-12T16:01:27.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.749+0000 7f6ae3797700 1 -- start start 2023-09-12T16:01:27.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae3797700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc0ff570 0x7f6adc1011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:27.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae3797700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc0fff30 0x7f6adc101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:27.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae3797700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6adc105610 0x7f6adc101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:27.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae3797700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1a4bb0 con 0x7f6adc0ff570 2023-09-12T16:01:27.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae3797700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1a4d20 con 0x7f6adc105610 2023-09-12T16:01:27.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae3797700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1a4ee0 con 0x7f6adc0fff30 2023-09-12T16:01:27.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1533700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc0ff570 0x7f6adc1011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:27.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1533700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc0ff570 0x7f6adc1011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52916/0 (socket says 172.21.15.130:52916) 2023-09-12T16:01:27.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1533700 1 -- 172.21.15.130:0/2857761224 learned_addr learned my addr 172.21.15.130:0/2857761224 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:27.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1d34700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6adc105610 0x7f6adc101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:27.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae0d32700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc0fff30 0x7f6adc101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:27.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1533700 1 -- 172.21.15.130:0/2857761224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc0fff30 msgr2=0x7f6adc101710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:27.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1533700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc0fff30 0x7f6adc101710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1533700 1 -- 172.21.15.130:0/2857761224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6adc105610 msgr2=0x7f6adc101c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:27.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1533700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6adc105610 0x7f6adc101c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1533700 1 -- 172.21.15.130:0/2857761224 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad80095f0 con 0x7f6adc0ff570 2023-09-12T16:01:27.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1d34700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6adc105610 0x7f6adc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:01:27.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ae1533700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc0ff570 0x7f6adc1011d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800b580 tx=0x7f6ad8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:27.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.750+0000 7f6ad27fc700 1 -- 172.21.15.130:0/2857761224 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad801d070 con 0x7f6adc0ff570 2023-09-12T16:01:27.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.751+0000 7f6ad27fc700 1 -- 172.21.15.130:0/2857761224 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ad8022470 con 0x7f6adc0ff570 2023-09-12T16:01:27.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.751+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc1a5160 con 0x7f6adc0ff570 2023-09-12T16:01:27.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.751+0000 7f6ad27fc700 1 -- 172.21.15.130:0/2857761224 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800f6c0 con 0x7f6adc0ff570 2023-09-12T16:01:27.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.751+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1a55d0 con 0x7f6adc0ff570 2023-09-12T16:01:27.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.752+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc04e300 con 0x7f6adc0ff570 2023-09-12T16:01:27.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.753+0000 7f6ad27fc700 1 -- 172.21.15.130:0/2857761224 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f6ad80225e0 con 0x7f6adc0ff570 2023-09-12T16:01:27.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.753+0000 7f6ad27fc700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6ac80a06c0 0x7f6ac80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:27.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.753+0000 7f6ae0d32700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6ac80a06c0 0x7f6ac80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:27.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.753+0000 7f6ad27fc700 1 -- 172.21.15.130:0/2857761224 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6ad80ccab0 con 0x7f6adc0ff570 2023-09-12T16:01:27.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.754+0000 7f6ae0d32700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6ac80a06c0 0x7f6ac80a2b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6acc005fd0 tx=0x7f6acc00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:27.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.757+0000 7f6ad27fc700 1 -- 172.21.15.130:0/2857761224 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6ad8096cc0 con 0x7f6adc0ff570 2023-09-12T16:01:27.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.945+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6adc10f600 con 0x7f6adc0ff570 2023-09-12T16:01:27.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.946+0000 7f6ad27fc700 1 -- 172.21.15.130:0/2857761224 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f6ad809a2e0 con 0x7f6adc0ff570 2023-09-12T16:01:27.947 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:27.947 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":77,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:01:23.890141+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:01:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6ac80a06c0 msgr2=0x7f6ac80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6ac80a06c0 0x7f6ac80a2b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6acc005fd0 tx=0x7f6acc00a560 comp rx=0 tx=0).stop 2023-09-12T16:01:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc0ff570 msgr2=0x7f6adc1011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc0ff570 0x7f6adc1011d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800b580 tx=0x7f6ad8005ef0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 shutdown_connections 2023-09-12T16:01:27.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6ac80a06c0 0x7f6ac80a2b80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6adc105610 0x7f6adc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6adc0ff570 0x7f6adc1011d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 --2- 172.21.15.130:0/2857761224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc0fff30 0x7f6adc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:27.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 >> 172.21.15.130:0/2857761224 conn(0x7f6adc076680 msgr2=0x7f6adc068aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:27.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 shutdown_connections 2023-09-12T16:01:27.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:27.948+0000 7f6ae3797700 1 -- 172.21.15.130:0/2857761224 wait complete. 2023-09-12T16:01:27.970 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:27 smithi130 ceph-mon[98670]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:27.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:27 smithi159 ceph-mon[102522]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:27 smithi203 ceph-mon[103431]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:28.279 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T16:01:28.280 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd pool get .mgr pg_num 2023-09-12T16:01:28.832 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:01:28.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:28 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2857761224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:01:28.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:28 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2857761224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:01:28.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:28 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2857761224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:01:29.877 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:29 smithi130 ceph-mon[98670]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:29.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:29 smithi159 ceph-mon[102522]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:29.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:29 smithi203 ceph-mon[103431]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:30.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.026+0000 7f2bf23cc700 1 -- 172.21.15.130:0/1340315218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 msgr2=0x7f2bec104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:30.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.026+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/1340315218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 0x7f2bec104910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009990 tx=0x7f2be0009ca0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.026+0000 7f2bf23cc700 1 -- 172.21.15.130:0/1340315218 shutdown_connections 2023-09-12T16:01:30.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.026+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/1340315218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bec0ffee0 0x7f2bec10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.026+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/1340315218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 0x7f2bec104910 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.026+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/1340315218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bec0ff520 0x7f2bec0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.026+0000 7f2bf23cc700 1 -- 172.21.15.130:0/1340315218 >> 172.21.15.130:0/1340315218 conn(0x7f2bec076600 msgr2=0x7f2bec076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:30.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.026+0000 7f2bf23cc700 1 -- 172.21.15.130:0/1340315218 shutdown_connections 2023-09-12T16:01:30.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.026+0000 7f2bf23cc700 1 -- 172.21.15.130:0/1340315218 wait complete. 2023-09-12T16:01:30.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf23cc700 1 Processor -- start 2023-09-12T16:01:30.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf23cc700 1 -- start start 2023-09-12T16:01:30.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf23cc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bec0ff520 0x7f2bec101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:30.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf23cc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bec0ffee0 0x7f2bec1016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:30.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf23cc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 0x7f2bec101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:30.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf23cc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1a4bd0 con 0x7f2bec104530 2023-09-12T16:01:30.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf23cc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1a4d40 con 0x7f2bec0ff520 2023-09-12T16:01:30.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf23cc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1a4f00 con 0x7f2bec0ffee0 2023-09-12T16:01:30.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf0969700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 0x7f2bec101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:30.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf0969700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 0x7f2bec101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52932/0 (socket says 172.21.15.130:52932) 2023-09-12T16:01:30.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf0969700 1 -- 172.21.15.130:0/2069832332 learned_addr learned my addr 172.21.15.130:0/2069832332 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:30.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bebfff700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bec0ff520 0x7f2bec101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:30.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2beb7fe700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bec0ffee0 0x7f2bec1016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:30.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf0969700 1 -- 172.21.15.130:0/2069832332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bec0ffee0 msgr2=0x7f2bec1016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:30.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf0969700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bec0ffee0 0x7f2bec1016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf0969700 1 -- 172.21.15.130:0/2069832332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bec0ff520 msgr2=0x7f2bec101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:30.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.027+0000 7f2bf0969700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bec0ff520 0x7f2bec101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.028+0000 7f2bf0969700 1 -- 172.21.15.130:0/2069832332 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be00095f0 con 0x7f2bec104530 2023-09-12T16:01:30.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.028+0000 7f2bf0969700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 0x7f2bec101c40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2be4007f10 tx=0x7f2be400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:30.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.028+0000 7f2be97fa700 1 -- 172.21.15.130:0/2069832332 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4017070 con 0x7f2bec104530 2023-09-12T16:01:30.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.028+0000 7f2be97fa700 1 -- 172.21.15.130:0/2069832332 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be400f040 con 0x7f2bec104530 2023-09-12T16:01:30.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.028+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec1a51e0 con 0x7f2bec104530 2023-09-12T16:01:30.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.028+0000 7f2be97fa700 1 -- 172.21.15.130:0/2069832332 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4005740 con 0x7f2bec104530 2023-09-12T16:01:30.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.028+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec1a5690 con 0x7f2bec104530 2023-09-12T16:01:30.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.029+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec04e300 con 0x7f2bec104530 2023-09-12T16:01:30.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.030+0000 7f2be97fa700 1 -- 172.21.15.130:0/2069832332 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f2be400dc60 con 0x7f2bec104530 2023-09-12T16:01:30.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.030+0000 7f2be97fa700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2bd40a0640 0x7f2bd40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:30.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.031+0000 7f2bebfff700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2bd40a0640 0x7f2bd40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:30.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.031+0000 7f2be97fa700 1 -- 172.21.15.130:0/2069832332 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2be40cae30 con 0x7f2bec104530 2023-09-12T16:01:30.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.031+0000 7f2bebfff700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2bd40a0640 0x7f2bd40a2b00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2be000b5c0 tx=0x7f2be0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:30.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.035+0000 7f2be97fa700 1 -- 172.21.15.130:0/2069832332 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2be4094e00 con 0x7f2bec104530 2023-09-12T16:01:30.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.222+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2bec10f450 con 0x7f2bec104530 2023-09-12T16:01:30.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.222+0000 7f2be97fa700 1 -- 172.21.15.130:0/2069832332 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2be4098420 con 0x7f2bec104530 2023-09-12T16:01:30.223 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:01:30.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2bd40a0640 msgr2=0x7f2bd40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:30.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2bd40a0640 0x7f2bd40a2b00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2be000b5c0 tx=0x7f2be0005fb0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 msgr2=0x7f2bec101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:30.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 0x7f2bec101c40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2be4007f10 tx=0x7f2be400d3b0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 shutdown_connections 2023-09-12T16:01:30.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2bd40a0640 0x7f2bd40a2b00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bec104530 0x7f2bec101c40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bec0ff520 0x7f2bec101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 --2- 172.21.15.130:0/2069832332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bec0ffee0 0x7f2bec1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:30.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 >> 172.21.15.130:0/2069832332 conn(0x7f2bec076600 msgr2=0x7f2bec0681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:30.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 shutdown_connections 2023-09-12T16:01:30.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:30.224+0000 7f2bf23cc700 1 -- 172.21.15.130:0/2069832332 wait complete. 2023-09-12T16:01:30.580 INFO:tasks.cephadm:Adding mds.a on smithi130 2023-09-12T16:01:30.581 INFO:tasks.cephadm:Adding mds.d on smithi130 2023-09-12T16:01:30.581 INFO:tasks.cephadm:Adding mds.g on smithi130 2023-09-12T16:01:30.581 INFO:tasks.cephadm:Adding mds.j on smithi130 2023-09-12T16:01:30.581 INFO:tasks.cephadm:Adding mds.b on smithi159 2023-09-12T16:01:30.582 INFO:tasks.cephadm:Adding mds.e on smithi159 2023-09-12T16:01:30.582 INFO:tasks.cephadm:Adding mds.h on smithi159 2023-09-12T16:01:30.582 INFO:tasks.cephadm:Adding mds.k on smithi159 2023-09-12T16:01:30.582 INFO:tasks.cephadm:Adding mds.c on smithi203 2023-09-12T16:01:30.583 INFO:tasks.cephadm:Adding mds.f on smithi203 2023-09-12T16:01:30.583 INFO:tasks.cephadm:Adding mds.i on smithi203 2023-09-12T16:01:30.583 INFO:tasks.cephadm:Adding mds.l on smithi203 2023-09-12T16:01:30.584 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph orch apply mds all '12;smithi130=a;smithi130=d;smithi130=g;smithi130=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l' 2023-09-12T16:01:30.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:30 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2069832332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:01:30.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:30 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2069832332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:01:30.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:30 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2069832332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:01:31.153 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T16:01:31.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:31 smithi203 ceph-mon[103431]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:31.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:31 smithi130 ceph-mon[98670]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:31.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:31 smithi159 ceph-mon[102522]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:32.484 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.480+0000 7f3d8439a700 1 -- 172.21.15.203:0/1182875022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c100790 msgr2=0x7f3d7c100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:32.484 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.480+0000 7f3d8439a700 1 --2- 172.21.15.203:0/1182875022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c100790 0x7f3d7c100bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009990 tx=0x7f3d78009ca0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.484 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.480+0000 7f3d8439a700 1 -- 172.21.15.203:0/1182875022 shutdown_connections 2023-09-12T16:01:32.484 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.480+0000 7f3d8439a700 1 --2- 172.21.15.203:0/1182875022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c101130 0x7f3d7c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.485 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.480+0000 7f3d8439a700 1 --2- 172.21.15.203:0/1182875022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d7c0ff590 0x7f3d7c0ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.485 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.480+0000 7f3d8439a700 1 --2- 172.21.15.203:0/1182875022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c100790 0x7f3d7c100bf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.485 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.480+0000 7f3d8439a700 1 -- 172.21.15.203:0/1182875022 >> 172.21.15.203:0/1182875022 conn(0x7f3d7c0fab10 msgr2=0x7f3d7c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:32.485 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.480+0000 7f3d8439a700 1 -- 172.21.15.203:0/1182875022 shutdown_connections 2023-09-12T16:01:32.485 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.480+0000 7f3d8439a700 1 -- 172.21.15.203:0/1182875022 wait complete. 2023-09-12T16:01:32.486 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d8439a700 1 Processor -- start 2023-09-12T16:01:32.486 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d8439a700 1 -- start start 2023-09-12T16:01:32.486 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d8439a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c0ff590 0x7f3d7c196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:32.486 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d8439a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d7c100790 0x7f3d7c1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:32.487 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d8439a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c101130 0x7f3d7c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:32.487 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d8439a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c19df10 con 0x7f3d7c0ff590 2023-09-12T16:01:32.487 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d8439a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c19e050 con 0x7f3d7c100790 2023-09-12T16:01:32.487 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d8439a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c19e1e0 con 0x7f3d7c101130 2023-09-12T16:01:32.488 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d82937700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c101130 0x7f3d7c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:32.488 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d82937700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c101130 0x7f3d7c19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:41882/0 (socket says 172.21.15.203:41882) 2023-09-12T16:01:32.488 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.481+0000 7f3d82937700 1 -- 172.21.15.203:0/626845820 learned_addr learned my addr 172.21.15.203:0/626845820 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-09-12T16:01:32.489 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d82136700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c0ff590 0x7f3d7c196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:32.489 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d81935700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d7c100790 0x7f3d7c1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:32.489 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d82937700 1 -- 172.21.15.203:0/626845820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d7c100790 msgr2=0x7f3d7c1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:32.489 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d82937700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d7c100790 0x7f3d7c1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.490 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d82937700 1 -- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c0ff590 msgr2=0x7f3d7c196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:32.490 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d82937700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c0ff590 0x7f3d7c196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.490 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d82937700 1 -- 172.21.15.203:0/626845820 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d780095f0 con 0x7f3d7c101130 2023-09-12T16:01:32.491 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d81935700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d7c100790 0x7f3d7c1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:01:32.491 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d82136700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c0ff590 0x7f3d7c196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:01:32.491 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d82937700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c101130 0x7f3d7c19d940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400add0 tx=0x7f3d7400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:32.491 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d737fe700 1 -- 172.21.15.203:0/626845820 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400f070 con 0x7f3d7c101130 2023-09-12T16:01:32.492 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d737fe700 1 -- 172.21.15.203:0/626845820 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d74007b30 con 0x7f3d7c101130 2023-09-12T16:01:32.492 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c19e4c0 con 0x7f3d7c101130 2023-09-12T16:01:32.492 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d737fe700 1 -- 172.21.15.203:0/626845820 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d740053e0 con 0x7f3d7c101130 2023-09-12T16:01:32.492 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.482+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c19e970 con 0x7f3d7c101130 2023-09-12T16:01:32.492 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.483+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c10ada0 con 0x7f3d7c101130 2023-09-12T16:01:32.493 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.484+0000 7f3d737fe700 1 -- 172.21.15.203:0/626845820 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f3d7400c940 con 0x7f3d7c101130 2023-09-12T16:01:32.493 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.484+0000 7f3d737fe700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d680a0640 0x7f3d680a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:32.494 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.485+0000 7f3d737fe700 1 -- 172.21.15.203:0/626845820 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3d740cb380 con 0x7f3d7c101130 2023-09-12T16:01:32.494 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.485+0000 7f3d82136700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d680a0640 0x7f3d680a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:32.494 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.485+0000 7f3d82136700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d680a0640 0x7f3d680a2b00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c005950 tx=0x7f3d6c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:32.494 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.488+0000 7f3d737fe700 1 -- 172.21.15.203:0/626845820 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3d740953e0 con 0x7f3d7c101130 2023-09-12T16:01:32.684 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.684+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi130=a;smithi130=d;smithi130=g;smithi130=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d7c068950 con 0x7f3d680a0640 2023-09-12T16:01:32.693 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.692+0000 7f3d737fe700 1 -- 172.21.15.203:0/626845820 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f3d7c068950 con 0x7f3d680a0640 2023-09-12T16:01:32.693 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mds.all update... 2023-09-12T16:01:32.695 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.694+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d680a0640 msgr2=0x7f3d680a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:32.695 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.694+0000 7f3d8439a700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d680a0640 0x7f3d680a2b00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c005950 tx=0x7f3d6c0058e0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.695 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.694+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c101130 msgr2=0x7f3d7c19d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.694+0000 7f3d8439a700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c101130 0x7f3d7c19d940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400add0 tx=0x7f3d7400c330 comp rx=0 tx=0).stop 2023-09-12T16:01:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.695+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 shutdown_connections 2023-09-12T16:01:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.695+0000 7f3d8439a700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d680a0640 0x7f3d680a2b00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.695+0000 7f3d8439a700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7c101130 0x7f3d7c19d940 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.695+0000 7f3d8439a700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c0ff590 0x7f3d7c196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.697 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.695+0000 7f3d8439a700 1 --2- 172.21.15.203:0/626845820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d7c100790 0x7f3d7c1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:32.697 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.695+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 >> 172.21.15.203:0/626845820 conn(0x7f3d7c0fab10 msgr2=0x7f3d7c068140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:32.697 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.695+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 shutdown_connections 2023-09-12T16:01:32.697 INFO:teuthology.orchestra.run.smithi203.stderr:2023-09-12T16:01:32.695+0000 7f3d8439a700 1 -- 172.21.15.203:0/626845820 wait complete. 2023-09-12T16:01:33.097 DEBUG:teuthology.orchestra.run.smithi130:mds.a> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.a.service 2023-09-12T16:01:33.100 DEBUG:teuthology.orchestra.run.smithi130:mds.d> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.d.service 2023-09-12T16:01:33.138 INFO:journalctl@ceph.mds.a.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T16:01:33.139 DEBUG:teuthology.orchestra.run.smithi130:mds.g> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.g.service 2023-09-12T16:01:33.168 INFO:journalctl@ceph.mds.d.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T16:01:33.169 DEBUG:teuthology.orchestra.run.smithi130:mds.j> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.j.service 2023-09-12T16:01:33.196 INFO:journalctl@ceph.mds.g.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T16:01:33.197 DEBUG:teuthology.orchestra.run.smithi159:mds.b> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.b.service 2023-09-12T16:01:33.200 DEBUG:teuthology.orchestra.run.smithi159:mds.e> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.e.service 2023-09-12T16:01:33.223 INFO:journalctl@ceph.mds.j.smithi130.stdout:-- Logs begin at Tue 2023-09-12 15:41:02 UTC. -- 2023-09-12T16:01:33.237 INFO:journalctl@ceph.mds.b.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T16:01:33.238 DEBUG:teuthology.orchestra.run.smithi159:mds.h> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.h.service 2023-09-12T16:01:33.268 INFO:journalctl@ceph.mds.e.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T16:01:33.269 DEBUG:teuthology.orchestra.run.smithi159:mds.k> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.k.service 2023-09-12T16:01:33.296 INFO:journalctl@ceph.mds.h.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T16:01:33.297 DEBUG:teuthology.orchestra.run.smithi203:mds.c> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.c.service 2023-09-12T16:01:33.300 DEBUG:teuthology.orchestra.run.smithi203:mds.f> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.f.service 2023-09-12T16:01:33.324 INFO:journalctl@ceph.mds.k.smithi159.stdout:-- Logs begin at Tue 2023-09-12 15:41:19 UTC. -- 2023-09-12T16:01:33.329 INFO:journalctl@ceph.mds.c.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T16:01:33.331 DEBUG:teuthology.orchestra.run.smithi203:mds.i> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.i.service 2023-09-12T16:01:33.357 INFO:journalctl@ceph.mds.f.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T16:01:33.358 DEBUG:teuthology.orchestra.run.smithi203:mds.l> sudo journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.l.service 2023-09-12T16:01:33.383 INFO:journalctl@ceph.mds.i.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T16:01:33.384 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-09-12T16:01:33.384 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:01:33.409 INFO:journalctl@ceph.mds.l.smithi203.stdout:-- Logs begin at Tue 2023-09-12 15:41:26 UTC. -- 2023-09-12T16:01:33.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.533+0000 7f0ba3017700 1 -- 172.21.15.130:0/3108539976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c1006d0 msgr2=0x7f0b9c100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:33.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.533+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3108539976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c1006d0 0x7f0b9c100b30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b880099c0 tx=0x7f0b88009cd0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 -- 172.21.15.130:0/3108539976 shutdown_connections 2023-09-12T16:01:33.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3108539976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b9c101070 0x7f0b9c107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3108539976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b9c0ff4e0 0x7f0b9c0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3108539976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c1006d0 0x7f0b9c100b30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 -- 172.21.15.130:0/3108539976 >> 172.21.15.130:0/3108539976 conn(0x7f0b9c0faa30 msgr2=0x7f0b9c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:33.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 -- 172.21.15.130:0/3108539976 shutdown_connections 2023-09-12T16:01:33.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 -- 172.21.15.130:0/3108539976 wait complete. 2023-09-12T16:01:33.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 Processor -- start 2023-09-12T16:01:33.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 -- start start 2023-09-12T16:01:33.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b9c0ff4e0 0x7f0b9c1951f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:33.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b9c1006d0 0x7f0b9c195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:33.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c101070 0x7f0b9c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:33.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c19cd70 con 0x7f0b9c1006d0 2023-09-12T16:01:33.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c19ceb0 con 0x7f0b9c101070 2023-09-12T16:01:33.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba3017700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c19d040 con 0x7f0b9c0ff4e0 2023-09-12T16:01:33.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba0db3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b9c0ff4e0 0x7f0b9c1951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:33.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0ba15b4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c101070 0x7f0b9c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:33.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.534+0000 7f0b9bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b9c1006d0 0x7f0b9c195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:33.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba0db3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b9c0ff4e0 0x7f0b9c1951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:60782/0 (socket says 172.21.15.130:60782) 2023-09-12T16:01:33.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba15b4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c101070 0x7f0b9c19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:36200/0 (socket says 172.21.15.130:36200) 2023-09-12T16:01:33.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba0db3700 1 -- 172.21.15.130:0/3458329604 learned_addr learned my addr 172.21.15.130:0/3458329604 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:33.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0b9bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b9c1006d0 0x7f0b9c195730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52940/0 (socket says 172.21.15.130:52940) 2023-09-12T16:01:33.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba15b4700 1 -- 172.21.15.130:0/3458329604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b9c0ff4e0 msgr2=0x7f0b9c1951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:33.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba15b4700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b9c0ff4e0 0x7f0b9c1951f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba15b4700 1 -- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b9c1006d0 msgr2=0x7f0b9c195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:33.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba15b4700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b9c1006d0 0x7f0b9c195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba15b4700 1 -- 172.21.15.130:0/3458329604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b90009520 con 0x7f0b9c101070 2023-09-12T16:01:33.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0b9bfff700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b9c1006d0 0x7f0b9c195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:01:33.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba15b4700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c101070 0x7f0b9c19c7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400efc0 tx=0x7f0b9400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:33.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0b99ffb700 1 -- 172.21.15.130:0/3458329604 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94018070 con 0x7f0b9c101070 2023-09-12T16:01:33.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b880095f0 con 0x7f0b9c101070 2023-09-12T16:01:33.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0b99ffb700 1 -- 172.21.15.130:0/3458329604 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9400f040 con 0x7f0b9c101070 2023-09-12T16:01:33.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0b99ffb700 1 -- 172.21.15.130:0/3458329604 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400cd40 con 0x7f0b9c101070 2023-09-12T16:01:33.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.535+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c067ae0 con 0x7f0b9c101070 2023-09-12T16:01:33.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.536+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c0684d0 con 0x7f0b9c101070 2023-09-12T16:01:33.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.537+0000 7f0b99ffb700 1 -- 172.21.15.130:0/3458329604 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f0b94003a00 con 0x7f0b9c101070 2023-09-12T16:01:33.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.538+0000 7f0b99ffb700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0b8c0a0690 0x7f0b8c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:33.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.538+0000 7f0ba0db3700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0b8c0a0690 0x7f0b8c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:33.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.538+0000 7f0b99ffb700 1 -- 172.21.15.130:0/3458329604 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0b940cb900 con 0x7f0b9c101070 2023-09-12T16:01:33.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.538+0000 7f0ba0db3700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0b8c0a0690 0x7f0b8c0a2b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005950 tx=0x7f0b90010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:33.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.541+0000 7f0b99ffb700 1 -- 172.21.15.130:0/3458329604 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0b94095960 con 0x7f0b9c101070 2023-09-12T16:01:33.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.732+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b9c10ad00 con 0x7f0b9c101070 2023-09-12T16:01:33.734 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.734+0000 7f0b99ffb700 1 -- 172.21.15.130:0/3458329604 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f0b94098f80 con 0x7f0b9c101070 2023-09-12T16:01:33.735 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:33.735 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":77,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:01:23.890141+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:01:33.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0b8c0a0690 msgr2=0x7f0b8c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:33.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0b8c0a0690 0x7f0b8c0a2b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005950 tx=0x7f0b90010040 comp rx=0 tx=0).stop 2023-09-12T16:01:33.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c101070 msgr2=0x7f0b9c19c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:33.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c101070 0x7f0b9c19c7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400efc0 tx=0x7f0b9400c3a0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 shutdown_connections 2023-09-12T16:01:33.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0b8c0a0690 0x7f0b8c0a2b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b9c101070 0x7f0b9c19c7a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b9c0ff4e0 0x7f0b9c1951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 --2- 172.21.15.130:0/3458329604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b9c1006d0 0x7f0b9c195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 >> 172.21.15.130:0/3458329604 conn(0x7f0b9c0faa30 msgr2=0x7f0b9c103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:33.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 shutdown_connections 2023-09-12T16:01:33.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.736+0000 7f0ba3017700 1 -- 172.21.15.130:0/3458329604 wait complete. 2023-09-12T16:01:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T16:01:33.750 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:01:33.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.873+0000 7f9abff5f700 1 -- 172.21.15.130:0/2141488092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab8101080 msgr2=0x7f9ab8107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:33.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.873+0000 7f9abff5f700 1 --2- 172.21.15.130:0/2141488092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab8101080 0x7f9ab8107930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b0d0 tx=0x7f9ab000b3e0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.873+0000 7f9abff5f700 1 -- 172.21.15.130:0/2141488092 shutdown_connections 2023-09-12T16:01:33.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.873+0000 7f9abff5f700 1 --2- 172.21.15.130:0/2141488092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab8101080 0x7f9ab8107930 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.873+0000 7f9abff5f700 1 --2- 172.21.15.130:0/2141488092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab80ff4f0 0x7f9ab80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.873+0000 7f9abff5f700 1 --2- 172.21.15.130:0/2141488092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab81006e0 0x7f9ab8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.873+0000 7f9abff5f700 1 -- 172.21.15.130:0/2141488092 >> 172.21.15.130:0/2141488092 conn(0x7f9ab80faa60 msgr2=0x7f9ab80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:33.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.873+0000 7f9abff5f700 1 -- 172.21.15.130:0/2141488092 shutdown_connections 2023-09-12T16:01:33.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.873+0000 7f9abff5f700 1 -- 172.21.15.130:0/2141488092 wait complete. 2023-09-12T16:01:33.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abff5f700 1 Processor -- start 2023-09-12T16:01:33.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abff5f700 1 -- start start 2023-09-12T16:01:33.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abff5f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab80ff4f0 0x7f9ab80682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:33.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abff5f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab81006e0 0x7f9ab819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:33.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abff5f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab8101080 0x7f9ab819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:33.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abff5f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab819df00 con 0x7f9ab8101080 2023-09-12T16:01:33.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abff5f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab819e070 con 0x7f9ab81006e0 2023-09-12T16:01:33.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abff5f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab819e230 con 0x7f9ab80ff4f0 2023-09-12T16:01:33.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abdcfb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab80ff4f0 0x7f9ab80682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:33.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abe4fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab8101080 0x7f9ab819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:33.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abdcfb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab80ff4f0 0x7f9ab80682f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:60796/0 (socket says 172.21.15.130:60796) 2023-09-12T16:01:33.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abdcfb700 1 -- 172.21.15.130:0/427037922 learned_addr learned my addr 172.21.15.130:0/427037922 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:33.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abd4fa700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab81006e0 0x7f9ab819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:33.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abe4fc700 1 -- 172.21.15.130:0/427037922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab80ff4f0 msgr2=0x7f9ab80682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:33.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abe4fc700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab80ff4f0 0x7f9ab80682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abe4fc700 1 -- 172.21.15.130:0/427037922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab81006e0 msgr2=0x7f9ab819b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:33.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abe4fc700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab81006e0 0x7f9ab819b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:33.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abe4fc700 1 -- 172.21.15.130:0/427037922 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab0009d00 con 0x7f9ab8101080 2023-09-12T16:01:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abd4fa700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab81006e0 0x7f9ab819b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:01:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.874+0000 7f9abe4fc700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab8101080 0x7f9ab819d730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b820 tx=0x7f9ab00094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.875+0000 7f9aaeffd700 1 -- 172.21.15.130:0/427037922 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000e040 con 0x7f9ab8101080 2023-09-12T16:01:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.875+0000 7f9aaeffd700 1 -- 172.21.15.130:0/427037922 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ab0012dc0 con 0x7f9ab8101080 2023-09-12T16:01:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.875+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab819e4b0 con 0x7f9ab8101080 2023-09-12T16:01:33.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.875+0000 7f9aaeffd700 1 -- 172.21.15.130:0/427037922 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab001de90 con 0x7f9ab8101080 2023-09-12T16:01:33.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.875+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab819e930 con 0x7f9ab8101080 2023-09-12T16:01:33.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.875+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab8106420 con 0x7f9ab8101080 2023-09-12T16:01:33.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.877+0000 7f9aaeffd700 1 -- 172.21.15.130:0/427037922 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f9ab001d580 con 0x7f9ab8101080 2023-09-12T16:01:33.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.877+0000 7f9aaeffd700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa40a0640 0x7f9aa40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:33.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.878+0000 7f9abdcfb700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa40a0640 0x7f9aa40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:33.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.878+0000 7f9aaeffd700 1 -- 172.21.15.130:0/427037922 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9ab00cdbb0 con 0x7f9ab8101080 2023-09-12T16:01:33.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.878+0000 7f9abdcfb700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa40a0640 0x7f9aa40a2b00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800bd70 tx=0x7f9aa800b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:33.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:33.880+0000 7f9aaeffd700 1 -- 172.21.15.130:0/427037922 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9ab0097c10 con 0x7f9ab8101080 2023-09-12T16:01:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi130=a;smithi130=d;smithi130=g;smithi130=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:01:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: Saving service mds.all spec with placement smithi130=a;smithi130=d;smithi130=g;smithi130=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2023-09-12T16:01:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:01:33.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:33.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:33.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:33.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:33.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:33.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:33.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:33.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:33 smithi130 ceph-mon[98670]: Deploying daemon mds.l on smithi203 2023-09-12T16:01:33.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi130=a;smithi130=d;smithi130=g;smithi130=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:01:33.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: Saving service mds.all spec with placement smithi130=a;smithi130=d;smithi130=g;smithi130=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2023-09-12T16:01:33.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:33.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:01:33.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:33.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:33.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:33.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:33.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:33.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:33.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:33.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:33 smithi159 ceph-mon[102522]: Deploying daemon mds.l on smithi203 2023-09-12T16:01:33.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi130=a;smithi130=d;smithi130=g;smithi130=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:01:33.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: Saving service mds.all spec with placement smithi130=a;smithi130=d;smithi130=g;smithi130=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2023-09-12T16:01:33.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:33.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:01:33.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:33.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:01:33.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:33.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:33.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:33.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:33.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:33.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:33 smithi203 ceph-mon[103431]: Deploying daemon mds.l on smithi203 2023-09-12T16:01:34.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.070+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ab810ae90 con 0x7f9ab8101080 2023-09-12T16:01:34.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.070+0000 7f9aaeffd700 1 -- 172.21.15.130:0/427037922 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ab009b230 con 0x7f9ab8101080 2023-09-12T16:01:34.071 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:01:34.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.072+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa40a0640 msgr2=0x7f9aa40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:34.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.072+0000 7f9abff5f700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa40a0640 0x7f9aa40a2b00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800bd70 tx=0x7f9aa800b4a0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.072+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab8101080 msgr2=0x7f9ab819d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:34.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.072+0000 7f9abff5f700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab8101080 0x7f9ab819d730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b820 tx=0x7f9ab00094c0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.072+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 shutdown_connections 2023-09-12T16:01:34.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.072+0000 7f9abff5f700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa40a0640 0x7f9aa40a2b00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.072+0000 7f9abff5f700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab8101080 0x7f9ab819d730 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.072+0000 7f9abff5f700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab80ff4f0 0x7f9ab80682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.073+0000 7f9abff5f700 1 --2- 172.21.15.130:0/427037922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab81006e0 0x7f9ab819b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.073+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 >> 172.21.15.130:0/427037922 conn(0x7f9ab80faa60 msgr2=0x7f9ab80fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:34.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.073+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 shutdown_connections 2023-09-12T16:01:34.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.073+0000 7f9abff5f700 1 -- 172.21.15.130:0/427037922 wait complete. 2023-09-12T16:01:34.085 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:34.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.207+0000 7f9bf75d7700 1 -- 172.21.15.130:0/921000344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 msgr2=0x7f9bf0107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:34.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.207+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/921000344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 0x7f9bf0107950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9be800b0a0 tx=0x7f9be800b3b0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.207+0000 7f9bf75d7700 1 -- 172.21.15.130:0/921000344 shutdown_connections 2023-09-12T16:01:34.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.207+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/921000344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 0x7f9bf0107950 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.207+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/921000344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf0100aa0 0x7f9bf0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.207+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/921000344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf00ff6f0 0x7f9bf00ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.207+0000 7f9bf75d7700 1 -- 172.21.15.130:0/921000344 >> 172.21.15.130:0/921000344 conn(0x7f9bf00fa1c0 msgr2=0x7f9bf00fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:34.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.207+0000 7f9bf75d7700 1 -- 172.21.15.130:0/921000344 shutdown_connections 2023-09-12T16:01:34.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.207+0000 7f9bf75d7700 1 -- 172.21.15.130:0/921000344 wait complete. 2023-09-12T16:01:34.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf75d7700 1 Processor -- start 2023-09-12T16:01:34.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf75d7700 1 -- start start 2023-09-12T16:01:34.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf75d7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf00ff6f0 0x7f9bf00ff170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:34.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf75d7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 0x7f9bf019b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:34.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf75d7700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf0100aa0 0x7f9bf019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:34.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf75d7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf019df00 con 0x7f9bf01000b0 2023-09-12T16:01:34.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf75d7700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf019e070 con 0x7f9bf0100aa0 2023-09-12T16:01:34.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf75d7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf019e230 con 0x7f9bf00ff6f0 2023-09-12T16:01:34.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf4b72700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 0x7f9bf019b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:34.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf4b72700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 0x7f9bf019b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52966/0 (socket says 172.21.15.130:52966) 2023-09-12T16:01:34.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf4b72700 1 -- 172.21.15.130:0/4062673623 learned_addr learned my addr 172.21.15.130:0/4062673623 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:34.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf5373700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf00ff6f0 0x7f9bf00ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:34.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf5b74700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf0100aa0 0x7f9bf019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:34.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf4b72700 1 -- 172.21.15.130:0/4062673623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf00ff6f0 msgr2=0x7f9bf00ff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:34.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf4b72700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf00ff6f0 0x7f9bf00ff170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf4b72700 1 -- 172.21.15.130:0/4062673623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf0100aa0 msgr2=0x7f9bf019d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:34.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf4b72700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf0100aa0 0x7f9bf019d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf4b72700 1 -- 172.21.15.130:0/4062673623 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be8009d00 con 0x7f9bf01000b0 2023-09-12T16:01:34.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf5b74700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf0100aa0 0x7f9bf019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:01:34.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.208+0000 7f9bf4b72700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 0x7f9bf019b0e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00d980 tx=0x7f9bec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:34.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.209+0000 7f9be67fc700 1 -- 172.21.15.130:0/4062673623 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec004220 con 0x7f9bf01000b0 2023-09-12T16:01:34.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.209+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf019e510 con 0x7f9bf01000b0 2023-09-12T16:01:34.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.209+0000 7f9be67fc700 1 -- 172.21.15.130:0/4062673623 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bec007640 con 0x7f9bf01000b0 2023-09-12T16:01:34.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.209+0000 7f9be67fc700 1 -- 172.21.15.130:0/4062673623 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec005610 con 0x7f9bf01000b0 2023-09-12T16:01:34.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.209+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf019ea20 con 0x7f9bf01000b0 2023-09-12T16:01:34.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.210+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf010ad60 con 0x7f9bf01000b0 2023-09-12T16:01:34.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.211+0000 7f9be67fc700 1 -- 172.21.15.130:0/4062673623 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f9bec004750 con 0x7f9bf01000b0 2023-09-12T16:01:34.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.211+0000 7f9be67fc700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9bdc0a0690 0x7f9bdc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:34.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.211+0000 7f9bf5373700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9bdc0a0690 0x7f9bdc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:34.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.211+0000 7f9be67fc700 1 -- 172.21.15.130:0/4062673623 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9bec0cb5d0 con 0x7f9bf01000b0 2023-09-12T16:01:34.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.211+0000 7f9bf5373700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9bdc0a0690 0x7f9bdc0a2b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009a10 tx=0x7f9be0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:34.218 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.215+0000 7f9be67fc700 1 -- 172.21.15.130:0/4062673623 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9bec0955a0 con 0x7f9bf01000b0 2023-09-12T16:01:34.366 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:01:34 smithi203 systemd[1]: Starting Ceph mds.l for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:34.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.448+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bf010af20 con 0x7f9bf01000b0 2023-09-12T16:01:34.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.448+0000 7f9be67fc700 1 -- 172.21.15.130:0/4062673623 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f9bec098bc0 con 0x7f9bf01000b0 2023-09-12T16:01:34.449 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:34.449 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-09-12T16:01:34.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.450+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9bdc0a0690 msgr2=0x7f9bdc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:34.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.450+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9bdc0a0690 0x7f9bdc0a2b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009a10 tx=0x7f9be0009190 comp rx=0 tx=0).stop 2023-09-12T16:01:34.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.450+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 msgr2=0x7f9bf019b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:34.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.450+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 0x7f9bf019b0e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00d980 tx=0x7f9bec00dc90 comp rx=0 tx=0).stop 2023-09-12T16:01:34.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.451+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 shutdown_connections 2023-09-12T16:01:34.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.451+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9bdc0a0690 0x7f9bdc0a2b50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.451+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf0100aa0 0x7f9bf019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.451+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf00ff6f0 0x7f9bf00ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.451+0000 7f9bf75d7700 1 --2- 172.21.15.130:0/4062673623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bf01000b0 0x7f9bf019b0e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:34.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.451+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 >> 172.21.15.130:0/4062673623 conn(0x7f9bf00fa1c0 msgr2=0x7f9bf00fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:34.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.451+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 shutdown_connections 2023-09-12T16:01:34.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:34.451+0000 7f9bf75d7700 1 -- 172.21.15.130:0/4062673623 wait complete. 2023-09-12T16:01:34.454 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 1 2023-09-12T16:01:34.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:34 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3458329604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:01:34.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:34 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/427037922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:01:34.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:34 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4062673623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:34.962 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:34 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3458329604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:01:34.962 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:34 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/427037922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:01:34.963 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:34 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4062673623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:34.963 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:01:34 smithi203 podman[139870]: 2023-09-12T16:01:34.963 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:01:34 smithi203 ceph-mds[139910]: -- 172.21.15.203:0/3098175661 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556ea7afd1e0 con 0x556ea7ad2800 2023-09-12T16:01:34.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:34 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3458329604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:01:34.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:34 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/427037922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:01:34.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:34 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4062673623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:35.245 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:01:34 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-l[139886]: starting mds.l at 2023-09-12T16:01:35.245 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:01:34 smithi203 bash[139870]: 234b96c09b89f441796adccb04a731b5a6a119b876e448d578f14d8c78bf35ca 2023-09-12T16:01:35.246 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:01:35 smithi203 systemd[1]: Started Ceph mds.l for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:36.084 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:36.084 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.203:6832/2012369878,v1:172.21.15.203:6833/2012369878] up:boot 2023-09-12T16:01:36.084 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: fsmap 1 up:standby 2023-09-12T16:01:36.084 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-12T16:01:36.085 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:36.085 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:36.085 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:36.085 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:36.085 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:36.086 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:35 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:36.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.203:6832/2012369878,v1:172.21.15.203:6833/2012369878] up:boot 2023-09-12T16:01:36.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: fsmap 1 up:standby 2023-09-12T16:01:36.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-12T16:01:36.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:36.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:36.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:36.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:36.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:36.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:35 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:36.464 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:36.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:36.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.203:6832/2012369878,v1:172.21.15.203:6833/2012369878] up:boot 2023-09-12T16:01:36.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: fsmap 1 up:standby 2023-09-12T16:01:36.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-12T16:01:36.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:36.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:36.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:36.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:36.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:36.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:35 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:36.602 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.599+0000 7f0a052ee700 1 -- 172.21.15.130:0/2661765078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a00100720 msgr2=0x7f0a00100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:36.602 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.599+0000 7f0a052ee700 1 --2- 172.21.15.130:0/2661765078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a00100720 0x7f0a00100b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f09f40099c0 tx=0x7f09f4009cd0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.603 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.599+0000 7f0a052ee700 1 -- 172.21.15.130:0/2661765078 shutdown_connections 2023-09-12T16:01:36.603 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.599+0000 7f0a052ee700 1 --2- 172.21.15.130:0/2661765078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a001010c0 0x7f0a00107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.603 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.599+0000 7f0a052ee700 1 --2- 172.21.15.130:0/2661765078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a000ff530 0x7f0a000ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.599+0000 7f0a052ee700 1 --2- 172.21.15.130:0/2661765078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a00100720 0x7f0a00100b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.599+0000 7f0a052ee700 1 -- 172.21.15.130:0/2661765078 >> 172.21.15.130:0/2661765078 conn(0x7f0a000faa80 msgr2=0x7f0a000fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:36.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.599+0000 7f0a052ee700 1 -- 172.21.15.130:0/2661765078 shutdown_connections 2023-09-12T16:01:36.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.599+0000 7f0a052ee700 1 -- 172.21.15.130:0/2661765078 wait complete. 2023-09-12T16:01:36.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.600+0000 7f0a052ee700 1 Processor -- start 2023-09-12T16:01:36.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.600+0000 7f0a052ee700 1 -- start start 2023-09-12T16:01:36.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.600+0000 7f0a052ee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a000ff530 0x7f0a00069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:36.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.600+0000 7f0a052ee700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00100720 0x7f0a0019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:36.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.600+0000 7f0a052ee700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a001010c0 0x7f0a0019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:36.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.600+0000 7f0a052ee700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0019dd80 con 0x7f0a001010c0 2023-09-12T16:01:36.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.600+0000 7f0a052ee700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0019def0 con 0x7f0a00100720 2023-09-12T16:01:36.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.600+0000 7f0a052ee700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0019e0b0 con 0x7f0a000ff530 2023-09-12T16:01:36.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09ff7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a001010c0 0x7f0a0019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:36.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09ff7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a001010c0 0x7f0a0019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52980/0 (socket says 172.21.15.130:52980) 2023-09-12T16:01:36.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09ff7fe700 1 -- 172.21.15.130:0/828515337 learned_addr learned my addr 172.21.15.130:0/828515337 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:36.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09fe7fc700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00100720 0x7f0a0019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:36.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09feffd700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a000ff530 0x7f0a00069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:36.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09ff7fe700 1 -- 172.21.15.130:0/828515337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a000ff530 msgr2=0x7f0a00069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:36.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09ff7fe700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a000ff530 0x7f0a00069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09ff7fe700 1 -- 172.21.15.130:0/828515337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00100720 msgr2=0x7f0a0019af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:36.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09ff7fe700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00100720 0x7f0a0019af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09ff7fe700 1 -- 172.21.15.130:0/828515337 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f40095f0 con 0x7f0a001010c0 2023-09-12T16:01:36.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09ff7fe700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a001010c0 0x7f0a0019d5b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09f800ef30 tx=0x7f09f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:36.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09e7fff700 1 -- 172.21.15.130:0/828515337 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8018070 con 0x7f0a001010c0 2023-09-12T16:01:36.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0019e330 con 0x7f0a001010c0 2023-09-12T16:01:36.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09e7fff700 1 -- 172.21.15.130:0/828515337 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f800f040 con 0x7f0a001010c0 2023-09-12T16:01:36.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f09e7fff700 1 -- 172.21.15.130:0/828515337 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f800cd00 con 0x7f0a001010c0 2023-09-12T16:01:36.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.601+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0019e810 con 0x7f0a001010c0 2023-09-12T16:01:36.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.602+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a00106a70 con 0x7f0a001010c0 2023-09-12T16:01:36.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.603+0000 7f09e7fff700 1 -- 172.21.15.130:0/828515337 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f09f8003a00 con 0x7f0a001010c0 2023-09-12T16:01:36.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.604+0000 7f09e7fff700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e80a0690 0x7f09e80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:36.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.604+0000 7f09feffd700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e80a0690 0x7f09e80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:36.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.604+0000 7f09e7fff700 1 -- 172.21.15.130:0/828515337 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09f80cbc50 con 0x7f0a001010c0 2023-09-12T16:01:36.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.604+0000 7f09feffd700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e80a0690 0x7f09e80a2b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a00100580 tx=0x7f09f000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:36.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.607+0000 7f09e7fff700 1 -- 172.21.15.130:0/828515337 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f09f8095cb0 con 0x7f0a001010c0 2023-09-12T16:01:36.620 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:01:36 smithi159 systemd[1]: Starting Ceph mds.k for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:36.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.842+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a0010ad50 con 0x7f0a001010c0 2023-09-12T16:01:36.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.842+0000 7f09e7fff700 1 -- 172.21.15.130:0/828515337 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f09f80992d0 con 0x7f0a001010c0 2023-09-12T16:01:36.843 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:36.844 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-12T16:01:36.845 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.844+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e80a0690 msgr2=0x7f09e80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:36.845 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.844+0000 7f0a052ee700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e80a0690 0x7f09e80a2b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a00100580 tx=0x7f09f000b410 comp rx=0 tx=0).stop 2023-09-12T16:01:36.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.844+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a001010c0 msgr2=0x7f0a0019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:36.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.844+0000 7f0a052ee700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a001010c0 0x7f0a0019d5b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09f800ef30 tx=0x7f09f800c3a0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.845+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 shutdown_connections 2023-09-12T16:01:36.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.845+0000 7f0a052ee700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f09e80a0690 0x7f09e80a2b50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.845+0000 7f0a052ee700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a001010c0 0x7f0a0019d5b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.845+0000 7f0a052ee700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a000ff530 0x7f0a00069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.845+0000 7f0a052ee700 1 --2- 172.21.15.130:0/828515337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00100720 0x7f0a0019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:36.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.845+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 >> 172.21.15.130:0/828515337 conn(0x7f0a000faa80 msgr2=0x7f0a00103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:36.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.845+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 shutdown_connections 2023-09-12T16:01:36.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:36.845+0000 7f0a052ee700 1 -- 172.21.15.130:0/828515337 wait complete. 2023-09-12T16:01:36.848 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 2 2023-09-12T16:01:37.140 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:36 smithi159 ceph-mon[102522]: Deploying daemon mds.k on smithi159 2023-09-12T16:01:37.140 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:36 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/828515337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:37.140 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:01:36 smithi159 podman[138133]: 2023-09-12T16:01:37.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:36 smithi130 ceph-mon[98670]: Deploying daemon mds.k on smithi159 2023-09-12T16:01:37.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:36 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/828515337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:37.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:36 smithi203 ceph-mon[103431]: Deploying daemon mds.k on smithi159 2023-09-12T16:01:37.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:36 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/828515337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:37.475 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:01:37 smithi159 ceph-mds[138174]: -- 172.21.15.159:0/3188054530 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55583b3131e0 con 0x55583b2e8c00 2023-09-12T16:01:37.476 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:01:37 smithi159 bash[138133]: f741ecea79c8f46096aa9b87b9eee3ca5b74c009251bf6b85462461075edaf84 2023-09-12T16:01:37.476 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:01:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-k[138149]: starting mds.k at 2023-09-12T16:01:37.476 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:01:37 smithi159 systemd[1]: Started Ceph mds.k for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:38.115 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:37 smithi130 ceph-mon[98670]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:38.115 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:38.116 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:38.116 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:38.116 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:38.116 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:38.116 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:37 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:38.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:37 smithi203 ceph-mon[103431]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:38.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:37 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:38.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:37 smithi159 ceph-mon[102522]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:38.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:38.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:37 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:38.725 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:01:38 smithi130 systemd[1]: Starting Ceph mds.j for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:38.859 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:38.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.995+0000 7fb6becdf700 1 -- 172.21.15.130:0/2286580740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 msgr2=0x7fb6b8100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:39.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.995+0000 7fb6becdf700 1 --2- 172.21.15.130:0/2286580740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 0x7fb6b8100b40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40099c0 tx=0x7fb6a4009cd0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 -- 172.21.15.130:0/2286580740 shutdown_connections 2023-09-12T16:01:39.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 --2- 172.21.15.130:0/2286580740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6b8101080 0x7fb6b8107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 --2- 172.21.15.130:0/2286580740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b80ff4f0 0x7fb6b80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 --2- 172.21.15.130:0/2286580740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 0x7fb6b8100b40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 -- 172.21.15.130:0/2286580740 >> 172.21.15.130:0/2286580740 conn(0x7fb6b80faa20 msgr2=0x7fb6b80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:39.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 -- 172.21.15.130:0/2286580740 shutdown_connections 2023-09-12T16:01:39.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 -- 172.21.15.130:0/2286580740 wait complete. 2023-09-12T16:01:39.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 Processor -- start 2023-09-12T16:01:39.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 -- start start 2023-09-12T16:01:39.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6b80ff4f0 0x7fb6b8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:39.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 0x7fb6b819af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:39.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b8101080 0x7fb6b819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:39.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b819dd70 con 0x7fb6b81006e0 2023-09-12T16:01:39.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b819dee0 con 0x7fb6b80ff4f0 2023-09-12T16:01:39.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.996+0000 7fb6becdf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b819e0a0 con 0x7fb6b8101080 2023-09-12T16:01:39.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 0x7fb6b819af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:39.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 0x7fb6b819af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52994/0 (socket says 172.21.15.130:52994) 2023-09-12T16:01:39.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6b7fff700 1 -- 172.21.15.130:0/561452854 learned_addr learned my addr 172.21.15.130:0/561452854 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:39.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6bd27c700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b8101080 0x7fb6b819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:39.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6bca7b700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6b80ff4f0 0x7fb6b8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:39.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6b7fff700 1 -- 172.21.15.130:0/561452854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b8101080 msgr2=0x7fb6b819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:39.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6b7fff700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b8101080 0x7fb6b819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6b7fff700 1 -- 172.21.15.130:0/561452854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6b80ff4f0 msgr2=0x7fb6b8069300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:39.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6b7fff700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6b80ff4f0 0x7fb6b8069300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6b7fff700 1 -- 172.21.15.130:0/561452854 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a40095f0 con 0x7fb6b81006e0 2023-09-12T16:01:39.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:38.997+0000 7fb6b7fff700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 0x7fb6b819af90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009990 tx=0x7fb6a400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:39.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.002+0000 7fb6b5ffb700 1 -- 172.21.15.130:0/561452854 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a401d070 con 0x7fb6b81006e0 2023-09-12T16:01:39.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.002+0000 7fb6b5ffb700 1 -- 172.21.15.130:0/561452854 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6a4005030 con 0x7fb6b81006e0 2023-09-12T16:01:39.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.002+0000 7fb6b5ffb700 1 -- 172.21.15.130:0/561452854 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4004b30 con 0x7fb6b81006e0 2023-09-12T16:01:39.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.002+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b819e320 con 0x7fb6b81006e0 2023-09-12T16:01:39.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.002+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b819e6f0 con 0x7fb6b81006e0 2023-09-12T16:01:39.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.003+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b81069f0 con 0x7fb6b81006e0 2023-09-12T16:01:39.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.004+0000 7fb6b5ffb700 1 -- 172.21.15.130:0/561452854 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fb6a4005410 con 0x7fb6b81006e0 2023-09-12T16:01:39.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.005+0000 7fb6b5ffb700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb6a80a0690 0x7fb6a80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:39.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.005+0000 7fb6b5ffb700 1 -- 172.21.15.130:0/561452854 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb6a40cc950 con 0x7fb6b81006e0 2023-09-12T16:01:39.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.005+0000 7fb6bca7b700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb6a80a0690 0x7fb6a80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:39.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.005+0000 7fb6bca7b700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb6a80a0690 0x7fb6a80a2b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8100540 tx=0x7fb6ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:39.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.008+0000 7fb6b5ffb700 1 -- 172.21.15.130:0/561452854 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb6a40969b0 con 0x7fb6b81006e0 2023-09-12T16:01:39.139 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:01:38 smithi130 podman[152930]: 2023-09-12T16:01:39.139 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:38 smithi130 ceph-mon[98670]: Deploying daemon mds.j on smithi130 2023-09-12T16:01:39.140 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:38 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.159:6832/2681608700,v1:172.21.15.159:6833/2681608700] up:boot 2023-09-12T16:01:39.140 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:38 smithi130 ceph-mon[98670]: fsmap 2 up:standby 2023-09-12T16:01:39.140 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:38 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-12T16:01:39.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.262+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6b802cf70 con 0x7fb6b81006e0 2023-09-12T16:01:39.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.265+0000 7fb6b5ffb700 1 -- 172.21.15.130:0/561452854 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fb6a4099fd0 con 0x7fb6b81006e0 2023-09-12T16:01:39.265 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:39.266 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-12T16:01:39.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb6a80a0690 msgr2=0x7fb6a80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:39.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb6a80a0690 0x7fb6a80a2b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8100540 tx=0x7fb6ac009260 comp rx=0 tx=0).stop 2023-09-12T16:01:39.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 msgr2=0x7fb6b819af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:39.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 0x7fb6b819af90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009990 tx=0x7fb6a400f690 comp rx=0 tx=0).stop 2023-09-12T16:01:39.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 shutdown_connections 2023-09-12T16:01:39.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb6a80a0690 0x7fb6a80a2b50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b8101080 0x7fb6b819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6b80ff4f0 0x7fb6b8069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 --2- 172.21.15.130:0/561452854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6b81006e0 0x7fb6b819af90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:39.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 >> 172.21.15.130:0/561452854 conn(0x7fb6b80faa20 msgr2=0x7fb6b8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:39.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 shutdown_connections 2023-09-12T16:01:39.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:39.267+0000 7fb6becdf700 1 -- 172.21.15.130:0/561452854 wait complete. 2023-09-12T16:01:39.271 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 3 2023-09-12T16:01:39.475 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:01:39 smithi130 bash[152930]: 5a7e5e9315277cf9ff6b63b36d8e600c54662d18f7e2f75ef61ee6aa1a2007f9 2023-09-12T16:01:39.475 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:01:39 smithi130 ceph-mds[153012]: -- 172.21.15.130:0/4084448090 <== mon.0 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556a980011e0 con 0x556a97fd6800 2023-09-12T16:01:39.476 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:01:39 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-j[152968]: starting mds.j at 2023-09-12T16:01:39.476 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:01:39 smithi130 systemd[1]: Started Ceph mds.j for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:39.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:38 smithi159 ceph-mon[102522]: Deploying daemon mds.j on smithi130 2023-09-12T16:01:39.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:38 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.159:6832/2681608700,v1:172.21.15.159:6833/2681608700] up:boot 2023-09-12T16:01:39.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:38 smithi159 ceph-mon[102522]: fsmap 2 up:standby 2023-09-12T16:01:39.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:38 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-12T16:01:39.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:38 smithi203 ceph-mon[103431]: Deploying daemon mds.j on smithi130 2023-09-12T16:01:39.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:38 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.159:6832/2681608700,v1:172.21.15.159:6833/2681608700] up:boot 2023-09-12T16:01:39.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:38 smithi203 ceph-mon[103431]: fsmap 2 up:standby 2023-09-12T16:01:39.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:38 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-12T16:01:40.173 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:40 smithi130 ceph-mon[98670]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:40.173 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:40.174 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:40.174 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:40 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/561452854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:40.174 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:40.174 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:40.174 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:40.175 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:40.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:40 smithi159 ceph-mon[102522]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:40.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:40.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:40.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:40 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/561452854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:40.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:40.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:40.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:40.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:40.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:40 smithi203 ceph-mon[103431]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:40.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:40.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:40.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:40 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/561452854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:40.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:40.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:40.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:40.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:41.011 INFO:journalctl@ceph.mds.d.smithi130.stdout:Sep 12 16:01:40 smithi130 systemd[1]: Starting Ceph mds.d for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:41.281 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:41.287 INFO:journalctl@ceph.mds.d.smithi130.stdout:Sep 12 16:01:41 smithi130 podman[153314]: 2023-09-12T16:01:41.288 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:41 smithi130 ceph-mon[98670]: Deploying daemon mds.d on smithi130 2023-09-12T16:01:41.288 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:41 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.130:6834/2074628565,v1:172.21.15.130:6835/2074628565] up:boot 2023-09-12T16:01:41.288 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:41 smithi130 ceph-mon[98670]: fsmap 3 up:standby 2023-09-12T16:01:41.289 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-12T16:01:41.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 -- 172.21.15.130:0/1728432252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 msgr2=0x7f44a0107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:41.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 --2- 172.21.15.130:0/1728432252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 0x7f44a0107970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f449800b0d0 tx=0x7f449800b3e0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 -- 172.21.15.130:0/1728432252 shutdown_connections 2023-09-12T16:01:41.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 --2- 172.21.15.130:0/1728432252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 0x7f44a0107970 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 --2- 172.21.15.130:0/1728432252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44a00ff530 0x7f44a00ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 --2- 172.21.15.130:0/1728432252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a0100720 0x7f44a0100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 -- 172.21.15.130:0/1728432252 >> 172.21.15.130:0/1728432252 conn(0x7f44a00faa80 msgr2=0x7f44a00fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:41.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 -- 172.21.15.130:0/1728432252 shutdown_connections 2023-09-12T16:01:41.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 -- 172.21.15.130:0/1728432252 wait complete. 2023-09-12T16:01:41.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.431+0000 7f44a5afa700 1 Processor -- start 2023-09-12T16:01:41.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f44a5afa700 1 -- start start 2023-09-12T16:01:41.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f44a5afa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44a00ff530 0x7f44a0069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:41.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f44a5afa700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a0100720 0x7f44a019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:41.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f44a5afa700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 0x7f44a019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:41.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f44a5afa700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a019de30 con 0x7f44a01010c0 2023-09-12T16:01:41.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f44a5afa700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a019dfa0 con 0x7f44a0100720 2023-09-12T16:01:41.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f44a5afa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a019e160 con 0x7f44a00ff530 2023-09-12T16:01:41.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 0x7f44a019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:41.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 0x7f44a019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53036/0 (socket says 172.21.15.130:53036) 2023-09-12T16:01:41.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449ffff700 1 -- 172.21.15.130:0/2429050828 learned_addr learned my addr 172.21.15.130:0/2429050828 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:41.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449effd700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a0100720 0x7f44a019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:41.445 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449f7fe700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44a00ff530 0x7f44a0069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:41.445 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449ffff700 1 -- 172.21.15.130:0/2429050828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44a00ff530 msgr2=0x7f44a0069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:41.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449ffff700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44a00ff530 0x7f44a0069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449ffff700 1 -- 172.21.15.130:0/2429050828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a0100720 msgr2=0x7f44a019b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:41.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449ffff700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a0100720 0x7f44a019b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449ffff700 1 -- 172.21.15.130:0/2429050828 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4498009d00 con 0x7f44a01010c0 2023-09-12T16:01:41.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449f7fe700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44a00ff530 0x7f44a0069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:01:41.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449effd700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a0100720 0x7f44a019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:01:41.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449ffff700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 0x7f44a019d660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4498007a00 tx=0x7f4498009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:41.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449cff9700 1 -- 172.21.15.130:0/2429050828 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449800e040 con 0x7f44a01010c0 2023-09-12T16:01:41.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449cff9700 1 -- 172.21.15.130:0/2429050828 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4498003e10 con 0x7f44a01010c0 2023-09-12T16:01:41.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a019e3e0 con 0x7f44a01010c0 2023-09-12T16:01:41.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.432+0000 7f449cff9700 1 -- 172.21.15.130:0/2429050828 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498005610 con 0x7f44a01010c0 2023-09-12T16:01:41.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.433+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a019e800 con 0x7f44a01010c0 2023-09-12T16:01:41.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.433+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a0106930 con 0x7f44a01010c0 2023-09-12T16:01:41.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.435+0000 7f449cff9700 1 -- 172.21.15.130:0/2429050828 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f4498005770 con 0x7f44a01010c0 2023-09-12T16:01:41.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.435+0000 7f449cff9700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f44880a0690 0x7f44880a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:41.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.435+0000 7f449f7fe700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f44880a0690 0x7f44880a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:41.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.435+0000 7f449cff9700 1 -- 172.21.15.130:0/2429050828 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f44980cd6b0 con 0x7f44a01010c0 2023-09-12T16:01:41.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.436+0000 7f449f7fe700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f44880a0690 0x7f44880a2b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4490005fd0 tx=0x7f4490005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:41.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.438+0000 7f449cff9700 1 -- 172.21.15.130:0/2429050828 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4498097710 con 0x7f44a01010c0 2023-09-12T16:01:41.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:41 smithi159 ceph-mon[102522]: Deploying daemon mds.d on smithi130 2023-09-12T16:01:41.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:41 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.130:6834/2074628565,v1:172.21.15.130:6835/2074628565] up:boot 2023-09-12T16:01:41.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:41 smithi159 ceph-mon[102522]: fsmap 3 up:standby 2023-09-12T16:01:41.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-12T16:01:41.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:41 smithi203 ceph-mon[103431]: Deploying daemon mds.d on smithi130 2023-09-12T16:01:41.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:41 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.130:6834/2074628565,v1:172.21.15.130:6835/2074628565] up:boot 2023-09-12T16:01:41.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:41 smithi203 ceph-mon[103431]: fsmap 3 up:standby 2023-09-12T16:01:41.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-12T16:01:41.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.676+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44a002d0c0 con 0x7f44a01010c0 2023-09-12T16:01:41.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.676+0000 7f449cff9700 1 -- 172.21.15.130:0/2429050828 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2916 (secure 0 0 0) 0x7f449809ad30 con 0x7f44a01010c0 2023-09-12T16:01:41.677 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:41.677 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-12T16:01:41.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f44880a0690 msgr2=0x7f44880a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:41.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f44880a0690 0x7f44880a2b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4490005fd0 tx=0x7f4490005e40 comp rx=0 tx=0).stop 2023-09-12T16:01:41.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 msgr2=0x7f44a019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:41.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 0x7f44a019d660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4498007a00 tx=0x7f4498009570 comp rx=0 tx=0).stop 2023-09-12T16:01:41.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 shutdown_connections 2023-09-12T16:01:41.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f44880a0690 0x7f44880a2b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a01010c0 0x7f44a019d660 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44a00ff530 0x7f44a0069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 --2- 172.21.15.130:0/2429050828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a0100720 0x7f44a019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:41.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.678+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 >> 172.21.15.130:0/2429050828 conn(0x7f44a00faa80 msgr2=0x7f44a0103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:41.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.679+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 shutdown_connections 2023-09-12T16:01:41.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:41.679+0000 7f44a5afa700 1 -- 172.21.15.130:0/2429050828 wait complete. 2023-09-12T16:01:41.681 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 4 2023-09-12T16:01:41.690 INFO:journalctl@ceph.mds.d.smithi130.stdout:Sep 12 16:01:41 smithi130 ceph-mds[153354]: -- 172.21.15.130:0/2615095345 <== mon.0 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563714f8d1e0 con 0x563714f63000 2023-09-12T16:01:41.690 INFO:journalctl@ceph.mds.d.smithi130.stdout:Sep 12 16:01:41 smithi130 bash[153314]: 594c30dd43bd9556f4dd211da3232299acc54f870418499ae0da88dc7c13cc66 2023-09-12T16:01:41.691 INFO:journalctl@ceph.mds.d.smithi130.stdout:Sep 12 16:01:41 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-d[153330]: starting mds.d at 2023-09-12T16:01:41.691 INFO:journalctl@ceph.mds.d.smithi130.stdout:Sep 12 16:01:41 smithi130 systemd[1]: Started Ceph mds.d for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:42.333 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:42 smithi159 ceph-mon[102522]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:42.333 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:42.333 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:42.334 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:42.334 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:42.334 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:42.334 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:42 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:42.334 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:42 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2429050828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:42 smithi130 ceph-mon[98670]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:42.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:42.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:42.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:42 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:42.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:42 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2429050828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:42.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:42 smithi203 ceph-mon[103431]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:42.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:42.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:42.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:42.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:42.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:42.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:42 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:42.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:42 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2429050828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:43.226 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:01:42 smithi159 systemd[1]: Starting Ceph mds.e for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:43.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:43 smithi159 ceph-mon[102522]: Deploying daemon mds.e on smithi159 2023-09-12T16:01:43.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:43 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.130:6836/372380878,v1:172.21.15.130:6837/372380878] up:boot 2023-09-12T16:01:43.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:43 smithi159 ceph-mon[102522]: fsmap 4 up:standby 2023-09-12T16:01:43.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-12T16:01:43.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:43 smithi130 ceph-mon[98670]: Deploying daemon mds.e on smithi159 2023-09-12T16:01:43.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:43 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.130:6836/372380878,v1:172.21.15.130:6837/372380878] up:boot 2023-09-12T16:01:43.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:43 smithi130 ceph-mon[98670]: fsmap 4 up:standby 2023-09-12T16:01:43.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-12T16:01:43.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:43 smithi203 ceph-mon[103431]: Deploying daemon mds.e on smithi159 2023-09-12T16:01:43.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:43 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.130:6836/372380878,v1:172.21.15.130:6837/372380878] up:boot 2023-09-12T16:01:43.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:43 smithi203 ceph-mon[103431]: fsmap 4 up:standby 2023-09-12T16:01:43.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-12T16:01:43.505 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:01:43 smithi159 podman[138482]: 2023-09-12T16:01:43.506 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:01:43 smithi159 ceph-mds[138523]: -- 172.21.15.159:0/2791283878 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5558ed1531e0 con 0x5558ed129000 2023-09-12T16:01:43.692 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:43.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.827+0000 7f9dee3f2700 1 -- 172.21.15.130:0/3155076324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de81062c0 msgr2=0x7f9de8108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:43.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.827+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/3155076324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de81062c0 0x7f9de8108730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9de000b0a0 tx=0x7f9de000b3b0 comp rx=0 tx=0).stop 2023-09-12T16:01:43.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.827+0000 7f9dee3f2700 1 -- 172.21.15.130:0/3155076324 shutdown_connections 2023-09-12T16:01:43.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.827+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/3155076324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de81062c0 0x7f9de8108730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:43.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.827+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/3155076324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de8101060 0x7f9de8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:43.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.827+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/3155076324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de8103990 0x7f9de8105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:43.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.827+0000 7f9dee3f2700 1 -- 172.21.15.130:0/3155076324 >> 172.21.15.130:0/3155076324 conn(0x7f9de80faa30 msgr2=0x7f9de80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:43.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.827+0000 7f9dee3f2700 1 -- 172.21.15.130:0/3155076324 shutdown_connections 2023-09-12T16:01:43.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.827+0000 7f9dee3f2700 1 -- 172.21.15.130:0/3155076324 wait complete. 2023-09-12T16:01:43.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9dee3f2700 1 Processor -- start 2023-09-12T16:01:43.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9dee3f2700 1 -- start start 2023-09-12T16:01:43.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9dee3f2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de8101060 0x7f9de8196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:43.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9dee3f2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8103990 0x7f9de81968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:43.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9dee3f2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de81062c0 0x7f9de819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:43.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9dee3f2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de819dee0 con 0x7f9de8103990 2023-09-12T16:01:43.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9dee3f2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de819e020 con 0x7f9de8101060 2023-09-12T16:01:43.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9dee3f2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de819e1b0 con 0x7f9de81062c0 2023-09-12T16:01:43.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9de77fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8103990 0x7f9de81968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:43.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9de77fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8103990 0x7f9de81968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53048/0 (socket says 172.21.15.130:53048) 2023-09-12T16:01:43.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.828+0000 7f9de77fe700 1 -- 172.21.15.130:0/121770463 learned_addr learned my addr 172.21.15.130:0/121770463 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:43.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de7fff700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de8101060 0x7f9de8196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:43.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9dec98f700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de81062c0 0x7f9de819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:43.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de77fe700 1 -- 172.21.15.130:0/121770463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de81062c0 msgr2=0x7f9de819d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:43.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de77fe700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de81062c0 0x7f9de819d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:43.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de77fe700 1 -- 172.21.15.130:0/121770463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de8101060 msgr2=0x7f9de8196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:43.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de77fe700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de8101060 0x7f9de8196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:43.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de77fe700 1 -- 172.21.15.130:0/121770463 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de0009d00 con 0x7f9de8103990 2023-09-12T16:01:43.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de77fe700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8103990 0x7f9de81968a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00d980 tx=0x7f9ddc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:43.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de57fa700 1 -- 172.21.15.130:0/121770463 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc004220 con 0x7f9de8103990 2023-09-12T16:01:43.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de57fa700 1 -- 172.21.15.130:0/121770463 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ddc007640 con 0x7f9de8103990 2023-09-12T16:01:43.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de819e490 con 0x7f9de8103990 2023-09-12T16:01:43.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9de57fa700 1 -- 172.21.15.130:0/121770463 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc005690 con 0x7f9de8103990 2023-09-12T16:01:43.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.829+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de819e9a0 con 0x7f9de8103990 2023-09-12T16:01:43.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.830+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de80fc5b0 con 0x7f9de8103990 2023-09-12T16:01:43.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.831+0000 7f9de57fa700 1 -- 172.21.15.130:0/121770463 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f9ddc004750 con 0x7f9de8103990 2023-09-12T16:01:43.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.832+0000 7f9de57fa700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9dd00a0690 0x7f9dd00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:43.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.832+0000 7f9de7fff700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9dd00a0690 0x7f9dd00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:43.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.832+0000 7f9de57fa700 1 -- 172.21.15.130:0/121770463 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9ddc0cba60 con 0x7f9de8103990 2023-09-12T16:01:43.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.832+0000 7f9de7fff700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9dd00a0690 0x7f9dd00a2b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005950 tx=0x7f9dd800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:43.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:43.836+0000 7f9de57fa700 1 -- 172.21.15.130:0/121770463 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9ddc0959d0 con 0x7f9de8103990 2023-09-12T16:01:43.975 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:01:43 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-e[138498]: starting mds.e at 2023-09-12T16:01:43.976 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:01:43 smithi159 bash[138482]: 04fc5a2baef358bdfe39a506396698c3080b5d075019d03445b9f1e9f6c615ef 2023-09-12T16:01:43.976 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:01:43 smithi159 systemd[1]: Started Ceph mds.e for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:44.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.071+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9de802d020 con 0x7f9de8103990 2023-09-12T16:01:44.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.072+0000 7f9de57fa700 1 -- 172.21.15.130:0/121770463 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4501 (secure 0 0 0) 0x7f9ddc098ff0 con 0x7f9de8103990 2023-09-12T16:01:44.072 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:44.073 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-12T16:01:44.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9dd00a0690 msgr2=0x7f9dd00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:44.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9dd00a0690 0x7f9dd00a2b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005950 tx=0x7f9dd800b410 comp rx=0 tx=0).stop 2023-09-12T16:01:44.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8103990 msgr2=0x7f9de81968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:44.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8103990 0x7f9de81968a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00d980 tx=0x7f9ddc00dc90 comp rx=0 tx=0).stop 2023-09-12T16:01:44.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 shutdown_connections 2023-09-12T16:01:44.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9dd00a0690 0x7f9dd00a2b50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:44.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de81062c0 0x7f9de819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:44.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de8101060 0x7f9de8196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:44.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 --2- 172.21.15.130:0/121770463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9de8103990 0x7f9de81968a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:44.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 >> 172.21.15.130:0/121770463 conn(0x7f9de80faa30 msgr2=0x7f9de8101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:44.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 shutdown_connections 2023-09-12T16:01:44.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:44.074+0000 7f9dee3f2700 1 -- 172.21.15.130:0/121770463 wait complete. 2023-09-12T16:01:44.077 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 6 2023-09-12T16:01:44.330 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:44 smithi203 ceph-mon[103431]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:44.330 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:44 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:44.330 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:44 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:44.331 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:44 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:44.331 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:44 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:44.331 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:44 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:44.331 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:44 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:44.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:44 smithi130 ceph-mon[98670]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:44.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:44 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:44.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:44 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:44.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:44 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:44.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:44 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:44.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:44 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:44.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:44 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:44.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:44 smithi159 ceph-mon[102522]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:44.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:44 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:44.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:44 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:44.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:44 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:44.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:44 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:44.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:44 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:44.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:44 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:45.302 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:45 smithi203 ceph-mon[103431]: Deploying daemon mds.f on smithi203 2023-09-12T16:01:45.303 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:45 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.159:6834/3667735298,v1:172.21.15.159:6835/3667735298] up:boot 2023-09-12T16:01:45.303 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:45 smithi203 ceph-mon[103431]: fsmap 5 up:standby 2023-09-12T16:01:45.303 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-12T16:01:45.303 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:45 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/121770463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:45.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:45 smithi130 ceph-mon[98670]: Deploying daemon mds.f on smithi203 2023-09-12T16:01:45.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:45 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.159:6834/3667735298,v1:172.21.15.159:6835/3667735298] up:boot 2023-09-12T16:01:45.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:45 smithi130 ceph-mon[98670]: fsmap 5 up:standby 2023-09-12T16:01:45.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-12T16:01:45.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:45 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/121770463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:45.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:45 smithi159 ceph-mon[102522]: Deploying daemon mds.f on smithi203 2023-09-12T16:01:45.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:45 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.159:6834/3667735298,v1:172.21.15.159:6835/3667735298] up:boot 2023-09-12T16:01:45.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:45 smithi159 ceph-mon[102522]: fsmap 5 up:standby 2023-09-12T16:01:45.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-12T16:01:45.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:45 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/121770463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:45.623 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:01:45 smithi203 systemd[1]: Starting Ceph mds.f for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:45.914 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:01:45 smithi203 podman[140219]: 2023-09-12T16:01:46.087 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:46.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.217+0000 7f48e1520700 1 -- 172.21.15.130:0/1280006966 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc106350 msgr2=0x7f48dc108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:46.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.217+0000 7f48e1520700 1 --2- 172.21.15.130:0/1280006966 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc106350 0x7f48dc108760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48d400b0a0 tx=0x7f48d400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.217+0000 7f48e1520700 1 -- 172.21.15.130:0/1280006966 shutdown_connections 2023-09-12T16:01:46.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.217+0000 7f48e1520700 1 --2- 172.21.15.130:0/1280006966 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc106350 0x7f48dc108760 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.217+0000 7f48e1520700 1 --2- 172.21.15.130:0/1280006966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48dc1010f0 0x7f48dc1034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.217+0000 7f48e1520700 1 --2- 172.21.15.130:0/1280006966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc103a20 0x7f48dc105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.217+0000 7f48e1520700 1 -- 172.21.15.130:0/1280006966 >> 172.21.15.130:0/1280006966 conn(0x7f48dc0faa60 msgr2=0x7f48dc0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:46.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.218+0000 7f48e1520700 1 -- 172.21.15.130:0/1280006966 shutdown_connections 2023-09-12T16:01:46.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.218+0000 7f48e1520700 1 -- 172.21.15.130:0/1280006966 wait complete. 2023-09-12T16:01:46.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.218+0000 7f48e1520700 1 Processor -- start 2023-09-12T16:01:46.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.218+0000 7f48e1520700 1 -- start start 2023-09-12T16:01:46.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.218+0000 7f48e1520700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48dc1010f0 0x7f48dc1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:46.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.218+0000 7f48e1520700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc103a20 0x7f48dc196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:46.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.218+0000 7f48e1520700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc106350 0x7f48dc19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:46.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48e1520700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc19de50 con 0x7f48dc103a20 2023-09-12T16:01:46.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48e1520700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc19df90 con 0x7f48dc106350 2023-09-12T16:01:46.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48e1520700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc19e120 con 0x7f48dc1010f0 2023-09-12T16:01:46.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48da7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc103a20 0x7f48dc196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:46.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48da7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc103a20 0x7f48dc196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53062/0 (socket says 172.21.15.130:53062) 2023-09-12T16:01:46.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48da7fc700 1 -- 172.21.15.130:0/2756752268 learned_addr learned my addr 172.21.15.130:0/2756752268 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:46.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48daffd700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48dc1010f0 0x7f48dc1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:46.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48db7fe700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc106350 0x7f48dc19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:46.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48da7fc700 1 -- 172.21.15.130:0/2756752268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48dc1010f0 msgr2=0x7f48dc1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:46.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48da7fc700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48dc1010f0 0x7f48dc1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48da7fc700 1 -- 172.21.15.130:0/2756752268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc106350 msgr2=0x7f48dc19d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:46.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48da7fc700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc106350 0x7f48dc19d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48da7fc700 1 -- 172.21.15.130:0/2756752268 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4009d00 con 0x7f48dc103a20 2023-09-12T16:01:46.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48daffd700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48dc1010f0 0x7f48dc1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:01:46.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48da7fc700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc103a20 0x7f48dc196810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c9b0 tx=0x7f48d000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:46.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48c3fff700 1 -- 172.21.15.130:0/2756752268 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0007b00 con 0x7f48dc103a20 2023-09-12T16:01:46.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48c3fff700 1 -- 172.21.15.130:0/2756752268 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d0007c60 con 0x7f48dc103a20 2023-09-12T16:01:46.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc19e400 con 0x7f48dc103a20 2023-09-12T16:01:46.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48c3fff700 1 -- 172.21.15.130:0/2756752268 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0018660 con 0x7f48dc103a20 2023-09-12T16:01:46.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.219+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc19e910 con 0x7f48dc103a20 2023-09-12T16:01:46.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.220+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48dc0fc5e0 con 0x7f48dc103a20 2023-09-12T16:01:46.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.221+0000 7f48c3fff700 1 -- 172.21.15.130:0/2756752268 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f48d000f450 con 0x7f48dc103a20 2023-09-12T16:01:46.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.222+0000 7f48c3fff700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48c40a0690 0x7f48c40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:46.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.222+0000 7f48daffd700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48c40a0690 0x7f48c40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:46.229 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.222+0000 7f48c3fff700 1 -- 172.21.15.130:0/2756752268 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48d00cb760 con 0x7f48dc103a20 2023-09-12T16:01:46.229 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.222+0000 7f48daffd700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48c40a0690 0x7f48c40a2b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f48cc011490 tx=0x7f48cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:46.229 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.226+0000 7f48c3fff700 1 -- 172.21.15.130:0/2756752268 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f48d0095690 con 0x7f48dc103a20 2023-09-12T16:01:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:46 smithi203 ceph-mon[103431]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:46.246 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:01:45 smithi203 ceph-mds[140259]: -- 172.21.15.203:0/878973804 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f9cc2631e0 con 0x55f9cc238800 2023-09-12T16:01:46.246 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:01:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-f[140235]: starting mds.f at 2023-09-12T16:01:46.246 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:01:45 smithi203 bash[140219]: 35eb579dcf1e0ce319cb0990ebc2e9a0d68953b5171823b93011a10551e1ef11 2023-09-12T16:01:46.246 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:01:45 smithi203 systemd[1]: Started Ceph mds.f for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:46.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.460+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48dc02cea0 con 0x7f48dc103a20 2023-09-12T16:01:46.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.461+0000 7f48c3fff700 1 -- 172.21.15.130:0/2756752268 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5295 (secure 0 0 0) 0x7f48d0098cb0 con 0x7f48dc103a20 2023-09-12T16:01:46.462 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:46.463 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-09-12T16:01:46.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48c40a0690 msgr2=0x7f48c40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:46.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48c40a0690 0x7f48c40a2b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f48cc011490 tx=0x7f48cc009260 comp rx=0 tx=0).stop 2023-09-12T16:01:46.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc103a20 msgr2=0x7f48dc196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:46.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc103a20 0x7f48dc196810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c9b0 tx=0x7f48d000ccc0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 shutdown_connections 2023-09-12T16:01:46.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48c40a0690 0x7f48c40a2b50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc106350 0x7f48dc19d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48dc1010f0 0x7f48dc1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 --2- 172.21.15.130:0/2756752268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48dc103a20 0x7f48dc196810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:46.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 >> 172.21.15.130:0/2756752268 conn(0x7f48dc0faa60 msgr2=0x7f48dc101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:46.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 shutdown_connections 2023-09-12T16:01:46.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:46.463+0000 7f48e1520700 1 -- 172.21.15.130:0/2756752268 wait complete. 2023-09-12T16:01:46.468 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 7 2023-09-12T16:01:46.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:46 smithi130 ceph-mon[98670]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:46.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:46 smithi159 ceph-mon[102522]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:47.288 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:47.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:47.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.203:6834/1801270748,v1:172.21.15.203:6835/1801270748] up:boot 2023-09-12T16:01:47.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: fsmap 6 up:standby 2023-09-12T16:01:47.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-12T16:01:47.290 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:47.290 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:47.290 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:47.290 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:47.290 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: Deploying daemon mds.c on smithi203 2023-09-12T16:01:47.291 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2756752268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:47.291 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:47 smithi203 ceph-mon[103431]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:47.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:47.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:47.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.203:6834/1801270748,v1:172.21.15.203:6835/1801270748] up:boot 2023-09-12T16:01:47.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: fsmap 6 up:standby 2023-09-12T16:01:47.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-12T16:01:47.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:47.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:47.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:47.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:47.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: Deploying daemon mds.c on smithi203 2023-09-12T16:01:47.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2756752268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:47.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:47 smithi130 ceph-mon[98670]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:47.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:47.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:47.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.203:6834/1801270748,v1:172.21.15.203:6835/1801270748] up:boot 2023-09-12T16:01:47.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: fsmap 6 up:standby 2023-09-12T16:01:47.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-12T16:01:47.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:47.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:47.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:47.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:47.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: Deploying daemon mds.c on smithi203 2023-09-12T16:01:47.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2756752268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:47.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:47 smithi159 ceph-mon[102522]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:47.960 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:01:47 smithi203 systemd[1]: Starting Ceph mds.c for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:48.245 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:01:47 smithi203 podman[140562]: 2023-09-12T16:01:48.476 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:48.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.605+0000 7fd226025700 1 -- 172.21.15.130:0/382546521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 msgr2=0x7fd2200ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:48.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.605+0000 7fd226025700 1 --2- 172.21.15.130:0/382546521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 0x7fd2200ffc10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd210009990 tx=0x7fd210009ca0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 -- 172.21.15.130:0/382546521 shutdown_connections 2023-09-12T16:01:48.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 --2- 172.21.15.130:0/382546521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd220100150 0x7fd220107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 --2- 172.21.15.130:0/382546521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd220100a80 0x7fd220100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 --2- 172.21.15.130:0/382546521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 0x7fd2200ffc10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 -- 172.21.15.130:0/382546521 >> 172.21.15.130:0/382546521 conn(0x7fd2200fa1c0 msgr2=0x7fd2200fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:48.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 -- 172.21.15.130:0/382546521 shutdown_connections 2023-09-12T16:01:48.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 -- 172.21.15.130:0/382546521 wait complete. 2023-09-12T16:01:48.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 Processor -- start 2023-09-12T16:01:48.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 -- start start 2023-09-12T16:01:48.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 0x7fd220195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:48.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd220100150 0x7fd2201956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:48.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd220100a80 0x7fd22019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:48.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22019cd00 con 0x7fd2200ff7b0 2023-09-12T16:01:48.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.606+0000 7fd226025700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22019ce40 con 0x7fd220100a80 2023-09-12T16:01:48.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd226025700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22019cfd0 con 0x7fd220100150 2023-09-12T16:01:48.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 0x7fd220195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:48.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 0x7fd220195180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53074/0 (socket says 172.21.15.130:53074) 2023-09-12T16:01:48.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21f7fe700 1 -- 172.21.15.130:0/2025368070 learned_addr learned my addr 172.21.15.130:0/2025368070 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:48.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21effd700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd220100150 0x7fd2201956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:48.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21ffff700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd220100a80 0x7fd22019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:48.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21f7fe700 1 -- 172.21.15.130:0/2025368070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd220100150 msgr2=0x7fd2201956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:48.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21f7fe700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd220100150 0x7fd2201956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21f7fe700 1 -- 172.21.15.130:0/2025368070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd220100a80 msgr2=0x7fd22019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:48.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21f7fe700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd220100a80 0x7fd22019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21f7fe700 1 -- 172.21.15.130:0/2025368070 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2100095f0 con 0x7fd2200ff7b0 2023-09-12T16:01:48.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21f7fe700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 0x7fd220195180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd214007b50 tx=0x7fd214007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:48.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21cff9700 1 -- 172.21.15.130:0/2025368070 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214011070 con 0x7fd2200ff7b0 2023-09-12T16:01:48.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21cff9700 1 -- 172.21.15.130:0/2025368070 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd21400c2e0 con 0x7fd2200ff7b0 2023-09-12T16:01:48.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd21cff9700 1 -- 172.21.15.130:0/2025368070 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21400e580 con 0x7fd2200ff7b0 2023-09-12T16:01:48.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2200fe700 con 0x7fd2200ff7b0 2023-09-12T16:01:48.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.607+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2200fec10 con 0x7fd2200ff7b0 2023-09-12T16:01:48.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.608+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22010acf0 con 0x7fd2200ff7b0 2023-09-12T16:01:48.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.609+0000 7fd21cff9700 1 -- 172.21.15.130:0/2025368070 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fd214003f90 con 0x7fd2200ff7b0 2023-09-12T16:01:48.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.610+0000 7fd21cff9700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd2080a0710 0x7fd2080a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:48.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.610+0000 7fd21effd700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd2080a0710 0x7fd2080a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:48.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.610+0000 7fd21cff9700 1 -- 172.21.15.130:0/2025368070 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2140ccce0 con 0x7fd2200ff7b0 2023-09-12T16:01:48.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.610+0000 7fd21effd700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd2080a0710 0x7fd2080a2bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd21000b5c0 tx=0x7fd210005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:48.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.613+0000 7fd21cff9700 1 -- 172.21.15.130:0/2025368070 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd214096ca0 con 0x7fd2200ff7b0 2023-09-12T16:01:48.745 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:01:48 smithi203 ceph-mds[140602]: -- 172.21.15.203:0/924666903 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bd7b3b31e0 con 0x55bd7b388800 2023-09-12T16:01:48.745 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:01:48 smithi203 bash[140562]: f6a60204aa53a815a4887d68ad10d87029ef95c52cf3023b7d1e5ae2fb3e6506 2023-09-12T16:01:48.746 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:01:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-c[140578]: starting mds.c at 2023-09-12T16:01:48.746 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:01:48 smithi203 systemd[1]: Started Ceph mds.c for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:48.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.847+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2200ff0b0 con 0x7fd2200ff7b0 2023-09-12T16:01:48.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.848+0000 7fd21cff9700 1 -- 172.21.15.130:0/2025368070 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6089 (secure 0 0 0) 0x7fd21409a2c0 con 0x7fd2200ff7b0 2023-09-12T16:01:48.849 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:48.849 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-12T16:01:48.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd2080a0710 msgr2=0x7fd2080a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:48.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd2080a0710 0x7fd2080a2bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd21000b5c0 tx=0x7fd210005fd0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 msgr2=0x7fd220195180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:48.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 0x7fd220195180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd214007b50 tx=0x7fd214007e60 comp rx=0 tx=0).stop 2023-09-12T16:01:48.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 shutdown_connections 2023-09-12T16:01:48.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd2080a0710 0x7fd2080a2bd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd220100a80 0x7fd22019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd2200ff7b0 0x7fd220195180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 --2- 172.21.15.130:0/2025368070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd220100150 0x7fd2201956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:48.853 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 >> 172.21.15.130:0/2025368070 conn(0x7fd2200fa1c0 msgr2=0x7fd2200fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:48.853 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 shutdown_connections 2023-09-12T16:01:48.853 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:48.850+0000 7fd226025700 1 -- 172.21.15.130:0/2025368070 wait complete. 2023-09-12T16:01:48.853 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 8 2023-09-12T16:01:49.657 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:49.657 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:49.657 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:49.657 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:49.658 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:49.658 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:49.658 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: Deploying daemon mds.h on smithi159 2023-09-12T16:01:49.658 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] up:boot 2023-09-12T16:01:49.659 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: fsmap 7 up:standby 2023-09-12T16:01:49.659 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-12T16:01:49.659 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:49.659 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:49 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2025368070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:49.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:49.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:49.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:49.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:49.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:49.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:49.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: Deploying daemon mds.h on smithi159 2023-09-12T16:01:49.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] up:boot 2023-09-12T16:01:49.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: fsmap 7 up:standby 2023-09-12T16:01:49.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-12T16:01:49.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:49.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:49 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2025368070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:49.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:49.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:49.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:49.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:49.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:49.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:49.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: Deploying daemon mds.h on smithi159 2023-09-12T16:01:49.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] up:boot 2023-09-12T16:01:49.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: fsmap 7 up:standby 2023-09-12T16:01:49.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-12T16:01:49.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:49.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:49 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2025368070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:50.612 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:01:50 smithi159 systemd[1]: Starting Ceph mds.h for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:50.863 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:01:50 smithi159 podman[138823]: 2023-09-12T16:01:50.863 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:51.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.002+0000 7f4dbbfff700 1 -- 172.21.15.130:0/851702696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc0ff370 msgr2=0x7f4dbc0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:51.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.002+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/851702696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc0ff370 0x7f4dbc0ff790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0099c0 tx=0x7f4dac009cd0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.002+0000 7f4dbbfff700 1 -- 172.21.15.130:0/851702696 shutdown_connections 2023-09-12T16:01:51.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.002+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/851702696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dbc100f00 0x7f4dbc1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.002+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/851702696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc0ff370 0x7f4dbc0ff790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.002+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/851702696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dbc100560 0x7f4dbc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.002+0000 7f4dbbfff700 1 -- 172.21.15.130:0/851702696 >> 172.21.15.130:0/851702696 conn(0x7f4dbc0fa8c0 msgr2=0x7f4dbc0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:51.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.002+0000 7f4dbbfff700 1 -- 172.21.15.130:0/851702696 shutdown_connections 2023-09-12T16:01:51.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.002+0000 7f4dbbfff700 1 -- 172.21.15.130:0/851702696 wait complete. 2023-09-12T16:01:51.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.003+0000 7f4dbbfff700 1 Processor -- start 2023-09-12T16:01:51.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.003+0000 7f4dbbfff700 1 -- start start 2023-09-12T16:01:51.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.003+0000 7f4dbbfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dbc0ff370 0x7f4dbc194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:51.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.003+0000 7f4dbbfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dbc100560 0x7f4dbc195420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:51.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.003+0000 7f4dbbfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc100f00 0x7f4dbc19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:51.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.003+0000 7f4dbbfff700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc19ca60 con 0x7f4dbc100f00 2023-09-12T16:01:51.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.003+0000 7f4dbbfff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc19cba0 con 0x7f4dbc0ff370 2023-09-12T16:01:51.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.003+0000 7f4dbbfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc19cd30 con 0x7f4dbc100560 2023-09-12T16:01:51.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbb7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc100f00 0x7f4dbc19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:51.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbb7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc100f00 0x7f4dbc19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53092/0 (socket says 172.21.15.130:53092) 2023-09-12T16:01:51.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbb7fe700 1 -- 172.21.15.130:0/3680357256 learned_addr learned my addr 172.21.15.130:0/3680357256 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:51.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbaffd700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dbc0ff370 0x7f4dbc194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:51.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dba7fc700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dbc100560 0x7f4dbc195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:51.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbb7fe700 1 -- 172.21.15.130:0/3680357256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dbc100560 msgr2=0x7f4dbc195420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:51.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbb7fe700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dbc100560 0x7f4dbc195420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbb7fe700 1 -- 172.21.15.130:0/3680357256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dbc0ff370 msgr2=0x7f4dbc194ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:51.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbb7fe700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dbc0ff370 0x7f4dbc194ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbb7fe700 1 -- 172.21.15.130:0/3680357256 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac0095f0 con 0x7f4dbc100f00 2023-09-12T16:01:51.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbb7fe700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc100f00 0x7f4dbc19c490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4db4007ee0 tx=0x7f4db400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:51.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4da3fff700 1 -- 172.21.15.130:0/3680357256 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db400dd20 con 0x7f4dbc100f00 2023-09-12T16:01:51.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dbc19cfb0 con 0x7f4dbc100f00 2023-09-12T16:01:51.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4da3fff700 1 -- 172.21.15.130:0/3680357256 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4db400f040 con 0x7f4dbc100f00 2023-09-12T16:01:51.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.004+0000 7f4da3fff700 1 -- 172.21.15.130:0/3680357256 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db40048a0 con 0x7f4dbc100f00 2023-09-12T16:01:51.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.005+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dbc067a50 con 0x7f4dbc100f00 2023-09-12T16:01:51.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.006+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dbc068440 con 0x7f4dbc100f00 2023-09-12T16:01:51.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.007+0000 7f4da3fff700 1 -- 172.21.15.130:0/3680357256 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f4db4003a00 con 0x7f4dbc100f00 2023-09-12T16:01:51.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.008+0000 7f4da3fff700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4da40a0730 0x7f4da40a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:51.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.008+0000 7f4dbaffd700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4da40a0730 0x7f4da40a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:51.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.008+0000 7f4da3fff700 1 -- 172.21.15.130:0/3680357256 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4db40caf30 con 0x7f4dbc100f00 2023-09-12T16:01:51.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.008+0000 7f4dbaffd700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4da40a0730 0x7f4da40a2bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009fd0 tx=0x7f4dac005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:51.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.012+0000 7f4da3fff700 1 -- 172.21.15.130:0/3680357256 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4db404ee10 con 0x7f4dbc100f00 2023-09-12T16:01:51.225 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:01:50 smithi159 bash[138823]: c9ccabea5aef515addb853fd589f736284fa398a96ed5d4299a3f5dea3c06657 2023-09-12T16:01:51.226 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:01:50 smithi159 ceph-mds[138863]: -- 172.21.15.159:0/2986725656 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c0d54051e0 con 0x55c0d53da800 2023-09-12T16:01:51.226 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:01:50 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-h[138839]: starting mds.h at 2023-09-12T16:01:51.226 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:01:50 smithi159 systemd[1]: Started Ceph mds.h for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:51.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.249+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dbc10ab80 con 0x7f4dbc100f00 2023-09-12T16:01:51.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.250+0000 7f4da3fff700 1 -- 172.21.15.130:0/3680357256 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6089 (secure 0 0 0) 0x7f4db4094f00 con 0x7f4dbc100f00 2023-09-12T16:01:51.251 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:51.251 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-12T16:01:51.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4da40a0730 msgr2=0x7f4da40a2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:51.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4da40a0730 0x7f4da40a2bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009fd0 tx=0x7f4dac005fb0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc100f00 msgr2=0x7f4dbc19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:51.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc100f00 0x7f4dbc19c490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4db4007ee0 tx=0x7f4db400d3b0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 shutdown_connections 2023-09-12T16:01:51.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4da40a0730 0x7f4da40a2bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dbc100f00 0x7f4dbc19c490 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dbc0ff370 0x7f4dbc194ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 --2- 172.21.15.130:0/3680357256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dbc100560 0x7f4dbc195420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:51.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 >> 172.21.15.130:0/3680357256 conn(0x7f4dbc0fa8c0 msgr2=0x7f4dbc0fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:51.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 shutdown_connections 2023-09-12T16:01:51.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:51.252+0000 7f4dbbfff700 1 -- 172.21.15.130:0/3680357256 wait complete. 2023-09-12T16:01:51.255 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 8 2023-09-12T16:01:52.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:52.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:52.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:52.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:52.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:52.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:52.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:52.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: Deploying daemon mds.i on smithi203 2023-09-12T16:01:52.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3680357256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:52.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.159:6836/3413713845,v1:172.21.15.159:6837/3413713845] up:boot 2023-09-12T16:01:52.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: fsmap 8 up:standby 2023-09-12T16:01:52.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-12T16:01:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: Deploying daemon mds.i on smithi203 2023-09-12T16:01:52.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3680357256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:52.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.159:6836/3413713845,v1:172.21.15.159:6837/3413713845] up:boot 2023-09-12T16:01:52.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: fsmap 8 up:standby 2023-09-12T16:01:52.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-12T16:01:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:52.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:52.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:52.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:52.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: Deploying daemon mds.i on smithi203 2023-09-12T16:01:52.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3680357256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:52.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.159:6836/3413713845,v1:172.21.15.159:6837/3413713845] up:boot 2023-09-12T16:01:52.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: fsmap 8 up:standby 2023-09-12T16:01:52.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-12T16:01:52.792 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:01:52 smithi203 systemd[1]: Starting Ceph mds.i for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:53.070 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:01:52 smithi203 podman[140902]: 2023-09-12T16:01:53.266 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:53.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 -- 172.21.15.130:0/3760038451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d70103990 msgr2=0x7f3d70105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:53.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 --2- 172.21.15.130:0/3760038451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d70103990 0x7f3d70105d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009990 tx=0x7f3d5c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 -- 172.21.15.130:0/3760038451 shutdown_connections 2023-09-12T16:01:53.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 --2- 172.21.15.130:0/3760038451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d701062c0 0x7f3d70108730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.402 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 --2- 172.21.15.130:0/3760038451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d70101060 0x7f3d70103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 --2- 172.21.15.130:0/3760038451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d70103990 0x7f3d70105d80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 -- 172.21.15.130:0/3760038451 >> 172.21.15.130:0/3760038451 conn(0x7f3d700faa30 msgr2=0x7f3d700fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:53.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 -- 172.21.15.130:0/3760038451 shutdown_connections 2023-09-12T16:01:53.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 -- 172.21.15.130:0/3760038451 wait complete. 2023-09-12T16:01:53.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 Processor -- start 2023-09-12T16:01:53.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.396+0000 7f3d76b79700 1 -- start start 2023-09-12T16:01:53.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d76b79700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d70101060 0x7f3d70196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:53.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d76b79700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d70103990 0x7f3d70196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:53.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d76b79700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d701062c0 0x7f3d7019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:53.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d76b79700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7019dd80 con 0x7f3d701062c0 2023-09-12T16:01:53.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d76b79700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7019dec0 con 0x7f3d70103990 2023-09-12T16:01:53.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d76b79700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7019e050 con 0x7f3d70101060 2023-09-12T16:01:53.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d74915700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d70101060 0x7f3d70196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:53.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d75116700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d701062c0 0x7f3d7019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:53.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d74915700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d70101060 0x7f3d70196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:60946/0 (socket says 172.21.15.130:60946) 2023-09-12T16:01:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d75116700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d701062c0 0x7f3d7019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53106/0 (socket says 172.21.15.130:53106) 2023-09-12T16:01:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d74915700 1 -- 172.21.15.130:0/2552769827 learned_addr learned my addr 172.21.15.130:0/2552769827 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d6ffff700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d70103990 0x7f3d70196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d74915700 1 -- 172.21.15.130:0/2552769827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d70103990 msgr2=0x7f3d70196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d74915700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d70103990 0x7f3d70196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d74915700 1 -- 172.21.15.130:0/2552769827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d701062c0 msgr2=0x7f3d7019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d74915700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d701062c0 0x7f3d7019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d74915700 1 -- 172.21.15.130:0/2552769827 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0095f0 con 0x7f3d70101060 2023-09-12T16:01:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d74915700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d70101060 0x7f3d70196200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400e9a0 tx=0x7f3d6400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.397+0000 7f3d6dffb700 1 -- 172.21.15.130:0/2552769827 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64009790 con 0x7f3d70101060 2023-09-12T16:01:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.398+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7019e330 con 0x7f3d70101060 2023-09-12T16:01:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.398+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7019e890 con 0x7f3d70101060 2023-09-12T16:01:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.399+0000 7f3d6dffb700 1 -- 172.21.15.130:0/2552769827 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d6400cbd0 con 0x7f3d70101060 2023-09-12T16:01:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.399+0000 7f3d6dffb700 1 -- 172.21.15.130:0/2552769827 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64018680 con 0x7f3d70101060 2023-09-12T16:01:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.400+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d54005320 con 0x7f3d70101060 2023-09-12T16:01:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.401+0000 7f3d6dffb700 1 -- 172.21.15.130:0/2552769827 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f3d64010430 con 0x7f3d70101060 2023-09-12T16:01:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.401+0000 7f3d6dffb700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d600a0690 0x7f3d600a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.401+0000 7f3d6dffb700 1 -- 172.21.15.130:0/2552769827 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3d64014070 con 0x7f3d70101060 2023-09-12T16:01:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.402+0000 7f3d6ffff700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d600a0690 0x7f3d600a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:53.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.402+0000 7f3d6ffff700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d600a0690 0x7f3d600a2b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006010 tx=0x7f3d5c00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:53.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.406+0000 7f3d6dffb700 1 -- 172.21.15.130:0/2552769827 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3d64051e10 con 0x7f3d70101060 2023-09-12T16:01:53.496 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-mds[140942]: -- 172.21.15.203:0/1598731383 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b5a061f1e0 con 0x55b5a05f4800 2023-09-12T16:01:53.496 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:01:53 smithi203 bash[140902]: 22af2c5b0e146d5fe7ef6987537551937295f6871dba8b77bc255fa17b7ca913 2023-09-12T16:01:53.497 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-i[140918]: starting mds.i at 2023-09-12T16:01:53.497 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:01:53 smithi203 systemd[1]: Started Ceph mds.i for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:53.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.655+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d540055d0 con 0x7f3d70101060 2023-09-12T16:01:53.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.656+0000 7f3d6dffb700 1 -- 172.21.15.130:0/2552769827 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6883 (secure 0 0 0) 0x7f3d64095880 con 0x7f3d70101060 2023-09-12T16:01:53.658 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:53.658 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24557,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-12T16:01:53.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d600a0690 msgr2=0x7f3d600a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:53.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d600a0690 0x7f3d600a2b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006010 tx=0x7f3d5c00b260 comp rx=0 tx=0).stop 2023-09-12T16:01:53.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d70101060 msgr2=0x7f3d70196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:53.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d70101060 0x7f3d70196200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400e9a0 tx=0x7f3d6400ecb0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 shutdown_connections 2023-09-12T16:01:53.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d600a0690 0x7f3d600a2b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d701062c0 0x7f3d7019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d70101060 0x7f3d70196200 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 --2- 172.21.15.130:0/2552769827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d70103990 0x7f3d70196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:53.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 >> 172.21.15.130:0/2552769827 conn(0x7f3d700faa30 msgr2=0x7f3d70101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:53.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 shutdown_connections 2023-09-12T16:01:53.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:53.659+0000 7f3d76b79700 1 -- 172.21.15.130:0/2552769827 wait complete. 2023-09-12T16:01:53.663 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 9 2023-09-12T16:01:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:53 smithi130 ceph-mon[98670]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:54.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:54.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:54.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:54.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:54.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:53 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2552769827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:54.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:53 smithi159 ceph-mon[102522]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:54.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:54.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:54.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:54.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:54.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:54.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:54.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:53 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2552769827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:54.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-mon[103431]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:54.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:54.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:54.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:53 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2552769827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:54.911 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:01:54 smithi130 systemd[1]: Starting Ceph mds.g for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:54 smithi130 ceph-mon[98670]: Deploying daemon mds.g on smithi130 2023-09-12T16:01:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:54 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] up:boot 2023-09-12T16:01:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:54 smithi130 ceph-mon[98670]: fsmap 9 up:standby 2023-09-12T16:01:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-12T16:01:55.226 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:01:54 smithi130 podman[153923]: 2023-09-12T16:01:55.226 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:01:55 smithi130 ceph-mds[153963]: -- 172.21.15.130:0/2232342833 <== mon.0 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557b49fc51e0 con 0x557b49f9ac00 2023-09-12T16:01:55.226 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:01:55 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-g[153939]: starting mds.g at 2023-09-12T16:01:55.227 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:01:55 smithi130 bash[153923]: 6c022b3a6f99829c5cc788aa54170c11abc669121d2b553f11ac578412a342b7 2023-09-12T16:01:55.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:54 smithi159 ceph-mon[102522]: Deploying daemon mds.g on smithi130 2023-09-12T16:01:55.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:54 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] up:boot 2023-09-12T16:01:55.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:54 smithi159 ceph-mon[102522]: fsmap 9 up:standby 2023-09-12T16:01:55.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-12T16:01:55.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:54 smithi203 ceph-mon[103431]: Deploying daemon mds.g on smithi130 2023-09-12T16:01:55.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:54 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] up:boot 2023-09-12T16:01:55.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:54 smithi203 ceph-mon[103431]: fsmap 9 up:standby 2023-09-12T16:01:55.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-12T16:01:55.673 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:55.703 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:01:55 smithi130 systemd[1]: Started Ceph mds.g for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:55.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/866670919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b08101070 msgr2=0x7f7b08107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:55.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/866670919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b08101070 0x7f7b08107920 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000b0d0 tx=0x7f7b0000b3e0 comp rx=0 tx=0).stop 2023-09-12T16:01:55.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/866670919 shutdown_connections 2023-09-12T16:01:55.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/866670919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b08101070 0x7f7b08107920 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:55.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/866670919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b080ff4e0 0x7f7b080ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:55.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/866670919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b081006d0 0x7f7b08100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:55.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/866670919 >> 172.21.15.130:0/866670919 conn(0x7f7b080faa30 msgr2=0x7f7b080fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:55.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/866670919 shutdown_connections 2023-09-12T16:01:55.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/866670919 wait complete. 2023-09-12T16:01:55.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 Processor -- start 2023-09-12T16:01:55.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.811+0000 7f7b0f7bc700 1 -- start start 2023-09-12T16:01:55.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0f7bc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b080ff4e0 0x7f7b08195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:55.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0f7bc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b081006d0 0x7f7b081957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:55.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0f7bc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b08101070 0x7f7b0819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:55.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0f7bc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0819cde0 con 0x7f7b081006d0 2023-09-12T16:01:55.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0f7bc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0819cf20 con 0x7f7b08101070 2023-09-12T16:01:55.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0f7bc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0819d0b0 con 0x7f7b080ff4e0 2023-09-12T16:01:55.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0cd57700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b081006d0 0x7f7b081957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:55.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0cd57700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b081006d0 0x7f7b081957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53126/0 (socket says 172.21.15.130:53126) 2023-09-12T16:01:55.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0cd57700 1 -- 172.21.15.130:0/3075212323 learned_addr learned my addr 172.21.15.130:0/3075212323 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:55.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0d558700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b080ff4e0 0x7f7b08195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:55.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0dd59700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b08101070 0x7f7b0819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:55.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0cd57700 1 -- 172.21.15.130:0/3075212323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b080ff4e0 msgr2=0x7f7b08195260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:55.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0cd57700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b080ff4e0 0x7f7b08195260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:55.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0cd57700 1 -- 172.21.15.130:0/3075212323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b08101070 msgr2=0x7f7b0819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:55.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0cd57700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b08101070 0x7f7b0819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:55.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0cd57700 1 -- 172.21.15.130:0/3075212323 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b00009d00 con 0x7f7b081006d0 2023-09-12T16:01:55.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0cd57700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b081006d0 0x7f7b081957a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d980 tx=0x7f7b0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:55.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7afe7fc700 1 -- 172.21.15.130:0/3075212323 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04004220 con 0x7f7b081006d0 2023-09-12T16:01:55.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7afe7fc700 1 -- 172.21.15.130:0/3075212323 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b04007640 con 0x7f7b081006d0 2023-09-12T16:01:55.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0819d390 con 0x7f7b081006d0 2023-09-12T16:01:55.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7afe7fc700 1 -- 172.21.15.130:0/3075212323 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04005610 con 0x7f7b081006d0 2023-09-12T16:01:55.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.812+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0819d8a0 con 0x7f7b081006d0 2023-09-12T16:01:55.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.813+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b08106a00 con 0x7f7b081006d0 2023-09-12T16:01:55.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.814+0000 7f7afe7fc700 1 -- 172.21.15.130:0/3075212323 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f7b04004750 con 0x7f7b081006d0 2023-09-12T16:01:55.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.815+0000 7f7afe7fc700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a0690 0x7f7af40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:55.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.815+0000 7f7b0d558700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a0690 0x7f7af40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:55.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.815+0000 7f7afe7fc700 1 -- 172.21.15.130:0/3075212323 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b040cba10 con 0x7f7b081006d0 2023-09-12T16:01:55.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.815+0000 7f7b0d558700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a0690 0x7f7af40a2b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b08100530 tx=0x7f7af800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:55.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:55.818+0000 7f7afe7fc700 1 -- 172.21.15.130:0/3075212323 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7b040959e0 con 0x7f7b081006d0 2023-09-12T16:01:56.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.067+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b0810af40 con 0x7f7b081006d0 2023-09-12T16:01:56.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.068+0000 7f7afe7fc700 1 -- 172.21.15.130:0/3075212323 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8474 (secure 0 0 0) 0x7f7b04099000 con 0x7f7b081006d0 2023-09-12T16:01:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:56.069 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24557,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24592,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-12T16:01:56.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.070+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a0690 msgr2=0x7f7af40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:56.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.070+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a0690 0x7f7af40a2b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b08100530 tx=0x7f7af800b410 comp rx=0 tx=0).stop 2023-09-12T16:01:56.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.070+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b081006d0 msgr2=0x7f7b081957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:56.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.070+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b081006d0 0x7f7b081957a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d980 tx=0x7f7b0400dc90 comp rx=0 tx=0).stop 2023-09-12T16:01:56.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.070+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 shutdown_connections 2023-09-12T16:01:56.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.070+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a0690 0x7f7af40a2b50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:56.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.070+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b08101070 0x7f7b0819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:56.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.070+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b080ff4e0 0x7f7b08195260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:56.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.070+0000 7f7b0f7bc700 1 --2- 172.21.15.130:0/3075212323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b081006d0 0x7f7b081957a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:56.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.071+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 >> 172.21.15.130:0/3075212323 conn(0x7f7b080faa30 msgr2=0x7f7b08103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:56.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.071+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 shutdown_connections 2023-09-12T16:01:56.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:56.071+0000 7f7b0f7bc700 1 -- 172.21.15.130:0/3075212323 wait complete. 2023-09-12T16:01:56.074 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 11 2023-09-12T16:01:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:55 smithi130 ceph-mon[98670]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:56.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:56.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:56.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:56.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:55 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:56.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:55 smithi159 ceph-mon[102522]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:56.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:56.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:56.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:56.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:56.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:56.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:55 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:56.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:55 smithi203 ceph-mon[103431]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:56.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:56.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:56.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:56.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:56.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:56.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:55 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:56.772 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:01:56 smithi130 systemd[1]: Starting Ceph mds.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:57.225 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:01:57 smithi130 podman[154306]: 2023-09-12T16:01:57.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:56 smithi130 ceph-mon[98670]: Deploying daemon mds.a on smithi130 2023-09-12T16:01:57.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:56 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.130:6838/2430896629,v1:172.21.15.130:6839/2430896629] up:boot 2023-09-12T16:01:57.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:56 smithi130 ceph-mon[98670]: fsmap 10 up:standby 2023-09-12T16:01:57.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:56 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-12T16:01:57.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:56 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3075212323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:56 smithi159 ceph-mon[102522]: Deploying daemon mds.a on smithi130 2023-09-12T16:01:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:56 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.130:6838/2430896629,v1:172.21.15.130:6839/2430896629] up:boot 2023-09-12T16:01:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:56 smithi159 ceph-mon[102522]: fsmap 10 up:standby 2023-09-12T16:01:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:56 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-12T16:01:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:56 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3075212323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:57.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:56 smithi203 ceph-mon[103431]: Deploying daemon mds.a on smithi130 2023-09-12T16:01:57.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:56 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.130:6838/2430896629,v1:172.21.15.130:6839/2430896629] up:boot 2023-09-12T16:01:57.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:56 smithi203 ceph-mon[103431]: fsmap 10 up:standby 2023-09-12T16:01:57.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:56 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-12T16:01:57.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:56 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3075212323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:57.725 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:01:57 smithi130 bash[154306]: 5885aca20228db4199c6aa918cfdc160b7d647153ba949e11e18dccd34bb941a 2023-09-12T16:01:57.725 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:01:57 smithi130 ceph-mds[154346]: -- 172.21.15.130:0/1570682738 <== mon.0 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560817e7d1e0 con 0x560817e52c00 2023-09-12T16:01:57.725 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:01:57 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-a[154322]: starting mds.a at 2023-09-12T16:01:57.726 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:01:57 smithi130 systemd[1]: Started Ceph mds.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:01:58.085 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:01:58.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.217+0000 7fc612fdb700 1 -- 172.21.15.130:0/2972219859 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 msgr2=0x7fc60c0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:58.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.217+0000 7fc612fdb700 1 --2- 172.21.15.130:0/2972219859 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 0x7fc60c0ff910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc6000099c0 tx=0x7fc600009cd0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.217+0000 7fc612fdb700 1 -- 172.21.15.130:0/2972219859 shutdown_connections 2023-09-12T16:01:58.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.217+0000 7fc612fdb700 1 --2- 172.21.15.130:0/2972219859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60c101080 0x7fc60c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.217+0000 7fc612fdb700 1 --2- 172.21.15.130:0/2972219859 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 0x7fc60c0ff910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.217+0000 7fc612fdb700 1 --2- 172.21.15.130:0/2972219859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc60c1006e0 0x7fc60c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.217+0000 7fc612fdb700 1 -- 172.21.15.130:0/2972219859 >> 172.21.15.130:0/2972219859 conn(0x7fc60c0faa20 msgr2=0x7fc60c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.217+0000 7fc612fdb700 1 -- 172.21.15.130:0/2972219859 shutdown_connections 2023-09-12T16:01:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.217+0000 7fc612fdb700 1 -- 172.21.15.130:0/2972219859 wait complete. 2023-09-12T16:01:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc612fdb700 1 Processor -- start 2023-09-12T16:01:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc612fdb700 1 -- start start 2023-09-12T16:01:58.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc612fdb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 0x7fc60c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc612fdb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc60c1006e0 0x7fc60c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc612fdb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60c101080 0x7fc60c19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc612fdb700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c19de50 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc612fdb700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c19df90 con 0x7fc60c101080 2023-09-12T16:01:58.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc612fdb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c19e120 con 0x7fc60c1006e0 2023-09-12T16:01:58.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc610d77700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 0x7fc60c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:58.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc610d77700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 0x7fc60c1962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53148/0 (socket says 172.21.15.130:53148) 2023-09-12T16:01:58.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc610d77700 1 -- 172.21.15.130:0/1418497351 learned_addr learned my addr 172.21.15.130:0/1418497351 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:01:58.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc611578700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60c101080 0x7fc60c19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:58.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc610d77700 1 -- 172.21.15.130:0/1418497351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc60c1006e0 msgr2=0x7fc60c196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:58.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc60bfff700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc60c1006e0 0x7fc60c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:58.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc610d77700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc60c1006e0 0x7fc60c196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc610d77700 1 -- 172.21.15.130:0/1418497351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60c101080 msgr2=0x7fc60c19d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:58.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc610d77700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60c101080 0x7fc60c19d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc610d77700 1 -- 172.21.15.130:0/1418497351 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000095f0 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc611578700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60c101080 0x7fc60c19d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:01:58.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc60bfff700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc60c1006e0 0x7fc60c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:01:58.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.218+0000 7fc610d77700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 0x7fc60c1962d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc600009f50 tx=0x7fc6000057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:58.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.219+0000 7fc609ffb700 1 -- 172.21.15.130:0/1418497351 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60001d070 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.219+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c19e3a0 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.219+0000 7fc609ffb700 1 -- 172.21.15.130:0/1418497351 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc600004110 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.219+0000 7fc609ffb700 1 -- 172.21.15.130:0/1418497351 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600022dd0 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.219+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c19e850 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.220+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c10ad80 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.221+0000 7fc609ffb700 1 -- 172.21.15.130:0/1418497351 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fc600022500 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.221+0000 7fc609ffb700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc5fc0a0690 0x7fc5fc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:01:58.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.221+0000 7fc60bfff700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc5fc0a0690 0x7fc5fc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:01:58.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.221+0000 7fc609ffb700 1 -- 172.21.15.130:0/1418497351 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6000cd2a0 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.222+0000 7fc60bfff700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc5fc0a0690 0x7fc5fc0a2b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8007900 tx=0x7fc5f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:01:58.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.225+0000 7fc609ffb700 1 -- 172.21.15.130:0/1418497351 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc600097300 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:57 smithi203 ceph-mon[103431]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:58.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:58.252 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:57 smithi159 ceph-mon[102522]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:58.252 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:58.252 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:58.252 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:58.252 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:58.253 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:58.253 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:58.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.459+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc60c10afc0 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.459+0000 7fc609ffb700 1 -- 172.21.15.130:0/1418497351 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9269 (secure 0 0 0) 0x7fc60009a920 con 0x7fc60c0ff4f0 2023-09-12T16:01:58.459 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:01:58.460 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24557,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24592,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-12T16:01:58.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc5fc0a0690 msgr2=0x7fc5fc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:58.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc5fc0a0690 0x7fc5fc0a2b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8007900 tx=0x7fc5f8008040 comp rx=0 tx=0).stop 2023-09-12T16:01:58.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 msgr2=0x7fc60c1962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:01:58.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 0x7fc60c1962d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc600009f50 tx=0x7fc6000057d0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 shutdown_connections 2023-09-12T16:01:58.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc5fc0a0690 0x7fc5fc0a2b50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60c101080 0x7fc60c19d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc60c0ff4f0 0x7fc60c1962d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 --2- 172.21.15.130:0/1418497351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc60c1006e0 0x7fc60c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:01:58.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 >> 172.21.15.130:0/1418497351 conn(0x7fc60c0faa20 msgr2=0x7fc60c0687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:01:58.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 shutdown_connections 2023-09-12T16:01:58.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:01:58.462+0000 7fc612fdb700 1 -- 172.21.15.130:0/1418497351 wait complete. 2023-09-12T16:01:58.464 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 12 2023-09-12T16:01:58.472 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:57 smithi130 ceph-mon[98670]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:01:58.473 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:58.473 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:58.473 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:01:58.473 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-12T16:01:58.473 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-12T16:01:58.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:01:59.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:58 smithi159 ceph-mon[102522]: Deploying daemon mds.b on smithi159 2023-09-12T16:01:59.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:58 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.130:6840/1672964981,v1:172.21.15.130:6841/1672964981] up:boot 2023-09-12T16:01:59.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:58 smithi159 ceph-mon[102522]: fsmap 11 up:standby 2023-09-12T16:01:59.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:58 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-12T16:01:59.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:58 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1418497351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:59.227 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:01:58 smithi159 systemd[1]: Starting Ceph mds.b for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:01:59.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:58 smithi203 ceph-mon[103431]: Deploying daemon mds.b on smithi159 2023-09-12T16:01:59.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:58 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.130:6840/1672964981,v1:172.21.15.130:6841/1672964981] up:boot 2023-09-12T16:01:59.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:58 smithi203 ceph-mon[103431]: fsmap 11 up:standby 2023-09-12T16:01:59.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:58 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-12T16:01:59.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:58 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1418497351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:59.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:58 smithi130 ceph-mon[98670]: Deploying daemon mds.b on smithi159 2023-09-12T16:01:59.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:58 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.130:6840/1672964981,v1:172.21.15.130:6841/1672964981] up:boot 2023-09-12T16:01:59.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:58 smithi130 ceph-mon[98670]: fsmap 11 up:standby 2023-09-12T16:01:59.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:58 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-12T16:01:59.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:58 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1418497351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:01:59.552 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:01:59 smithi159 podman[139164]: 2023-09-12T16:01:59.975 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:01:59 smithi159 ceph-mds[139204]: -- 172.21.15.159:0/2772499449 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ac541611e0 con 0x55ac54136800 2023-09-12T16:01:59.976 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:01:59 smithi159 bash[139164]: fa58e30589acec557b1e448de88e9b93ffd84b5870e3052e0f6f6c851d66cc1b 2023-09-12T16:01:59.976 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:01:59 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-b[139180]: starting mds.b at 2023-09-12T16:01:59.976 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:01:59 smithi159 systemd[1]: Started Ceph mds.b for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:02:00.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:59 smithi203 ceph-mon[103431]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:00.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:01:59 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:02:00.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:59 smithi130 ceph-mon[98670]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:00.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:01:59 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:02:00.476 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:02:00.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:59 smithi159 ceph-mon[102522]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:00.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:00.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:01:59 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:02:00.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.607+0000 7f93fbd81700 1 -- 172.21.15.130:0/2768469540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41062d0 msgr2=0x7f93f4108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:00.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.607+0000 7f93fbd81700 1 --2- 172.21.15.130:0/2768469540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41062d0 0x7f93f4108740 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00b0a0 tx=0x7f93ec00b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.607+0000 7f93fbd81700 1 -- 172.21.15.130:0/2768469540 shutdown_connections 2023-09-12T16:02:00.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.607+0000 7f93fbd81700 1 --2- 172.21.15.130:0/2768469540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41062d0 0x7f93f4108740 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.607+0000 7f93fbd81700 1 --2- 172.21.15.130:0/2768469540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93f4101070 0x7f93f4103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.607+0000 7f93fbd81700 1 --2- 172.21.15.130:0/2768469540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f41039a0 0x7f93f4105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.607+0000 7f93fbd81700 1 -- 172.21.15.130:0/2768469540 >> 172.21.15.130:0/2768469540 conn(0x7f93f40faa20 msgr2=0x7f93f40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:00.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.608+0000 7f93fbd81700 1 -- 172.21.15.130:0/2768469540 shutdown_connections 2023-09-12T16:02:00.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.608+0000 7f93fbd81700 1 -- 172.21.15.130:0/2768469540 wait complete. 2023-09-12T16:02:00.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.608+0000 7f93fbd81700 1 Processor -- start 2023-09-12T16:02:00.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.608+0000 7f93fbd81700 1 -- start start 2023-09-12T16:02:00.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93fbd81700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93f4101070 0x7f93f4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:00.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93fbd81700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41039a0 0x7f93f4196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:00.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93fbd81700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f41062d0 0x7f93f419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:00.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93fbd81700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419dec0 con 0x7f93f41039a0 2023-09-12T16:02:00.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93fbd81700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419e000 con 0x7f93f4101070 2023-09-12T16:02:00.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93fbd81700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419e190 con 0x7f93f41062d0 2023-09-12T16:02:00.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f931c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41039a0 0x7f93f4196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:00.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f931c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41039a0 0x7f93f4196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53164/0 (socket says 172.21.15.130:53164) 2023-09-12T16:02:00.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f931c700 1 -- 172.21.15.130:0/3689116121 learned_addr learned my addr 172.21.15.130:0/3689116121 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:00.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f9b1d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93f4101070 0x7f93f4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:00.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93fa31e700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f41062d0 0x7f93f419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:00.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f931c700 1 -- 172.21.15.130:0/3689116121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f41062d0 msgr2=0x7f93f419d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:00.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f931c700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f41062d0 0x7f93f419d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f931c700 1 -- 172.21.15.130:0/3689116121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93f4101070 msgr2=0x7f93f4196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:00.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f931c700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93f4101070 0x7f93f4196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f931c700 1 -- 172.21.15.130:0/3689116121 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec009d00 con 0x7f93f41039a0 2023-09-12T16:02:00.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93f931c700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41039a0 0x7f93f4196880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9a0 tx=0x7f93e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:00.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93eaffd700 1 -- 172.21.15.130:0/3689116121 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4009790 con 0x7f93f41039a0 2023-09-12T16:02:00.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93eaffd700 1 -- 172.21.15.130:0/3689116121 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e4010430 con 0x7f93f41039a0 2023-09-12T16:02:00.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f419e470 con 0x7f93f41039a0 2023-09-12T16:02:00.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93eaffd700 1 -- 172.21.15.130:0/3689116121 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4018680 con 0x7f93f41039a0 2023-09-12T16:02:00.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.609+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f419e980 con 0x7f93f41039a0 2023-09-12T16:02:00.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.610+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f40fc5a0 con 0x7f93f41039a0 2023-09-12T16:02:00.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.612+0000 7f93eaffd700 1 -- 172.21.15.130:0/3689116121 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f93e40105a0 con 0x7f93f41039a0 2023-09-12T16:02:00.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.612+0000 7f93eaffd700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f93e00a0690 0x7f93e00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:00.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.612+0000 7f93f9b1d700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f93e00a0690 0x7f93e00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:00.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.612+0000 7f93eaffd700 1 -- 172.21.15.130:0/3689116121 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f93e4014070 con 0x7f93f41039a0 2023-09-12T16:02:00.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.615+0000 7f93f9b1d700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f93e00a0690 0x7f93e00a2b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009c60 tx=0x7f93f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:00.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.617+0000 7f93eaffd700 1 -- 172.21.15.130:0/3689116121 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f93e4095a50 con 0x7f93f41039a0 2023-09-12T16:02:00.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.869+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93f402d020 con 0x7f93f41039a0 2023-09-12T16:02:00.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.870+0000 7f93eaffd700 1 -- 172.21.15.130:0/3689116121 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10064 (secure 0 0 0) 0x7f93e4099070 con 0x7f93f41039a0 2023-09-12T16:02:00.870 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:00.870 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24557,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24592,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24593,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-09-12T16:02:00.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f93e00a0690 msgr2=0x7f93e00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:00.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f93e00a0690 0x7f93e00a2b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009c60 tx=0x7f93f0009260 comp rx=0 tx=0).stop 2023-09-12T16:02:00.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41039a0 msgr2=0x7f93f4196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:00.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41039a0 0x7f93f4196880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9a0 tx=0x7f93e400ecb0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 shutdown_connections 2023-09-12T16:02:00.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f93e00a0690 0x7f93e00a2b50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f41062d0 0x7f93f419d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93f4101070 0x7f93f4196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 --2- 172.21.15.130:0/3689116121 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93f41039a0 0x7f93f4196880 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:00.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 >> 172.21.15.130:0/3689116121 conn(0x7f93f40faa20 msgr2=0x7f93f4101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:00.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 shutdown_connections 2023-09-12T16:02:00.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:00.872+0000 7f93fbd81700 1 -- 172.21.15.130:0/3689116121 wait complete. 2023-09-12T16:02:00.874 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 13 2023-09-12T16:02:00.889 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:02:01.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.021+0000 7f01d9130700 1 -- 172.21.15.130:0/2659756506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d41018c0 msgr2=0x7f01d4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.021+0000 7f01d9130700 1 --2- 172.21.15.130:0/2659756506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d41018c0 0x7f01d4101d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009990 tx=0x7f01c8009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.021+0000 7f01d9130700 1 -- 172.21.15.130:0/2659756506 shutdown_connections 2023-09-12T16:02:01.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.021+0000 7f01d9130700 1 --2- 172.21.15.130:0/2659756506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d4102260 0x7f01d4108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.021+0000 7f01d9130700 1 --2- 172.21.15.130:0/2659756506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01d40fe8f0 0x7f01d40fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.021+0000 7f01d9130700 1 --2- 172.21.15.130:0/2659756506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d41018c0 0x7f01d4101d20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.021+0000 7f01d9130700 1 -- 172.21.15.130:0/2659756506 >> 172.21.15.130:0/2659756506 conn(0x7f01d40fa200 msgr2=0x7f01d40fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:01.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.021+0000 7f01d9130700 1 -- 172.21.15.130:0/2659756506 shutdown_connections 2023-09-12T16:02:01.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.021+0000 7f01d9130700 1 -- 172.21.15.130:0/2659756506 wait complete. 2023-09-12T16:02:01.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.022+0000 7f01d9130700 1 Processor -- start 2023-09-12T16:02:01.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.022+0000 7f01d9130700 1 -- start start 2023-09-12T16:02:01.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.022+0000 7f01d9130700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01d40fe8f0 0x7f01d41962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.022+0000 7f01d9130700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d41018c0 0x7f01d4196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.022+0000 7f01d9130700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d4102260 0x7f01d419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.022+0000 7f01d9130700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d419de50 con 0x7f01d4102260 2023-09-12T16:02:01.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.022+0000 7f01d9130700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d419df90 con 0x7f01d41018c0 2023-09-12T16:02:01.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.022+0000 7f01d9130700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d419e120 con 0x7f01d40fe8f0 2023-09-12T16:02:01.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d4102260 0x7f01d419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d4102260 0x7f01d419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53178/0 (socket says 172.21.15.130:53178) 2023-09-12T16:02:01.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d2d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01d40fe8f0 0x7f01d41962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d359e700 1 -- 172.21.15.130:0/2775326223 learned_addr learned my addr 172.21.15.130:0/2775326223 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:01.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d259c700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d41018c0 0x7f01d4196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d359e700 1 -- 172.21.15.130:0/2775326223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01d40fe8f0 msgr2=0x7f01d41962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d359e700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01d40fe8f0 0x7f01d41962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d359e700 1 -- 172.21.15.130:0/2775326223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d41018c0 msgr2=0x7f01d4196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d359e700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d41018c0 0x7f01d4196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d359e700 1 -- 172.21.15.130:0/2775326223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c80095f0 con 0x7f01d4102260 2023-09-12T16:02:01.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d359e700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d4102260 0x7f01d419d880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00ef30 tx=0x7f01cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:01.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01bbfff700 1 -- 172.21.15.130:0/2775326223 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc018070 con 0x7f01d4102260 2023-09-12T16:02:01.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01bbfff700 1 -- 172.21.15.130:0/2775326223 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01cc00f040 con 0x7f01d4102260 2023-09-12T16:02:01.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d419e3a0 con 0x7f01d4102260 2023-09-12T16:02:01.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01bbfff700 1 -- 172.21.15.130:0/2775326223 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc00cd00 con 0x7f01d4102260 2023-09-12T16:02:01.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.023+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d419e860 con 0x7f01d4102260 2023-09-12T16:02:01.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.024+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d410be80 con 0x7f01d4102260 2023-09-12T16:02:01.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.025+0000 7f01bbfff700 1 -- 172.21.15.130:0/2775326223 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f01cc003a00 con 0x7f01d4102260 2023-09-12T16:02:01.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.026+0000 7f01bbfff700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01bc0a0690 0x7f01bc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.026+0000 7f01bbfff700 1 -- 172.21.15.130:0/2775326223 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01cc0cb200 con 0x7f01d4102260 2023-09-12T16:02:01.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.026+0000 7f01d2d9d700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01bc0a0690 0x7f01bc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.026+0000 7f01d2d9d700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01bc0a0690 0x7f01bc0a2b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01d40687c0 tx=0x7f01c4007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:01.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.030+0000 7f01bbfff700 1 -- 172.21.15.130:0/2775326223 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f01cc095260 con 0x7f01d4102260 2023-09-12T16:02:01.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:00 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] up:boot 2023-09-12T16:02:01.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:00 smithi130 ceph-mon[98670]: fsmap 12 up:standby 2023-09-12T16:02:01.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-12T16:02:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:00 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:00 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3689116121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:01.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.239+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01d41075b0 con 0x7f01d4102260 2023-09-12T16:02:01.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.240+0000 7f01bbfff700 1 -- 172.21.15.130:0/2775326223 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f01cc098880 con 0x7f01d4102260 2023-09-12T16:02:01.240 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:01.240 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":77,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:01:23.890141+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:01.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.242+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01bc0a0690 msgr2=0x7f01bc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.242+0000 7f01d9130700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01bc0a0690 0x7f01bc0a2b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01d40687c0 tx=0x7f01c4007400 comp rx=0 tx=0).stop 2023-09-12T16:02:01.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.242+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d4102260 msgr2=0x7f01d419d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.242+0000 7f01d9130700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d4102260 0x7f01d419d880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00ef30 tx=0x7f01cc00c3a0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.242+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 shutdown_connections 2023-09-12T16:02:01.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.242+0000 7f01d9130700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01bc0a0690 0x7f01bc0a2b50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.242+0000 7f01d9130700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d4102260 0x7f01d419d880 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.242+0000 7f01d9130700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01d40fe8f0 0x7f01d41962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.242+0000 7f01d9130700 1 --2- 172.21.15.130:0/2775326223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d41018c0 0x7f01d4196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.243+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 >> 172.21.15.130:0/2775326223 conn(0x7f01d40fa200 msgr2=0x7f01d40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:01.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.243+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 shutdown_connections 2023-09-12T16:02:01.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.243+0000 7f01d9130700 1 -- 172.21.15.130:0/2775326223 wait complete. 2023-09-12T16:02:01.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:00 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] up:boot 2023-09-12T16:02:01.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:00 smithi203 ceph-mon[103431]: fsmap 12 up:standby 2023-09-12T16:02:01.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-12T16:02:01.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:00 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:01.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:00 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3689116121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:01.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T16:02:01.258 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:02:01.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.386+0000 7f594b1a1700 1 -- 172.21.15.130:0/3705249855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f59441022d0 msgr2=0x7f5944108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.386+0000 7f594b1a1700 1 --2- 172.21.15.130:0/3705249855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f59441022d0 0x7f5944108b80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f593c00b0a0 tx=0x7f593c00b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.386+0000 7f594b1a1700 1 -- 172.21.15.130:0/3705249855 shutdown_connections 2023-09-12T16:02:01.387 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.386+0000 7f594b1a1700 1 --2- 172.21.15.130:0/3705249855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f59441022d0 0x7f5944108b80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.387 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.386+0000 7f594b1a1700 1 --2- 172.21.15.130:0/3705249855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59440fe8c0 0x7f59440fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.387 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.386+0000 7f594b1a1700 1 --2- 172.21.15.130:0/3705249855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5944101930 0x7f5944101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.387 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.386+0000 7f594b1a1700 1 -- 172.21.15.130:0/3705249855 >> 172.21.15.130:0/3705249855 conn(0x7f59440fa1d0 msgr2=0x7f59440fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:01.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.386+0000 7f594b1a1700 1 -- 172.21.15.130:0/3705249855 shutdown_connections 2023-09-12T16:02:01.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.386+0000 7f594b1a1700 1 -- 172.21.15.130:0/3705249855 wait complete. 2023-09-12T16:02:01.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f594b1a1700 1 Processor -- start 2023-09-12T16:02:01.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f594b1a1700 1 -- start start 2023-09-12T16:02:01.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f594b1a1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59440fe8c0 0x7f5944196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f594b1a1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5944101930 0x7f59441969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f594b1a1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59441022d0 0x7f594419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f594b1a1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594419e000 con 0x7f5944101930 2023-09-12T16:02:01.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f594b1a1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594419e140 con 0x7f59441022d0 2023-09-12T16:02:01.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f594b1a1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594419e2d0 con 0x7f59440fe8c0 2023-09-12T16:02:01.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f5943fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5944101930 0x7f59441969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f5943fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5944101930 0x7f59441969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53188/0 (socket says 172.21.15.130:53188) 2023-09-12T16:02:01.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f5948f3d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59440fe8c0 0x7f5944196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.387+0000 7f5943fff700 1 -- 172.21.15.130:0/2600898822 learned_addr learned my addr 172.21.15.130:0/2600898822 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:01.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f594973e700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59441022d0 0x7f594419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f5943fff700 1 -- 172.21.15.130:0/2600898822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59440fe8c0 msgr2=0x7f5944196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f5943fff700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59440fe8c0 0x7f5944196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f5943fff700 1 -- 172.21.15.130:0/2600898822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59441022d0 msgr2=0x7f594419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f5943fff700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59441022d0 0x7f594419da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f5943fff700 1 -- 172.21.15.130:0/2600898822 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c009d00 con 0x7f5944101930 2023-09-12T16:02:01.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f594973e700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59441022d0 0x7f594419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:01.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f5943fff700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5944101930 0x7f59441969c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f593000b530 tx=0x7f593000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:01.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f5941ffb700 1 -- 172.21.15.130:0/2600898822 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930011830 con 0x7f5944101930 2023-09-12T16:02:01.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594419e5b0 con 0x7f5944101930 2023-09-12T16:02:01.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f5941ffb700 1 -- 172.21.15.130:0/2600898822 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5930011e50 con 0x7f5944101930 2023-09-12T16:02:01.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f5941ffb700 1 -- 172.21.15.130:0/2600898822 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000f2e0 con 0x7f5944101930 2023-09-12T16:02:01.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.388+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594419eb10 con 0x7f5944101930 2023-09-12T16:02:01.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.389+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594410bea0 con 0x7f5944101930 2023-09-12T16:02:01.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.390+0000 7f5941ffb700 1 -- 172.21.15.130:0/2600898822 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f5930010380 con 0x7f5944101930 2023-09-12T16:02:01.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.391+0000 7f5941ffb700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59340a0640 0x7f59340a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.391+0000 7f5948f3d700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59340a0640 0x7f59340a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.391+0000 7f5941ffb700 1 -- 172.21.15.130:0/2600898822 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f59300cb480 con 0x7f5944101930 2023-09-12T16:02:01.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.391+0000 7f5948f3d700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59340a0640 0x7f59340a2b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59440674a0 tx=0x7f5938007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:01.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.395+0000 7f5941ffb700 1 -- 172.21.15.130:0/2600898822 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5930095450 con 0x7f5944101930 2023-09-12T16:02:01.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:00 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] up:boot 2023-09-12T16:02:01.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:00 smithi159 ceph-mon[102522]: fsmap 12 up:standby 2023-09-12T16:02:01.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-12T16:02:01.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:00 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:01.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:00 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3689116121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:01.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.596+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f594410c060 con 0x7f5944101930 2023-09-12T16:02:01.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.597+0000 7f5941ffb700 1 -- 172.21.15.130:0/2600898822 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5930098a70 con 0x7f5944101930 2023-09-12T16:02:01.597 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:02:01.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59340a0640 msgr2=0x7f59340a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59340a0640 0x7f59340a2b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59440674a0 tx=0x7f5938007400 comp rx=0 tx=0).stop 2023-09-12T16:02:01.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5944101930 msgr2=0x7f59441969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5944101930 0x7f59441969c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f593000b530 tx=0x7f593000b840 comp rx=0 tx=0).stop 2023-09-12T16:02:01.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 shutdown_connections 2023-09-12T16:02:01.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59340a0640 0x7f59340a2b00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59441022d0 0x7f594419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59440fe8c0 0x7f5944196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 --2- 172.21.15.130:0/2600898822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5944101930 0x7f59441969c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 >> 172.21.15.130:0/2600898822 conn(0x7f59440fa1d0 msgr2=0x7f59440fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:01.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 shutdown_connections 2023-09-12T16:02:01.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.599+0000 7f594b1a1700 1 -- 172.21.15.130:0/2600898822 wait complete. 2023-09-12T16:02:01.612 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-12T16:02:01.747 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 -- 172.21.15.130:0/1463449529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c100aa0 msgr2=0x7f895c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.747 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 --2- 172.21.15.130:0/1463449529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c100aa0 0x7f895c100ec0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8954009990 tx=0x7f8954009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 -- 172.21.15.130:0/1463449529 shutdown_connections 2023-09-12T16:02:01.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 --2- 172.21.15.130:0/1463449529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895c1000b0 0x7f895c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 --2- 172.21.15.130:0/1463449529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c100aa0 0x7f895c100ec0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 --2- 172.21.15.130:0/1463449529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f895c0ff6f0 0x7f895c0ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 -- 172.21.15.130:0/1463449529 >> 172.21.15.130:0/1463449529 conn(0x7f895c0fa1c0 msgr2=0x7f895c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:01.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 -- 172.21.15.130:0/1463449529 shutdown_connections 2023-09-12T16:02:01.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 -- 172.21.15.130:0/1463449529 wait complete. 2023-09-12T16:02:01.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 Processor -- start 2023-09-12T16:02:01.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.746+0000 7f896137e700 1 -- start start 2023-09-12T16:02:01.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f896137e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c0ff6f0 0x7f895c0ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f896137e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f895c1000b0 0x7f895c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f896137e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895c100aa0 0x7f895c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f896137e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c19ddf0 con 0x7f895c0ff6f0 2023-09-12T16:02:01.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f896137e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c19df60 con 0x7f895c1000b0 2023-09-12T16:02:01.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f896137e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c19e120 con 0x7f895c100aa0 2023-09-12T16:02:01.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c0ff6f0 0x7f895c0ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c0ff6f0 0x7f895c0ff1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53200/0 (socket says 172.21.15.130:53200) 2023-09-12T16:02:01.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895affd700 1 -- 172.21.15.130:0/2358340799 learned_addr learned my addr 172.21.15.130:0/2358340799 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:01.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895a7fc700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f895c1000b0 0x7f895c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895b7fe700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895c100aa0 0x7f895c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895affd700 1 -- 172.21.15.130:0/2358340799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895c100aa0 msgr2=0x7f895c19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895affd700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895c100aa0 0x7f895c19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895affd700 1 -- 172.21.15.130:0/2358340799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f895c1000b0 msgr2=0x7f895c19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895affd700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f895c1000b0 0x7f895c19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895affd700 1 -- 172.21.15.130:0/2358340799 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540095f0 con 0x7f895c0ff6f0 2023-09-12T16:02:01.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f895affd700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c0ff6f0 0x7f895c0ff1b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8954004760 tx=0x7f8954004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:01.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f8953fff700 1 -- 172.21.15.130:0/2358340799 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895401d070 con 0x7f895c0ff6f0 2023-09-12T16:02:01.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c19e3a0 con 0x7f895c0ff6f0 2023-09-12T16:02:01.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f8953fff700 1 -- 172.21.15.130:0/2358340799 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8954004830 con 0x7f895c0ff6f0 2023-09-12T16:02:01.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f8953fff700 1 -- 172.21.15.130:0/2358340799 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954005350 con 0x7f895c0ff6f0 2023-09-12T16:02:01.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.747+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c19e760 con 0x7f895c0ff6f0 2023-09-12T16:02:01.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.748+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c10ad40 con 0x7f895c0ff6f0 2023-09-12T16:02:01.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.749+0000 7f8953fff700 1 -- 172.21.15.130:0/2358340799 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f8954005510 con 0x7f895c0ff6f0 2023-09-12T16:02:01.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.750+0000 7f8953fff700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89480a0690 0x7f89480a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:01.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.750+0000 7f895a7fc700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89480a0690 0x7f89480a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:01.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.750+0000 7f8953fff700 1 -- 172.21.15.130:0/2358340799 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f89540cc8c0 con 0x7f895c0ff6f0 2023-09-12T16:02:01.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.750+0000 7f895a7fc700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89480a0690 0x7f89480a2b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8944009c60 tx=0x7f8944009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:01.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.754+0000 7f8953fff700 1 -- 172.21.15.130:0/2358340799 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f895405ae10 con 0x7f895c0ff6f0 2023-09-12T16:02:01.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.958+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f895c19e9e0 con 0x7f895c0ff6f0 2023-09-12T16:02:01.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.958+0000 7f8953fff700 1 -- 172.21.15.130:0/2358340799 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8954096920 con 0x7f895c0ff6f0 2023-09-12T16:02:01.958 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:01.958 INFO:teuthology.orchestra.run.smithi130.stdout:[ 2023-09-12T16:02:01.958 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:02:01.959 INFO:teuthology.orchestra.run.smithi130.stdout: "poolnum": 1, 2023-09-12T16:02:01.959 INFO:teuthology.orchestra.run.smithi130.stdout: "poolname": ".mgr" 2023-09-12T16:02:01.959 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:02:01.959 INFO:teuthology.orchestra.run.smithi130.stdout:] 2023-09-12T16:02:01.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.960+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89480a0690 msgr2=0x7f89480a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.960+0000 7f896137e700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89480a0690 0x7f89480a2b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8944009c60 tx=0x7f8944009260 comp rx=0 tx=0).stop 2023-09-12T16:02:01.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.960+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c0ff6f0 msgr2=0x7f895c0ff1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:01.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.960+0000 7f896137e700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c0ff6f0 0x7f895c0ff1b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8954004760 tx=0x7f8954004790 comp rx=0 tx=0).stop 2023-09-12T16:02:01.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.961+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 shutdown_connections 2023-09-12T16:02:01.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.961+0000 7f896137e700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89480a0690 0x7f89480a2b50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.961+0000 7f896137e700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895c100aa0 0x7f895c19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.961+0000 7f896137e700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895c0ff6f0 0x7f895c0ff1b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.961+0000 7f896137e700 1 --2- 172.21.15.130:0/2358340799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f895c1000b0 0x7f895c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:01.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.961+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 >> 172.21.15.130:0/2358340799 conn(0x7f895c0fa1c0 msgr2=0x7f895c0fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:01.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.961+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 shutdown_connections 2023-09-12T16:02:01.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:01.961+0000 7f896137e700 1 -- 172.21.15.130:0/2358340799 wait complete. 2023-09-12T16:02:01.973 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-12T16:02:01.974 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-12T16:02:02.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 -- 172.21.15.130:0/22000517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec01006e0 msgr2=0x7efec0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:02.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 --2- 172.21.15.130:0/22000517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec01006e0 0x7efec0100b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efea80099c0 tx=0x7efea8009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:02.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 -- 172.21.15.130:0/22000517 shutdown_connections 2023-09-12T16:02:02.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 --2- 172.21.15.130:0/22000517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efec0101080 0x7efec0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:02.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 --2- 172.21.15.130:0/22000517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efec00ff4f0 0x7efec00ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:02.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 --2- 172.21.15.130:0/22000517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec01006e0 0x7efec0100b40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:02.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 -- 172.21.15.130:0/22000517 >> 172.21.15.130:0/22000517 conn(0x7efec00faa60 msgr2=0x7efec00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:02.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 -- 172.21.15.130:0/22000517 shutdown_connections 2023-09-12T16:02:02.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 -- 172.21.15.130:0/22000517 wait complete. 2023-09-12T16:02:02.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 Processor -- start 2023-09-12T16:02:02.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.105+0000 7efec6259700 1 -- start start 2023-09-12T16:02:02.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efec6259700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efec00ff4f0 0x7efec00682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:02.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efec6259700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efec01006e0 0x7efec019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:02.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efec6259700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec0101080 0x7efec019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:02.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efec6259700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec019ffc0 con 0x7efec0101080 2023-09-12T16:02:02.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efec6259700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec01a0130 con 0x7efec00ff4f0 2023-09-12T16:02:02.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efec6259700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec01a02f0 con 0x7efec01006e0 2023-09-12T16:02:02.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec0101080 0x7efec019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:02.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec0101080 0x7efec019f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53218/0 (socket says 172.21.15.130:53218) 2023-09-12T16:02:02.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebffff700 1 -- 172.21.15.130:0/2147254465 learned_addr learned my addr 172.21.15.130:0/2147254465 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:02.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebeffd700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efec01006e0 0x7efec019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:02.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebf7fe700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efec00ff4f0 0x7efec00682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:02.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebffff700 1 -- 172.21.15.130:0/2147254465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efec01006e0 msgr2=0x7efec019d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:02.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebffff700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efec01006e0 0x7efec019d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:02.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebffff700 1 -- 172.21.15.130:0/2147254465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efec00ff4f0 msgr2=0x7efec00682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:02.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebffff700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efec00ff4f0 0x7efec00682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:02.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebffff700 1 -- 172.21.15.130:0/2147254465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea80095f0 con 0x7efec0101080 2023-09-12T16:02:02.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebffff700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec0101080 0x7efec019f910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efeb000ef30 tx=0x7efeb000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:02.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebcff9700 1 -- 172.21.15.130:0/2147254465 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0018070 con 0x7efec0101080 2023-09-12T16:02:02.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebcff9700 1 -- 172.21.15.130:0/2147254465 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb000f040 con 0x7efec0101080 2023-09-12T16:02:02.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec01a0570 con 0x7efec0101080 2023-09-12T16:02:02.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efebcff9700 1 -- 172.21.15.130:0/2147254465 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb000cd00 con 0x7efec0101080 2023-09-12T16:02:02.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.106+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec01a0ad0 con 0x7efec0101080 2023-09-12T16:02:02.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.107+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec010ad10 con 0x7efec0101080 2023-09-12T16:02:02.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.108+0000 7efebcff9700 1 -- 172.21.15.130:0/2147254465 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7efeb0003a00 con 0x7efec0101080 2023-09-12T16:02:02.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.109+0000 7efebcff9700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efeac0a06c0 0x7efeac0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:02.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.109+0000 7efebf7fe700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efeac0a06c0 0x7efeac0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:02.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.109+0000 7efebcff9700 1 -- 172.21.15.130:0/2147254465 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efeb00cb1a0 con 0x7efec0101080 2023-09-12T16:02:02.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.109+0000 7efebf7fe700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efeac0a06c0 0x7efeac0a2b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efec0100540 tx=0x7efeb800a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:02.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.113+0000 7efebcff9700 1 -- 172.21.15.130:0/2147254465 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efeb0095200 con 0x7efec0101080 2023-09-12T16:02:02.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:01 smithi130 ceph-mon[98670]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:02.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2775326223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:02.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2600898822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:02:02.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2358340799' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-12T16:02:02.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:01 smithi203 ceph-mon[103431]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:02.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2775326223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:02.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2600898822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:02:02.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2358340799' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-12T16:02:02.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.317+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efec01a0cf0 con 0x7efec0101080 2023-09-12T16:02:02.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:01 smithi159 ceph-mon[102522]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:02.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2775326223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:02.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2600898822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:02:02.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2358340799' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-12T16:02:02.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:02.999+0000 7efebcff9700 1 -- 172.21.15.130:0/2147254465 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7efeb0098820 con 0x7efec0101080 2023-09-12T16:02:03.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.002+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efeac0a06c0 msgr2=0x7efeac0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:03.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.002+0000 7efec6259700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efeac0a06c0 0x7efeac0a2b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efec0100540 tx=0x7efeb800a3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.002+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec0101080 msgr2=0x7efec019f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:03.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.002+0000 7efec6259700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec0101080 0x7efec019f910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efeb000ef30 tx=0x7efeb000c3a0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.003+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 shutdown_connections 2023-09-12T16:02:03.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.003+0000 7efec6259700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efeac0a06c0 0x7efeac0a2b80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.003+0000 7efec6259700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efec0101080 0x7efec019f910 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.003+0000 7efec6259700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efec00ff4f0 0x7efec00682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.003+0000 7efec6259700 1 --2- 172.21.15.130:0/2147254465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efec01006e0 0x7efec019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.003+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 >> 172.21.15.130:0/2147254465 conn(0x7efec00faa60 msgr2=0x7efec00fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:03.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.003+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 shutdown_connections 2023-09-12T16:02:03.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.003+0000 7efec6259700 1 -- 172.21.15.130:0/2147254465 wait complete. 2023-09-12T16:02:03.007 INFO:teuthology.orchestra.run.smithi130.stderr:pool 'cephfs_metadata' created 2023-09-12T16:02:03.018 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-09-12T16:02:03.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.148+0000 7fd025d97700 1 -- 172.21.15.130:0/2382720051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 msgr2=0x7fd0200ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:03.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.148+0000 7fd025d97700 1 --2- 172.21.15.130:0/2382720051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 0x7fd0200ff910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0100099c0 tx=0x7fd010009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.148+0000 7fd025d97700 1 -- 172.21.15.130:0/2382720051 shutdown_connections 2023-09-12T16:02:03.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.148+0000 7fd025d97700 1 --2- 172.21.15.130:0/2382720051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd020101080 0x7fd020107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.148+0000 7fd025d97700 1 --2- 172.21.15.130:0/2382720051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 0x7fd0200ff910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.148+0000 7fd025d97700 1 --2- 172.21.15.130:0/2382720051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0201006e0 0x7fd020100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.148+0000 7fd025d97700 1 -- 172.21.15.130:0/2382720051 >> 172.21.15.130:0/2382720051 conn(0x7fd0200faa60 msgr2=0x7fd0200fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:03.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.148+0000 7fd025d97700 1 -- 172.21.15.130:0/2382720051 shutdown_connections 2023-09-12T16:02:03.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.148+0000 7fd025d97700 1 -- 172.21.15.130:0/2382720051 wait complete. 2023-09-12T16:02:03.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd025d97700 1 Processor -- start 2023-09-12T16:02:03.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd025d97700 1 -- start start 2023-09-12T16:02:03.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd025d97700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 0x7fd020069360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:03.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd025d97700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0201006e0 0x7fd02019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:03.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd025d97700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd020101080 0x7fd02019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:03.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd025d97700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02019de60 con 0x7fd0200ff4f0 2023-09-12T16:02:03.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd025d97700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02019dfd0 con 0x7fd0201006e0 2023-09-12T16:02:03.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd025d97700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02019e190 con 0x7fd020101080 2023-09-12T16:02:03.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd01f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 0x7fd020069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:03.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd01f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 0x7fd020069360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53228/0 (socket says 172.21.15.130:53228) 2023-09-12T16:02:03.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd01f7fe700 1 -- 172.21.15.130:0/791309697 learned_addr learned my addr 172.21.15.130:0/791309697 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:03.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd01ffff700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd020101080 0x7fd02019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:03.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.149+0000 7fd01effd700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0201006e0 0x7fd02019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:03.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd01f7fe700 1 -- 172.21.15.130:0/791309697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd020101080 msgr2=0x7fd02019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:03.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd01f7fe700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd020101080 0x7fd02019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd01f7fe700 1 -- 172.21.15.130:0/791309697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0201006e0 msgr2=0x7fd02019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:03.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd01f7fe700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0201006e0 0x7fd02019b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:03.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd01f7fe700 1 -- 172.21.15.130:0/791309697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0100095f0 con 0x7fd0200ff4f0 2023-09-12T16:02:03.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd01f7fe700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 0x7fd020069360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd010009990 tx=0x7fd01000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:03.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd01cff9700 1 -- 172.21.15.130:0/791309697 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01001d070 con 0x7fd0200ff4f0 2023-09-12T16:02:03.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd01cff9700 1 -- 172.21.15.130:0/791309697 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd010005030 con 0x7fd0200ff4f0 2023-09-12T16:02:03.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02019e410 con 0x7fd0200ff4f0 2023-09-12T16:02:03.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd01cff9700 1 -- 172.21.15.130:0/791309697 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010004a70 con 0x7fd0200ff4f0 2023-09-12T16:02:03.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.150+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02019e830 con 0x7fd0200ff4f0 2023-09-12T16:02:03.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.151+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd020106470 con 0x7fd0200ff4f0 2023-09-12T16:02:03.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.152+0000 7fd01cff9700 1 -- 172.21.15.130:0/791309697 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fd010005410 con 0x7fd0200ff4f0 2023-09-12T16:02:03.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.153+0000 7fd01cff9700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0080a0640 0x7fd0080a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:03.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.153+0000 7fd01effd700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0080a0640 0x7fd0080a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:03.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.153+0000 7fd01cff9700 1 -- 172.21.15.130:0/791309697 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fd0100ccaa0 con 0x7fd0200ff4f0 2023-09-12T16:02:03.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.153+0000 7fd01effd700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0080a0640 0x7fd0080a2b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd014009c60 tx=0x7fd014009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:03.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.156+0000 7fd01cff9700 1 -- 172.21.15.130:0/791309697 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd0100968f0 con 0x7fd0200ff4f0 2023-09-12T16:02:03.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:03.355+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fd02010ad60 con 0x7fd0200ff4f0 2023-09-12T16:02:03.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:02 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2147254465' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-12T16:02:03.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:02 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:02 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:02 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:02 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:02 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:02 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:02 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2147254465' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-12T16:02:03.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:02 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:02 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:02 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:02 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:02 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:02 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:02 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2147254465' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-12T16:02:03.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:02 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:02 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:02 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:02 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:02 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:03.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:02 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:04.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.011+0000 7fd01cff9700 1 -- 172.21.15.130:0/791309697 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v79) v1 ==== 171+0+0 (secure 0 0 0) 0x7fd010099f10 con 0x7fd0200ff4f0 2023-09-12T16:02:04.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0080a0640 msgr2=0x7fd0080a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:04.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0080a0640 0x7fd0080a2b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd014009c60 tx=0x7fd014009260 comp rx=0 tx=0).stop 2023-09-12T16:02:04.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 msgr2=0x7fd020069360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:04.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 0x7fd020069360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd010009990 tx=0x7fd01000f690 comp rx=0 tx=0).stop 2023-09-12T16:02:04.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 shutdown_connections 2023-09-12T16:02:04.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0080a0640 0x7fd0080a2b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:04.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd020101080 0x7fd02019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:04.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0200ff4f0 0x7fd020069360 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:04.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 --2- 172.21.15.130:0/791309697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0201006e0 0x7fd02019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:04.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.013+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 >> 172.21.15.130:0/791309697 conn(0x7fd0200faa60 msgr2=0x7fd0200fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:04.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.014+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 shutdown_connections 2023-09-12T16:02:04.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.014+0000 7fd025d97700 1 -- 172.21.15.130:0/791309697 wait complete. 2023-09-12T16:02:04.015 INFO:teuthology.orchestra.run.smithi130.stderr:pool 'cephfs_data' created 2023-09-12T16:02:04.035 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-12T16:02:04.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.163+0000 7fe0d58d0700 1 -- 172.21.15.130:0/857986205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d0100ab0 msgr2=0x7fe0d0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:04.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.163+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/857986205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d0100ab0 0x7fe0d0100ed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00098f0 tx=0x7fe0c0009c00 comp rx=0 tx=0).stop 2023-09-12T16:02:04.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.163+0000 7fe0d58d0700 1 -- 172.21.15.130:0/857986205 shutdown_connections 2023-09-12T16:02:04.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.163+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/857986205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d0100180 0x7fe0d0107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:04.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.163+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/857986205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d0100ab0 0x7fe0d0100ed0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:04.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.163+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/857986205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0d00ff7e0 0x7fe0d00ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:04.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.163+0000 7fe0d58d0700 1 -- 172.21.15.130:0/857986205 >> 172.21.15.130:0/857986205 conn(0x7fe0d00fa1f0 msgr2=0x7fe0d00fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:04.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.163+0000 7fe0d58d0700 1 -- 172.21.15.130:0/857986205 shutdown_connections 2023-09-12T16:02:04.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.163+0000 7fe0d58d0700 1 -- 172.21.15.130:0/857986205 wait complete. 2023-09-12T16:02:04.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.164+0000 7fe0d58d0700 1 Processor -- start 2023-09-12T16:02:04.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.164+0000 7fe0d58d0700 1 -- start start 2023-09-12T16:02:04.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.164+0000 7fe0d58d0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d00ff7e0 0x7fe0d0195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:04.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.164+0000 7fe0d58d0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d0100180 0x7fe0d01957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:04.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.164+0000 7fe0d58d0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0d0100ab0 0x7fe0d019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:04.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.164+0000 7fe0d58d0700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d019cde0 con 0x7fe0d0100180 2023-09-12T16:02:04.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.164+0000 7fe0d58d0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d019cf20 con 0x7fe0d0100ab0 2023-09-12T16:02:04.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.164+0000 7fe0d58d0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d019d0b0 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.164+0000 7fe0ce7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d0100180 0x7fe0d01957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:04.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ce7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d0100180 0x7fe0d01957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53248/0 (socket says 172.21.15.130:53248) 2023-09-12T16:02:04.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ce7fc700 1 -- 172.21.15.130:0/96743375 learned_addr learned my addr 172.21.15.130:0/96743375 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:04.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ceffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d00ff7e0 0x7fe0d0195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:04.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0cf7fe700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0d0100ab0 0x7fe0d019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:04.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ceffd700 1 -- 172.21.15.130:0/96743375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0d0100ab0 msgr2=0x7fe0d019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:04.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ceffd700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0d0100ab0 0x7fe0d019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:04.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ceffd700 1 -- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d0100180 msgr2=0x7fe0d01957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:04.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ceffd700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d0100180 0x7fe0d01957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:04.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ceffd700 1 -- 172.21.15.130:0/96743375 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c8009d00 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ce7fc700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d0100180 0x7fe0d01957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:02:04.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0ceffd700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d00ff7e0 0x7fe0d0195260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0000c00 tx=0x7fe0c000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:04.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0d48ce700 1 -- 172.21.15.130:0/96743375 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c001c070 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0d48ce700 1 -- 172.21.15.130:0/96743375 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c000f7d0 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0d48ce700 1 -- 172.21.15.130:0/96743375 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0003990 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c0009520 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.165+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d00fe910 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.166+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d010ad10 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.167+0000 7fe0d48ce700 1 -- 172.21.15.130:0/96743375 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fe0c0020070 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.168+0000 7fe0d48ce700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe0b80a0640 0x7fe0b80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:04.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.168+0000 7fe0ce7fc700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe0b80a0640 0x7fe0b80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:04.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.168+0000 7fe0d48ce700 1 -- 172.21.15.130:0/96743375 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fe0c00cc6b0 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.168+0000 7fe0ce7fc700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe0b80a0640 0x7fe0b80a2b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400e5f0 tx=0x7fe0c4010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:04.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.171+0000 7fe0d48ce700 1 -- 172.21.15.130:0/96743375 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe0c0096420 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:04.420+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe0d00feb30 con 0x7fe0d00ff7e0 2023-09-12T16:02:04.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:04 smithi130 ceph-mon[98670]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:04.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:04 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2147254465' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-12T16:02:04.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:04 smithi130 ceph-mon[98670]: osdmap e78: 12 total, 12 up, 12 in 2023-09-12T16:02:04.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:02:04.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:02:04.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:04.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:04 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/791309697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-12T16:02:04.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:04 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:02:04.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:04 smithi159 ceph-mon[102522]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:04.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:04 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2147254465' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-12T16:02:04.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:04 smithi159 ceph-mon[102522]: osdmap e78: 12 total, 12 up, 12 in 2023-09-12T16:02:04.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:04 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:02:04.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:04 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:02:04.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:04 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:04.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:04 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/791309697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-12T16:02:04.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:04 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:02:04.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:04 smithi203 ceph-mon[103431]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:04.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:04 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2147254465' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-12T16:02:04.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:04 smithi203 ceph-mon[103431]: osdmap e78: 12 total, 12 up, 12 in 2023-09-12T16:02:04.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:04 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:02:04.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:04 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:02:04.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:04 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:04.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:04 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/791309697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-12T16:02:04.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:04 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:02:05.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.029+0000 7fe0d48ce700 1 -- 172.21.15.130:0/96743375 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe0c0099a40 con 0x7fe0d00ff7e0 2023-09-12T16:02:05.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe0b80a0640 msgr2=0x7fe0b80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:05.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe0b80a0640 0x7fe0b80a2b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400e5f0 tx=0x7fe0c4010040 comp rx=0 tx=0).stop 2023-09-12T16:02:05.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d00ff7e0 msgr2=0x7fe0d0195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:05.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d00ff7e0 0x7fe0d0195260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0000c00 tx=0x7fe0c000b790 comp rx=0 tx=0).stop 2023-09-12T16:02:05.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 shutdown_connections 2023-09-12T16:02:05.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe0b80a0640 0x7fe0b80a2b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0d0100ab0 0x7fe0d019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d00ff7e0 0x7fe0d0195260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 --2- 172.21.15.130:0/96743375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d0100180 0x7fe0d01957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 >> 172.21.15.130:0/96743375 conn(0x7fe0d00fa1f0 msgr2=0x7fe0d00fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:05.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 shutdown_connections 2023-09-12T16:02:05.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.036+0000 7fe0d58d0700 1 -- 172.21.15.130:0/96743375 wait complete. 2023-09-12T16:02:05.042 INFO:teuthology.orchestra.run.smithi130.stderr:new fs with metadata pool 2 and data pool 3 2023-09-12T16:02:05.055 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-09-12T16:02:05.055 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-09-12T16:02:05.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.190+0000 7f65c6aa0700 1 -- 172.21.15.130:0/9716481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65c0100750 msgr2=0x7f65c0100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:05.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.190+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/9716481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65c0100750 0x7f65c0100bb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0099c0 tx=0x7f65ac009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.190+0000 7f65c6aa0700 1 -- 172.21.15.130:0/9716481 shutdown_connections 2023-09-12T16:02:05.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.190+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/9716481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c01010f0 0x7f65c01079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.190+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/9716481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c00ff560 0x7f65c00ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.190+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/9716481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65c0100750 0x7f65c0100bb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.190+0000 7f65c6aa0700 1 -- 172.21.15.130:0/9716481 >> 172.21.15.130:0/9716481 conn(0x7f65c00faad0 msgr2=0x7f65c00fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:05.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.190+0000 7f65c6aa0700 1 -- 172.21.15.130:0/9716481 shutdown_connections 2023-09-12T16:02:05.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.190+0000 7f65c6aa0700 1 -- 172.21.15.130:0/9716481 wait complete. 2023-09-12T16:02:05.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c6aa0700 1 Processor -- start 2023-09-12T16:02:05.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c6aa0700 1 -- start start 2023-09-12T16:02:05.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c6aa0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c00ff560 0x7f65c00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:05.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c6aa0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65c0100750 0x7f65c019af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:05.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c6aa0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c01010f0 0x7f65c019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:05.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c6aa0700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c019dda0 con 0x7f65c0100750 2023-09-12T16:02:05.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c6aa0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c019df10 con 0x7f65c00ff560 2023-09-12T16:02:05.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c6aa0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c0194d50 con 0x7f65c01010f0 2023-09-12T16:02:05.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c483c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c00ff560 0x7f65c00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:05.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c483c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c00ff560 0x7f65c00694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:36522/0 (socket says 172.21.15.130:36522) 2023-09-12T16:02:05.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c483c700 1 -- 172.21.15.130:0/2626318796 learned_addr learned my addr 172.21.15.130:0/2626318796 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:05.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c503d700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c01010f0 0x7f65c019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:05.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65bffff700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65c0100750 0x7f65c019af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:05.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c483c700 1 -- 172.21.15.130:0/2626318796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c01010f0 msgr2=0x7f65c019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:05.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c483c700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c01010f0 0x7f65c019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c483c700 1 -- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65c0100750 msgr2=0x7f65c019af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:05.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c483c700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65c0100750 0x7f65c019af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:05.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65c483c700 1 -- 172.21.15.130:0/2626318796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac0095f0 con 0x7f65c00ff560 2023-09-12T16:02:05.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.191+0000 7f65bffff700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65c0100750 0x7f65c019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:02:05.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.192+0000 7f65c483c700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c00ff560 0x7f65c00694a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f65b4014300 tx=0x7f65b4014610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:05.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.192+0000 7f65bdffb700 1 -- 172.21.15.130:0/2626318796 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b4017070 con 0x7f65c00ff560 2023-09-12T16:02:05.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.192+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c0195030 con 0x7f65c00ff560 2023-09-12T16:02:05.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.192+0000 7f65bdffb700 1 -- 172.21.15.130:0/2626318796 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65b4014dc0 con 0x7f65c00ff560 2023-09-12T16:02:05.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.192+0000 7f65bdffb700 1 -- 172.21.15.130:0/2626318796 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40128e0 con 0x7f65c00ff560 2023-09-12T16:02:05.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.192+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c0195510 con 0x7f65c00ff560 2023-09-12T16:02:05.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.193+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c010ad30 con 0x7f65c00ff560 2023-09-12T16:02:05.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.194+0000 7f65bdffb700 1 -- 172.21.15.130:0/2626318796 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f65b4010a40 con 0x7f65c00ff560 2023-09-12T16:02:05.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.194+0000 7f65bdffb700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65b00a0640 0x7f65b00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:05.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.195+0000 7f65bffff700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65b00a0640 0x7f65b00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:05.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.195+0000 7f65bdffb700 1 -- 172.21.15.130:0/2626318796 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f65b40d36c0 con 0x7f65c00ff560 2023-09-12T16:02:05.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.195+0000 7f65bffff700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65b00a0640 0x7f65b00a2b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00b580 tx=0x7f65ac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:05.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.198+0000 7f65bdffb700 1 -- 172.21.15.130:0/2626318796 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f65b409d340 con 0x7f65c00ff560 2023-09-12T16:02:05.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:05.405+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f65c010af10 con 0x7f65c00ff560 2023-09-12T16:02:05.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:05 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/791309697' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-12T16:02:05.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:05 smithi130 ceph-mon[98670]: osdmap e79: 12 total, 12 up, 12 in 2023-09-12T16:02:05.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:05 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/96743375' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-12T16:02:05.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:05 smithi130 ceph-mon[98670]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-12T16:02:05.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:05 smithi130 ceph-mon[98670]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-12T16:02:05.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:05 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/791309697' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-12T16:02:05.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:05 smithi159 ceph-mon[102522]: osdmap e79: 12 total, 12 up, 12 in 2023-09-12T16:02:05.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:05 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/96743375' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-12T16:02:05.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:05 smithi159 ceph-mon[102522]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-12T16:02:05.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:05 smithi159 ceph-mon[102522]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-12T16:02:05.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:05 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/791309697' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-12T16:02:05.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:05 smithi203 ceph-mon[103431]: osdmap e79: 12 total, 12 up, 12 in 2023-09-12T16:02:05.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:05 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/96743375' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-12T16:02:05.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:05 smithi203 ceph-mon[103431]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-12T16:02:05.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:05 smithi203 ceph-mon[103431]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-12T16:02:06.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.027+0000 7f65bdffb700 1 -- 172.21.15.130:0/2626318796 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) v1 ==== 157+0+0 (secure 0 0 0) 0x7f65b40a0960 con 0x7f65c00ff560 2023-09-12T16:02:06.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.030+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65b00a0640 msgr2=0x7f65b00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:06.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.030+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65b00a0640 0x7f65b00a2b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00b580 tx=0x7f65ac000bc0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.030+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c00ff560 msgr2=0x7f65c00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:06.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.030+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c00ff560 0x7f65c00694a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f65b4014300 tx=0x7f65b4014610 comp rx=0 tx=0).stop 2023-09-12T16:02:06.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.030+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 shutdown_connections 2023-09-12T16:02:06.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.030+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65b00a0640 0x7f65b00a2b00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.030+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c01010f0 0x7f65c019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.031+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c00ff560 0x7f65c00694a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.031+0000 7f65c6aa0700 1 --2- 172.21.15.130:0/2626318796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65c0100750 0x7f65c019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.031+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 >> 172.21.15.130:0/2626318796 conn(0x7f65c00faad0 msgr2=0x7f65c0103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:06.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.031+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 shutdown_connections 2023-09-12T16:02:06.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.031+0000 7f65c6aa0700 1 -- 172.21.15.130:0/2626318796 wait complete. 2023-09-12T16:02:06.047 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-09-12T16:02:06.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.180+0000 7f9d1fa93700 1 -- 172.21.15.130:0/2960715081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d181010a0 msgr2=0x7f9d18107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:06.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.180+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/2960715081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d181010a0 0x7f9d18107950 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d080099c0 tx=0x7f9d08009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.180+0000 7f9d1fa93700 1 -- 172.21.15.130:0/2960715081 shutdown_connections 2023-09-12T16:02:06.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.180+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/2960715081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d181010a0 0x7f9d18107950 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.180+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/2960715081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d180ff510 0x7f9d180ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.180+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/2960715081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d18100700 0x7f9d18100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.180+0000 7f9d1fa93700 1 -- 172.21.15.130:0/2960715081 >> 172.21.15.130:0/2960715081 conn(0x7f9d180faa40 msgr2=0x7f9d180fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:06.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.180+0000 7f9d1fa93700 1 -- 172.21.15.130:0/2960715081 shutdown_connections 2023-09-12T16:02:06.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.180+0000 7f9d1fa93700 1 -- 172.21.15.130:0/2960715081 wait complete. 2023-09-12T16:02:06.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1fa93700 1 Processor -- start 2023-09-12T16:02:06.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1fa93700 1 -- start start 2023-09-12T16:02:06.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1fa93700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d180ff510 0x7f9d18069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:06.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1fa93700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d18100700 0x7f9d1819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:06.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1fa93700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d181010a0 0x7f9d1819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:06.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1fa93700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1819dca0 con 0x7f9d180ff510 2023-09-12T16:02:06.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1fa93700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1819de10 con 0x7f9d18100700 2023-09-12T16:02:06.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1fa93700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1819dfd0 con 0x7f9d181010a0 2023-09-12T16:02:06.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1d82f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d180ff510 0x7f9d18069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:06.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1d82f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d180ff510 0x7f9d18069340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53278/0 (socket says 172.21.15.130:53278) 2023-09-12T16:02:06.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1d82f700 1 -- 172.21.15.130:0/1469474981 learned_addr learned my addr 172.21.15.130:0/1469474981 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:06.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1d02e700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d18100700 0x7f9d1819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:06.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.181+0000 7f9d1e030700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d181010a0 0x7f9d1819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:06.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d1d82f700 1 -- 172.21.15.130:0/1469474981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d181010a0 msgr2=0x7f9d1819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:06.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d1d82f700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d181010a0 0x7f9d1819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.188 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d1d82f700 1 -- 172.21.15.130:0/1469474981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d18100700 msgr2=0x7f9d1819ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:06.188 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d1d82f700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d18100700 0x7f9d1819ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:06.188 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d1d82f700 1 -- 172.21.15.130:0/1469474981 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d080095f0 con 0x7f9d180ff510 2023-09-12T16:02:06.188 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d1d82f700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d180ff510 0x7f9d18069340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000efc0 tx=0x7f9d1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:06.188 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d16ffd700 1 -- 172.21.15.130:0/1469474981 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10018070 con 0x7f9d180ff510 2023-09-12T16:02:06.188 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d16ffd700 1 -- 172.21.15.130:0/1469474981 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d1000f040 con 0x7f9d180ff510 2023-09-12T16:02:06.189 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1819e250 con 0x7f9d180ff510 2023-09-12T16:02:06.189 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d16ffd700 1 -- 172.21.15.130:0/1469474981 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10004280 con 0x7f9d180ff510 2023-09-12T16:02:06.189 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.182+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1819e760 con 0x7f9d180ff510 2023-09-12T16:02:06.189 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.183+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d18106970 con 0x7f9d180ff510 2023-09-12T16:02:06.189 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.184+0000 7f9d16ffd700 1 -- 172.21.15.130:0/1469474981 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f9d10003a00 con 0x7f9d180ff510 2023-09-12T16:02:06.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.184+0000 7f9d16ffd700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9d000a06e0 0x7f9d000a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:06.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.185+0000 7f9d1d02e700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9d000a06e0 0x7f9d000a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:06.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.185+0000 7f9d16ffd700 1 -- 172.21.15.130:0/1469474981 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f9d100cc430 con 0x7f9d180ff510 2023-09-12T16:02:06.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.185+0000 7f9d1d02e700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9d000a06e0 0x7f9d000a2ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d180690b0 tx=0x7f9d0c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:06.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.188+0000 7f9d16ffd700 1 -- 172.21.15.130:0/1469474981 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9d10095fb0 con 0x7f9d180ff510 2023-09-12T16:02:06.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:06.396+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9d1810ad20 con 0x7f9d180ff510 2023-09-12T16:02:06.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:06 smithi130 ceph-mon[98670]: pgmap v256: 97 pgs: 9 active+clean, 23 creating+peering, 65 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:06.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:06 smithi130 ceph-mon[98670]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-12T16:02:06.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:06 smithi130 ceph-mon[98670]: osdmap e80: 12 total, 12 up, 12 in 2023-09-12T16:02:06.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:06 smithi130 ceph-mon[98670]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-12T16:02:06.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:06 smithi130 ceph-mon[98670]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-12T16:02:06.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:06 smithi130 ceph-mon[98670]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-12T16:02:06.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:06 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2626318796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-12T16:02:06.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:06 smithi130 ceph-mon[98670]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-12T16:02:06.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:06 smithi159 ceph-mon[102522]: pgmap v256: 97 pgs: 9 active+clean, 23 creating+peering, 65 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:06.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:06 smithi159 ceph-mon[102522]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-12T16:02:06.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:06 smithi159 ceph-mon[102522]: osdmap e80: 12 total, 12 up, 12 in 2023-09-12T16:02:06.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:06 smithi159 ceph-mon[102522]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-12T16:02:06.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:06 smithi159 ceph-mon[102522]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-12T16:02:06.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:06 smithi159 ceph-mon[102522]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-12T16:02:06.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:06 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2626318796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-12T16:02:06.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:06 smithi159 ceph-mon[102522]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-12T16:02:06.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:06 smithi203 ceph-mon[103431]: pgmap v256: 97 pgs: 9 active+clean, 23 creating+peering, 65 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:02:06.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:06 smithi203 ceph-mon[103431]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-12T16:02:06.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:06 smithi203 ceph-mon[103431]: osdmap e80: 12 total, 12 up, 12 in 2023-09-12T16:02:06.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:06 smithi203 ceph-mon[103431]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-12T16:02:06.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:06 smithi203 ceph-mon[103431]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-12T16:02:06.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:06 smithi203 ceph-mon[103431]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-12T16:02:06.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:06 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2626318796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-12T16:02:06.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:06 smithi203 ceph-mon[103431]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-12T16:02:07.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-12T16:02:07.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: osdmap e81: 12 total, 12 up, 12 in 2023-09-12T16:02:07.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] up:active 2023-09-12T16:02:07.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:07.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:07.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:02:07.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:02:07.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:07 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1469474981' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-12T16:02:07.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-12T16:02:07.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: osdmap e81: 12 total, 12 up, 12 in 2023-09-12T16:02:07.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] up:active 2023-09-12T16:02:07.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:07.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:07.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:02:07.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:02:07.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:07 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1469474981' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-12T16:02:07.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-12T16:02:07.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: osdmap e81: 12 total, 12 up, 12 in 2023-09-12T16:02:07.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] up:active 2023-09-12T16:02:07.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:07.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:07.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:02:07.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:02:07.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:02:07.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:07 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1469474981' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-12T16:02:08.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.259+0000 7f9d16ffd700 1 -- 172.21.15.130:0/1469474981 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) v1 ==== 228+0+0 (secure 0 0 0) 0x7f9d100995d0 con 0x7f9d180ff510 2023-09-12T16:02:08.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9d000a06e0 msgr2=0x7f9d000a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:08.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9d000a06e0 0x7f9d000a2ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d180690b0 tx=0x7f9d0c00b410 comp rx=0 tx=0).stop 2023-09-12T16:02:08.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d180ff510 msgr2=0x7f9d18069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:08.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d180ff510 0x7f9d18069340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000efc0 tx=0x7f9d1000c3a0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 shutdown_connections 2023-09-12T16:02:08.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9d000a06e0 0x7f9d000a2ba0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d181010a0 0x7f9d1819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d180ff510 0x7f9d18069340 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000efc0 tx=0x7f9d1000c3a0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 --2- 172.21.15.130:0/1469474981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d18100700 0x7f9d1819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.263+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 >> 172.21.15.130:0/1469474981 conn(0x7f9d180faa40 msgr2=0x7f9d18103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:08.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.264+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 shutdown_connections 2023-09-12T16:02:08.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.265+0000 7f9d1fa93700 1 -- 172.21.15.130:0/1469474981 wait complete. 2023-09-12T16:02:08.271 INFO:teuthology.orchestra.run.smithi130.stderr:pool 'cephfs_data_ec' created 2023-09-12T16:02:08.285 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-09-12T16:02:08.311 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:08 smithi130 ceph-mon[98670]: pgmap v259: 97 pgs: 33 active+clean, 23 creating+peering, 41 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2023-09-12T16:02:08.311 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:08 smithi130 ceph-mon[98670]: osdmap e82: 12 total, 12 up, 12 in 2023-09-12T16:02:08.311 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:08 smithi130 ceph-mon[98670]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-12T16:02:08.311 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:08 smithi130 ceph-mon[98670]: Cluster is now healthy 2023-09-12T16:02:08.417 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 -- 172.21.15.130:0/3819147927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 msgr2=0x7f1de0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:08.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 --2- 172.21.15.130:0/3819147927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 0x7f1de0100b40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00099c0 tx=0x7f1dd0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 -- 172.21.15.130:0/3819147927 shutdown_connections 2023-09-12T16:02:08.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 --2- 172.21.15.130:0/3819147927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1de0101080 0x7f1de0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 --2- 172.21.15.130:0/3819147927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de00ff4f0 0x7f1de00ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 --2- 172.21.15.130:0/3819147927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 0x7f1de0100b40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 -- 172.21.15.130:0/3819147927 >> 172.21.15.130:0/3819147927 conn(0x7f1de00faa60 msgr2=0x7f1de00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:08.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 -- 172.21.15.130:0/3819147927 shutdown_connections 2023-09-12T16:02:08.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 -- 172.21.15.130:0/3819147927 wait complete. 2023-09-12T16:02:08.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 Processor -- start 2023-09-12T16:02:08.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 -- start start 2023-09-12T16:02:08.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de00ff4f0 0x7f1de0194120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:08.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 0x7f1de0194660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:08.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1de0101080 0x7f1de019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:08.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019bca0 con 0x7f1de01006e0 2023-09-12T16:02:08.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019bde0 con 0x7f1de00ff4f0 2023-09-12T16:02:08.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.416+0000 7f1de824b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019bf70 con 0x7f1de0101080 2023-09-12T16:02:08.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de57e6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 0x7f1de0194660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:08.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de57e6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 0x7f1de0194660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53298/0 (socket says 172.21.15.130:53298) 2023-09-12T16:02:08.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de57e6700 1 -- 172.21.15.130:0/1865963473 learned_addr learned my addr 172.21.15.130:0/1865963473 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:08.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de67e8700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1de0101080 0x7f1de019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:08.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de57e6700 1 -- 172.21.15.130:0/1865963473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1de0101080 msgr2=0x7f1de019b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:08.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de5fe7700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de00ff4f0 0x7f1de0194120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:08.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de57e6700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1de0101080 0x7f1de019b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de57e6700 1 -- 172.21.15.130:0/1865963473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de00ff4f0 msgr2=0x7f1de0194120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:08.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de57e6700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de00ff4f0 0x7f1de0194120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:08.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de57e6700 1 -- 172.21.15.130:0/1865963473 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd00095f0 con 0x7f1de01006e0 2023-09-12T16:02:08.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de57e6700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 0x7f1de0194660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009fd0 tx=0x7f1dd000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:08.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1dd6ffd700 1 -- 172.21.15.130:0/1865963473 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd001d070 con 0x7f1de01006e0 2023-09-12T16:02:08.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1dd6ffd700 1 -- 172.21.15.130:0/1865963473 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dd00054f0 con 0x7f1de01006e0 2023-09-12T16:02:08.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1dd6ffd700 1 -- 172.21.15.130:0/1865963473 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0004c70 con 0x7f1de01006e0 2023-09-12T16:02:08.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de019c1f0 con 0x7f1de01006e0 2023-09-12T16:02:08.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.417+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de019c5c0 con 0x7f1de01006e0 2023-09-12T16:02:08.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.418+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de0106b30 con 0x7f1de01006e0 2023-09-12T16:02:08.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.420+0000 7f1dd6ffd700 1 -- 172.21.15.130:0/1865963473 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f1dd000fc30 con 0x7f1de01006e0 2023-09-12T16:02:08.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.420+0000 7f1dd6ffd700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dcc0a0640 0x7f1dcc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:08.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.420+0000 7f1dd6ffd700 1 -- 172.21.15.130:0/1865963473 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f1dd00cd0b0 con 0x7f1de01006e0 2023-09-12T16:02:08.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.420+0000 7f1de5fe7700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dcc0a0640 0x7f1dcc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:08.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.421+0000 7f1de5fe7700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dcc0a0640 0x7f1dcc0a2b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1de0100540 tx=0x7f1ddc011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:08.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.424+0000 7f1dd6ffd700 1 -- 172.21.15.130:0/1865963473 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1dd0096aa0 con 0x7f1de01006e0 2023-09-12T16:02:08.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:08 smithi159 ceph-mon[102522]: pgmap v259: 97 pgs: 33 active+clean, 23 creating+peering, 41 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2023-09-12T16:02:08.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:08 smithi159 ceph-mon[102522]: osdmap e82: 12 total, 12 up, 12 in 2023-09-12T16:02:08.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:08 smithi159 ceph-mon[102522]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-12T16:02:08.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:08 smithi159 ceph-mon[102522]: Cluster is now healthy 2023-09-12T16:02:08.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:08 smithi203 ceph-mon[103431]: pgmap v259: 97 pgs: 33 active+clean, 23 creating+peering, 41 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2023-09-12T16:02:08.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:08 smithi203 ceph-mon[103431]: osdmap e82: 12 total, 12 up, 12 in 2023-09-12T16:02:08.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:08 smithi203 ceph-mon[103431]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-12T16:02:08.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:08 smithi203 ceph-mon[103431]: Cluster is now healthy 2023-09-12T16:02:08.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:08.615+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f1de00008d0 con 0x7f1de01006e0 2023-09-12T16:02:09.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.267+0000 7f1dd6ffd700 1 -- 172.21.15.130:0/1865963473 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1dd009a0c0 con 0x7f1de01006e0 2023-09-12T16:02:09.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dcc0a0640 msgr2=0x7f1dcc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:09.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dcc0a0640 0x7f1dcc0a2b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1de0100540 tx=0x7f1ddc011040 comp rx=0 tx=0).stop 2023-09-12T16:02:09.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 msgr2=0x7f1de0194660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:09.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 0x7f1de0194660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009fd0 tx=0x7f1dd000f740 comp rx=0 tx=0).stop 2023-09-12T16:02:09.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 shutdown_connections 2023-09-12T16:02:09.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dcc0a0640 0x7f1dcc0a2b00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1de0101080 0x7f1de019b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de00ff4f0 0x7f1de0194120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 --2- 172.21.15.130:0/1865963473 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de01006e0 0x7f1de0194660 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 >> 172.21.15.130:0/1865963473 conn(0x7f1de00faa60 msgr2=0x7f1de0103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:09.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 shutdown_connections 2023-09-12T16:02:09.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.270+0000 7f1de824b700 1 -- 172.21.15.130:0/1865963473 wait complete. 2023-09-12T16:02:09.272 INFO:teuthology.orchestra.run.smithi130.stderr:set pool 4 allow_ec_overwrites to true 2023-09-12T16:02:09.297 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-09-12T16:02:09.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 -- 172.21.15.130:0/2017863974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94681021b0 msgr2=0x7f9468108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:09.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 --2- 172.21.15.130:0/2017863974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94681021b0 0x7f9468108a60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f946000b0d0 tx=0x7f946000b3e0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 -- 172.21.15.130:0/2017863974 shutdown_connections 2023-09-12T16:02:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 --2- 172.21.15.130:0/2017863974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94681021b0 0x7f9468108a60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 --2- 172.21.15.130:0/2017863974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9468100620 0x7f9468100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 --2- 172.21.15.130:0/2017863974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9468101810 0x7f9468101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 -- 172.21.15.130:0/2017863974 >> 172.21.15.130:0/2017863974 conn(0x7f94680fbbb0 msgr2=0x7f94680fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 -- 172.21.15.130:0/2017863974 shutdown_connections 2023-09-12T16:02:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 -- 172.21.15.130:0/2017863974 wait complete. 2023-09-12T16:02:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 Processor -- start 2023-09-12T16:02:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 -- start start 2023-09-12T16:02:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9468100620 0x7f94681985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9468101810 0x7f9468198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94681021b0 0x7f946819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94681a0120 con 0x7f9468101810 2023-09-12T16:02:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94681a0260 con 0x7f94681021b0 2023-09-12T16:02:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.468+0000 7f946f88e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94681a03f0 con 0x7f9468100620 2023-09-12T16:02:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946ce29700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9468101810 0x7f9468198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946ce29700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9468101810 0x7f9468198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53320/0 (socket says 172.21.15.130:53320) 2023-09-12T16:02:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946ce29700 1 -- 172.21.15.130:0/3525192433 learned_addr learned my addr 172.21.15.130:0/3525192433 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946d62a700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9468100620 0x7f94681985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946de2b700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94681021b0 0x7f946819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946ce29700 1 -- 172.21.15.130:0/3525192433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9468100620 msgr2=0x7f94681985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946ce29700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9468100620 0x7f94681985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946ce29700 1 -- 172.21.15.130:0/3525192433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94681021b0 msgr2=0x7f946819fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946ce29700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94681021b0 0x7f946819fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946ce29700 1 -- 172.21.15.130:0/3525192433 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9460009d00 con 0x7f9468101810 2023-09-12T16:02:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946ce29700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9468101810 0x7f9468198ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f946400d950 tx=0x7f946400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f945e7fc700 1 -- 172.21.15.130:0/3525192433 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946400f890 con 0x7f9468101810 2023-09-12T16:02:09.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f945e7fc700 1 -- 172.21.15.130:0/3525192433 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9464010460 con 0x7f9468101810 2023-09-12T16:02:09.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94681a06d0 con 0x7f9468101810 2023-09-12T16:02:09.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f945e7fc700 1 -- 172.21.15.130:0/3525192433 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946400e610 con 0x7f9468101810 2023-09-12T16:02:09.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.469+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94681a0be0 con 0x7f9468101810 2023-09-12T16:02:09.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.470+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946810be40 con 0x7f9468101810 2023-09-12T16:02:09.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.471+0000 7f945e7fc700 1 -- 172.21.15.130:0/3525192433 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f94640108d0 con 0x7f9468101810 2023-09-12T16:02:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.472+0000 7f945e7fc700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94540a0640 0x7f94540a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.472+0000 7f946d62a700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94540a0640 0x7f94540a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.472+0000 7f945e7fc700 1 -- 172.21.15.130:0/3525192433 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f94640cbac0 con 0x7f9468101810 2023-09-12T16:02:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.472+0000 7f946d62a700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94540a0640 0x7f94540a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9468101670 tx=0x7f945800a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.475+0000 7f945e7fc700 1 -- 172.21.15.130:0/3525192433 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9464095420 con 0x7f9468101810 2023-09-12T16:02:09.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:09.714+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f946810c000 con 0x7f9468101810 2023-09-12T16:02:09.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:09 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1469474981' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-12T16:02:09.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:09 smithi130 ceph-mon[98670]: osdmap e83: 12 total, 12 up, 12 in 2023-09-12T16:02:09.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:09 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1865963473' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-12T16:02:09.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:09 smithi130 ceph-mon[98670]: pgmap v262: 129 pgs: 68 active+clean, 23 creating+peering, 38 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-09-12T16:02:09.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:09 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1469474981' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-12T16:02:09.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:09 smithi159 ceph-mon[102522]: osdmap e83: 12 total, 12 up, 12 in 2023-09-12T16:02:09.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:09 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1865963473' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-12T16:02:09.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:09 smithi159 ceph-mon[102522]: pgmap v262: 129 pgs: 68 active+clean, 23 creating+peering, 38 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-09-12T16:02:09.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:09 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1469474981' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-12T16:02:09.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:09 smithi203 ceph-mon[103431]: osdmap e83: 12 total, 12 up, 12 in 2023-09-12T16:02:09.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:09 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1865963473' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-12T16:02:09.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:09 smithi203 ceph-mon[103431]: pgmap v262: 129 pgs: 68 active+clean, 23 creating+peering, 38 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-09-12T16:02:10.262 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:02:09 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-b[139180]: 2023-09-12T16:02:09.990+0000 7f005ca0a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-12T16:02:10.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.271+0000 7f945e7fc700 1 -- 172.21.15.130:0/3525192433 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f94640160b0 con 0x7f9468101810 2023-09-12T16:02:10.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94540a0640 msgr2=0x7f94540a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94540a0640 0x7f94540a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9468101670 tx=0x7f945800a3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9468101810 msgr2=0x7f9468198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9468101810 0x7f9468198ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f946400d950 tx=0x7f946400dc60 comp rx=0 tx=0).stop 2023-09-12T16:02:10.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 shutdown_connections 2023-09-12T16:02:10.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94540a0640 0x7f94540a2b00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94681021b0 0x7f946819fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9468100620 0x7f94681985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 --2- 172.21.15.130:0/3525192433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9468101810 0x7f9468198ae0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.275+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 >> 172.21.15.130:0/3525192433 conn(0x7f94680fbbb0 msgr2=0x7f9468106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:10.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.276+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 shutdown_connections 2023-09-12T16:02:10.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.276+0000 7f946f88e700 1 -- 172.21.15.130:0/3525192433 wait complete. 2023-09-12T16:02:10.277 INFO:teuthology.orchestra.run.smithi130.stderr:added data pool 4 to fsmap 2023-09-12T16:02:10.289 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:02:10.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.412+0000 7f40c84d4700 1 -- 172.21.15.130:0/693835450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 msgr2=0x7f40c0107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.412+0000 7f40c84d4700 1 --2- 172.21.15.130:0/693835450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 0x7f40c0107960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40b800b0d0 tx=0x7f40b800b3e0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.412+0000 7f40c84d4700 1 -- 172.21.15.130:0/693835450 shutdown_connections 2023-09-12T16:02:10.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.412+0000 7f40c84d4700 1 --2- 172.21.15.130:0/693835450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 0x7f40c0107960 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.412+0000 7f40c84d4700 1 --2- 172.21.15.130:0/693835450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c00ff520 0x7f40c00ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.412+0000 7f40c84d4700 1 --2- 172.21.15.130:0/693835450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0100710 0x7f40c0100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.412+0000 7f40c84d4700 1 -- 172.21.15.130:0/693835450 >> 172.21.15.130:0/693835450 conn(0x7f40c00faa90 msgr2=0x7f40c00fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:10.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.412+0000 7f40c84d4700 1 -- 172.21.15.130:0/693835450 shutdown_connections 2023-09-12T16:02:10.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.412+0000 7f40c84d4700 1 -- 172.21.15.130:0/693835450 wait complete. 2023-09-12T16:02:10.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c84d4700 1 Processor -- start 2023-09-12T16:02:10.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c84d4700 1 -- start start 2023-09-12T16:02:10.417 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c84d4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c00ff520 0x7f40c01952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:10.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c84d4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0100710 0x7f40c0195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:10.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c84d4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 0x7f40c019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:10.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c84d4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c019ce50 con 0x7f40c01010b0 2023-09-12T16:02:10.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c84d4700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c019cf90 con 0x7f40c00ff520 2023-09-12T16:02:10.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c84d4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c019d120 con 0x7f40c0100710 2023-09-12T16:02:10.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c6a71700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 0x7f40c019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:10.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c6a71700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 0x7f40c019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53336/0 (socket says 172.21.15.130:53336) 2023-09-12T16:02:10.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c6a71700 1 -- 172.21.15.130:0/1280276408 learned_addr learned my addr 172.21.15.130:0/1280276408 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:10.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.413+0000 7f40c5a6f700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0100710 0x7f40c0195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:10.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c6270700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c00ff520 0x7f40c01952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:10.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c6a71700 1 -- 172.21.15.130:0/1280276408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0100710 msgr2=0x7f40c0195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c6a71700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0100710 0x7f40c0195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c6a71700 1 -- 172.21.15.130:0/1280276408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c00ff520 msgr2=0x7f40c01952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c6a71700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c00ff520 0x7f40c01952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c6a71700 1 -- 172.21.15.130:0/1280276408 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8009d00 con 0x7f40c01010b0 2023-09-12T16:02:10.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c6270700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c00ff520 0x7f40c01952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:10.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c6a71700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 0x7f40c019c880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f40b8007a00 tx=0x7f40b8009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:10.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40b77fe700 1 -- 172.21.15.130:0/1280276408 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b800e040 con 0x7f40c01010b0 2023-09-12T16:02:10.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40b77fe700 1 -- 172.21.15.130:0/1280276408 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b8003e10 con 0x7f40c01010b0 2023-09-12T16:02:10.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c0068730 con 0x7f40c01010b0 2023-09-12T16:02:10.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40b77fe700 1 -- 172.21.15.130:0/1280276408 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8005610 con 0x7f40c01010b0 2023-09-12T16:02:10.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.414+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c0068c60 con 0x7f40c01010b0 2023-09-12T16:02:10.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.415+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c010add0 con 0x7f40c01010b0 2023-09-12T16:02:10.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.416+0000 7f40b77fe700 1 -- 172.21.15.130:0/1280276408 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f40b8005770 con 0x7f40c01010b0 2023-09-12T16:02:10.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.417+0000 7f40b77fe700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f40ac0a0710 0x7f40ac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:10.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.417+0000 7f40c6270700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f40ac0a0710 0x7f40ac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:10.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.417+0000 7f40b77fe700 1 -- 172.21.15.130:0/1280276408 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f40b801e070 con 0x7f40c01010b0 2023-09-12T16:02:10.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.417+0000 7f40c6270700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f40ac0a0710 0x7f40ac0a2bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f40c0100570 tx=0x7f40b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:10.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.420+0000 7f40b77fe700 1 -- 172.21.15.130:0/1280276408 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f40b8098b70 con 0x7f40c01010b0 2023-09-12T16:02:10.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.653+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40c0106350 con 0x7f40c01010b0 2023-09-12T16:02:10.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.654+0000 7f40b77fe700 1 -- 172.21.15.130:0/1280276408 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11282 (secure 0 0 0) 0x7f40b809c190 con 0x7f40c01010b0 2023-09-12T16:02:10.654 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:10.654 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24557,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24592,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:09.714954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-12T16:02:10.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.656+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f40ac0a0710 msgr2=0x7f40ac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.656+0000 7f40c84d4700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f40ac0a0710 0x7f40ac0a2bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f40c0100570 tx=0x7f40b0008040 comp rx=0 tx=0).stop 2023-09-12T16:02:10.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 msgr2=0x7f40c019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 0x7f40c019c880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f40b8007a00 tx=0x7f40b8009570 comp rx=0 tx=0).stop 2023-09-12T16:02:10.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 shutdown_connections 2023-09-12T16:02:10.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f40ac0a0710 0x7f40ac0a2bd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c01010b0 0x7f40c019c880 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c00ff520 0x7f40c01952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 --2- 172.21.15.130:0/1280276408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0100710 0x7f40c0195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 >> 172.21.15.130:0/1280276408 conn(0x7f40c00faa90 msgr2=0x7f40c00fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:10.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 shutdown_connections 2023-09-12T16:02:10.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.657+0000 7f40c84d4700 1 -- 172.21.15.130:0/1280276408 wait complete. 2023-09-12T16:02:10.659 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 17 2023-09-12T16:02:10.667 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:10 smithi130 ceph-mon[98670]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-12T16:02:10.667 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:10 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1865963473' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-12T16:02:10.667 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:10 smithi130 ceph-mon[98670]: osdmap e84: 12 total, 12 up, 12 in 2023-09-12T16:02:10.668 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:10 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3525192433' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-12T16:02:10.669 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:02:10.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:10 smithi159 ceph-mon[102522]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-12T16:02:10.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:10 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1865963473' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-12T16:02:10.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:10 smithi159 ceph-mon[102522]: osdmap e84: 12 total, 12 up, 12 in 2023-09-12T16:02:10.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:10 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3525192433' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-12T16:02:10.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:10 smithi203 ceph-mon[103431]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-12T16:02:10.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:10 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1865963473' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-12T16:02:10.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:10 smithi203 ceph-mon[103431]: osdmap e84: 12 total, 12 up, 12 in 2023-09-12T16:02:10.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:10 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3525192433' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-12T16:02:10.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.791+0000 7f77f11ea700 1 -- 172.21.15.130:0/2506652047 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec1006d0 msgr2=0x7f77ec100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.791+0000 7f77f11ea700 1 --2- 172.21.15.130:0/2506652047 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec1006d0 0x7f77ec100b30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f77e00099c0 tx=0x7f77e0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.791+0000 7f77f11ea700 1 -- 172.21.15.130:0/2506652047 shutdown_connections 2023-09-12T16:02:10.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.791+0000 7f77f11ea700 1 --2- 172.21.15.130:0/2506652047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ec101070 0x7f77ec107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.791+0000 7f77f11ea700 1 --2- 172.21.15.130:0/2506652047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77ec0ff4e0 0x7f77ec0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.791+0000 7f77f11ea700 1 --2- 172.21.15.130:0/2506652047 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec1006d0 0x7f77ec100b30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.791+0000 7f77f11ea700 1 -- 172.21.15.130:0/2506652047 >> 172.21.15.130:0/2506652047 conn(0x7f77ec0faa30 msgr2=0x7f77ec0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:10.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.791+0000 7f77f11ea700 1 -- 172.21.15.130:0/2506652047 shutdown_connections 2023-09-12T16:02:10.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.791+0000 7f77f11ea700 1 -- 172.21.15.130:0/2506652047 wait complete. 2023-09-12T16:02:10.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.792+0000 7f77f11ea700 1 Processor -- start 2023-09-12T16:02:10.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.792+0000 7f77f11ea700 1 -- start start 2023-09-12T16:02:10.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77f11ea700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77ec0ff4e0 0x7f77ec198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:10.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77f11ea700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ec1006d0 0x7f77ec198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:10.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77f11ea700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec101070 0x7f77ec19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:10.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77f11ea700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec19ffa0 con 0x7f77ec101070 2023-09-12T16:02:10.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77f11ea700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec1a00e0 con 0x7f77ec0ff4e0 2023-09-12T16:02:10.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77f11ea700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec1a0270 con 0x7f77ec1006d0 2023-09-12T16:02:10.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77eb59e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec101070 0x7f77ec19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:10.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77eb59e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec101070 0x7f77ec19f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53350/0 (socket says 172.21.15.130:53350) 2023-09-12T16:02:10.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77eb59e700 1 -- 172.21.15.130:0/103554571 learned_addr learned my addr 172.21.15.130:0/103554571 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:10.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77ea59c700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ec1006d0 0x7f77ec198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:10.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77ead9d700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77ec0ff4e0 0x7f77ec198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:10.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77eb59e700 1 -- 172.21.15.130:0/103554571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ec1006d0 msgr2=0x7f77ec198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77eb59e700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ec1006d0 0x7f77ec198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77eb59e700 1 -- 172.21.15.130:0/103554571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77ec0ff4e0 msgr2=0x7f77ec198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77eb59e700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77ec0ff4e0 0x7f77ec198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77eb59e700 1 -- 172.21.15.130:0/103554571 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e00095f0 con 0x7f77ec101070 2023-09-12T16:02:10.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77eb59e700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec101070 0x7f77ec19f9d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f77e4007d70 tx=0x7f77e400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:10.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77d3fff700 1 -- 172.21.15.130:0/103554571 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e400ad40 con 0x7f77ec101070 2023-09-12T16:02:10.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ec1a0550 con 0x7f77ec101070 2023-09-12T16:02:10.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77d3fff700 1 -- 172.21.15.130:0/103554571 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77e40128e0 con 0x7f77ec101070 2023-09-12T16:02:10.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77d3fff700 1 -- 172.21.15.130:0/103554571 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4011780 con 0x7f77ec101070 2023-09-12T16:02:10.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.793+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ec1a0a60 con 0x7f77ec101070 2023-09-12T16:02:10.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.794+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ec10ad00 con 0x7f77ec101070 2023-09-12T16:02:10.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.795+0000 7f77d3fff700 1 -- 172.21.15.130:0/103554571 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f77e4012a50 con 0x7f77ec101070 2023-09-12T16:02:10.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.796+0000 7f77d3fff700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77d40a0690 0x7f77d40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:10.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.796+0000 7f77d3fff700 1 -- 172.21.15.130:0/103554571 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f77e40cc560 con 0x7f77ec101070 2023-09-12T16:02:10.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.796+0000 7f77ead9d700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77d40a0690 0x7f77d40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:10.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.796+0000 7f77ead9d700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77d40a0690 0x7f77d40a2b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f77ec100530 tx=0x7f77dc005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:10.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.799+0000 7f77d3fff700 1 -- 172.21.15.130:0/103554571 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f77e4095ea0 con 0x7f77ec101070 2023-09-12T16:02:10.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.993+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77ec10aec0 con 0x7f77ec101070 2023-09-12T16:02:10.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.995+0000 7f77d3fff700 1 -- 172.21.15.130:0/103554571 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24017 (secure 0 0 0) 0x7f77e40994c0 con 0x7f77ec101070 2023-09-12T16:02:10.995 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:10.995 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":85,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:09.714763+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:10.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77d40a0690 msgr2=0x7f77d40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77d40a0690 0x7f77d40a2b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f77ec100530 tx=0x7f77dc005cf0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec101070 msgr2=0x7f77ec19f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:10.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec101070 0x7f77ec19f9d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f77e4007d70 tx=0x7f77e400a340 comp rx=0 tx=0).stop 2023-09-12T16:02:10.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 shutdown_connections 2023-09-12T16:02:10.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77d40a0690 0x7f77d40a2b50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77ec101070 0x7f77ec19f9d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77ec0ff4e0 0x7f77ec198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 --2- 172.21.15.130:0/103554571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ec1006d0 0x7f77ec198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:10.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 >> 172.21.15.130:0/103554571 conn(0x7f77ec0faa30 msgr2=0x7f77ec0fb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:10.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 shutdown_connections 2023-09-12T16:02:10.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:10.997+0000 7f77f11ea700 1 -- 172.21.15.130:0/103554571 wait complete. 2023-09-12T16:02:11.010 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:02:11.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.131+0000 7f0516cda700 1 -- 172.21.15.130:0/1589727091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0510101070 msgr2=0x7f0510107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.131+0000 7f0516cda700 1 --2- 172.21.15.130:0/1589727091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0510101070 0x7f0510107920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f050800b0d0 tx=0x7f050800b3e0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.131+0000 7f0516cda700 1 -- 172.21.15.130:0/1589727091 shutdown_connections 2023-09-12T16:02:11.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.131+0000 7f0516cda700 1 --2- 172.21.15.130:0/1589727091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0510101070 0x7f0510107920 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.131+0000 7f0516cda700 1 --2- 172.21.15.130:0/1589727091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05100ff4e0 0x7f05100ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.131+0000 7f0516cda700 1 --2- 172.21.15.130:0/1589727091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05101006d0 0x7f0510100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.131+0000 7f0516cda700 1 -- 172.21.15.130:0/1589727091 >> 172.21.15.130:0/1589727091 conn(0x7f05100faa30 msgr2=0x7f05100fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:11.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.131+0000 7f0516cda700 1 -- 172.21.15.130:0/1589727091 shutdown_connections 2023-09-12T16:02:11.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.131+0000 7f0516cda700 1 -- 172.21.15.130:0/1589727091 wait complete. 2023-09-12T16:02:11.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0516cda700 1 Processor -- start 2023-09-12T16:02:11.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0516cda700 1 -- start start 2023-09-12T16:02:11.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0516cda700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05100ff4e0 0x7f05100682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0516cda700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05101006d0 0x7f051019b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0516cda700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0510101070 0x7f051019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0516cda700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051019def0 con 0x7f05101006d0 2023-09-12T16:02:11.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0516cda700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051019e060 con 0x7f0510101070 2023-09-12T16:02:11.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0516cda700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051019e220 con 0x7f05100ff4e0 2023-09-12T16:02:11.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05101006d0 0x7f051019b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05101006d0 0x7f051019b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53362/0 (socket says 172.21.15.130:53362) 2023-09-12T16:02:11.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050ffff700 1 -- 172.21.15.130:0/517918706 learned_addr learned my addr 172.21.15.130:0/517918706 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:11.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0515277700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0510101070 0x7f051019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0514a76700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05100ff4e0 0x7f05100682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050ffff700 1 -- 172.21.15.130:0/517918706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05100ff4e0 msgr2=0x7f05100682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050ffff700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05100ff4e0 0x7f05100682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050ffff700 1 -- 172.21.15.130:0/517918706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0510101070 msgr2=0x7f051019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050ffff700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0510101070 0x7f051019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050ffff700 1 -- 172.21.15.130:0/517918706 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0508009d00 con 0x7f05101006d0 2023-09-12T16:02:11.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0514a76700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05100ff4e0 0x7f05100682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:11.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050ffff700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05101006d0 0x7f051019b0d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00d950 tx=0x7f04fc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:11.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050dffb700 1 -- 172.21.15.130:0/517918706 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc009b40 con 0x7f05101006d0 2023-09-12T16:02:11.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051019e500 con 0x7f05101006d0 2023-09-12T16:02:11.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050dffb700 1 -- 172.21.15.130:0/517918706 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04fc009ca0 con 0x7f05101006d0 2023-09-12T16:02:11.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.132+0000 7f050dffb700 1 -- 172.21.15.130:0/517918706 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc00f620 con 0x7f05101006d0 2023-09-12T16:02:11.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.133+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051019ea60 con 0x7f05101006d0 2023-09-12T16:02:11.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.134+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0510106a00 con 0x7f05101006d0 2023-09-12T16:02:11.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.135+0000 7f050dffb700 1 -- 172.21.15.130:0/517918706 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f04fc010460 con 0x7f05101006d0 2023-09-12T16:02:11.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.135+0000 7f050dffb700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f05000a0690 0x7f05000a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.135+0000 7f0514a76700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f05000a0690 0x7f05000a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.135+0000 7f050dffb700 1 -- 172.21.15.130:0/517918706 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f04fc0cc160 con 0x7f05101006d0 2023-09-12T16:02:11.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.135+0000 7f0514a76700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f05000a0690 0x7f05000a2b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0504005fd0 tx=0x7f0504005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:11.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.139+0000 7f050dffb700 1 -- 172.21.15.130:0/517918706 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f04fc095aa0 con 0x7f05101006d0 2023-09-12T16:02:11.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.337+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f051002cee0 con 0x7f05101006d0 2023-09-12T16:02:11.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.338+0000 7f050dffb700 1 -- 172.21.15.130:0/517918706 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24004 (secure 0 0 0) 0x7f04fc0990c0 con 0x7f05101006d0 2023-09-12T16:02:11.338 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:11.339 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":86,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:11.262527+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:11.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f05000a0690 msgr2=0x7f05000a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f05000a0690 0x7f05000a2b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0504005fd0 tx=0x7f0504005f00 comp rx=0 tx=0).stop 2023-09-12T16:02:11.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05101006d0 msgr2=0x7f051019b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05101006d0 0x7f051019b0d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00d950 tx=0x7f04fc00dd10 comp rx=0 tx=0).stop 2023-09-12T16:02:11.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 shutdown_connections 2023-09-12T16:02:11.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f05000a0690 0x7f05000a2b50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0510101070 0x7f051019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05100ff4e0 0x7f05100682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 --2- 172.21.15.130:0/517918706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05101006d0 0x7f051019b0d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 >> 172.21.15.130:0/517918706 conn(0x7f05100faa30 msgr2=0x7f0510103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:11.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 shutdown_connections 2023-09-12T16:02:11.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.341+0000 7f0516cda700 1 -- 172.21.15.130:0/517918706 wait complete. 2023-09-12T16:02:11.368 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:02:11.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.490+0000 7fbf79982700 1 -- 172.21.15.130:0/3415514967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf740fe910 msgr2=0x7fbf740fed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.490+0000 7fbf79982700 1 --2- 172.21.15.130:0/3415514967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf740fe910 0x7fbf740fed30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009990 tx=0x7fbf68009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.490+0000 7fbf79982700 1 -- 172.21.15.130:0/3415514967 shutdown_connections 2023-09-12T16:02:11.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.490+0000 7fbf79982700 1 --2- 172.21.15.130:0/3415514967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf74102320 0x7fbf74108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.490+0000 7fbf79982700 1 --2- 172.21.15.130:0/3415514967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf740fe910 0x7fbf740fed30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.490+0000 7fbf79982700 1 --2- 172.21.15.130:0/3415514967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf74101980 0x7fbf74101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.490+0000 7fbf79982700 1 -- 172.21.15.130:0/3415514967 >> 172.21.15.130:0/3415514967 conn(0x7fbf740fa200 msgr2=0x7fbf740fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:11.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.490+0000 7fbf79982700 1 -- 172.21.15.130:0/3415514967 shutdown_connections 2023-09-12T16:02:11.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.490+0000 7fbf79982700 1 -- 172.21.15.130:0/3415514967 wait complete. 2023-09-12T16:02:11.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf79982700 1 Processor -- start 2023-09-12T16:02:11.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf79982700 1 -- start start 2023-09-12T16:02:11.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf79982700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf740fe910 0x7fbf741962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf79982700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf74101980 0x7fbf74196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf79982700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf74102320 0x7fbf7419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf79982700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7419de60 con 0x7fbf74101980 2023-09-12T16:02:11.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf79982700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7419dfa0 con 0x7fbf74102320 2023-09-12T16:02:11.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf79982700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7419e130 con 0x7fbf740fe910 2023-09-12T16:02:11.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf727fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf74101980 0x7fbf74196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf727fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf74101980 0x7fbf74196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53374/0 (socket says 172.21.15.130:53374) 2023-09-12T16:02:11.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf727fc700 1 -- 172.21.15.130:0/3271731650 learned_addr learned my addr 172.21.15.130:0/3271731650 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:11.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf72ffd700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf740fe910 0x7fbf741962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf737fe700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf74102320 0x7fbf7419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf727fc700 1 -- 172.21.15.130:0/3271731650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf740fe910 msgr2=0x7fbf741962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf727fc700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf740fe910 0x7fbf741962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf727fc700 1 -- 172.21.15.130:0/3271731650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf74102320 msgr2=0x7fbf7419d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf727fc700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf74102320 0x7fbf7419d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf727fc700 1 -- 172.21.15.130:0/3271731650 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680095f0 con 0x7fbf74101980 2023-09-12T16:02:11.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.491+0000 7fbf737fe700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf74102320 0x7fbf7419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:11.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.492+0000 7fbf727fc700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf74101980 0x7fbf74196820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400b530 tx=0x7fbf6400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:11.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.492+0000 7fbf78980700 1 -- 172.21.15.130:0/3271731650 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64011830 con 0x7fbf74101980 2023-09-12T16:02:11.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.492+0000 7fbf78980700 1 -- 172.21.15.130:0/3271731650 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf64011e50 con 0x7fbf74101980 2023-09-12T16:02:11.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.492+0000 7fbf78980700 1 -- 172.21.15.130:0/3271731650 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400f2e0 con 0x7fbf74101980 2023-09-12T16:02:11.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.492+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7419e410 con 0x7fbf74101980 2023-09-12T16:02:11.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.492+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7419e8d0 con 0x7fbf74101980 2023-09-12T16:02:11.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.493+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7410bed0 con 0x7fbf74101980 2023-09-12T16:02:11.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.494+0000 7fbf78980700 1 -- 172.21.15.130:0/3271731650 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fbf64010380 con 0x7fbf74101980 2023-09-12T16:02:11.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.495+0000 7fbf78980700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbf5c0a0690 0x7fbf5c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.495+0000 7fbf72ffd700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbf5c0a0690 0x7fbf5c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.495+0000 7fbf78980700 1 -- 172.21.15.130:0/3271731650 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbf640cbed0 con 0x7fbf74101980 2023-09-12T16:02:11.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.495+0000 7fbf72ffd700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbf5c0a0690 0x7fbf5c0a2b50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800b5c0 tx=0x7fbf68005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:11.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.498+0000 7fbf78980700 1 -- 172.21.15.130:0/3271731650 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbf640957b0 con 0x7fbf74101980 2023-09-12T16:02:11.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:11 smithi130 ceph-mon[98670]: osdmap e85: 12 total, 12 up, 12 in 2023-09-12T16:02:11.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:11 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3525192433' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-12T16:02:11.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:11 smithi130 ceph-mon[98670]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:11.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:11 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1280276408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:11.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:11 smithi130 ceph-mon[98670]: pgmap v265: 129 pgs: 104 active+clean, 1 creating+activating, 7 creating+peering, 17 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2023-09-12T16:02:11.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:11 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/103554571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:11.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:11 smithi159 ceph-mon[102522]: osdmap e85: 12 total, 12 up, 12 in 2023-09-12T16:02:11.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:11 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3525192433' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-12T16:02:11.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:11 smithi159 ceph-mon[102522]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:11.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:11 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1280276408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:11.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:11 smithi159 ceph-mon[102522]: pgmap v265: 129 pgs: 104 active+clean, 1 creating+activating, 7 creating+peering, 17 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2023-09-12T16:02:11.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:11 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/103554571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:11.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.733+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf74107640 con 0x7fbf74101980 2023-09-12T16:02:11.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.734+0000 7fbf78980700 1 -- 172.21.15.130:0/3271731650 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11282 (secure 0 0 0) 0x7fbf64098dd0 con 0x7fbf74101980 2023-09-12T16:02:11.734 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:11.734 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24557,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24592,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:09.714954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-12T16:02:11.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.736+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbf5c0a0690 msgr2=0x7fbf5c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.736+0000 7fbf79982700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbf5c0a0690 0x7fbf5c0a2b50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800b5c0 tx=0x7fbf68005fd0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.736+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf74101980 msgr2=0x7fbf74196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.736+0000 7fbf79982700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf74101980 0x7fbf74196820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400b530 tx=0x7fbf6400b840 comp rx=0 tx=0).stop 2023-09-12T16:02:11.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.737+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 shutdown_connections 2023-09-12T16:02:11.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.737+0000 7fbf79982700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbf5c0a0690 0x7fbf5c0a2b50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.737+0000 7fbf79982700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf74102320 0x7fbf7419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.737+0000 7fbf79982700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf740fe910 0x7fbf741962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.737+0000 7fbf79982700 1 --2- 172.21.15.130:0/3271731650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf74101980 0x7fbf74196820 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.737+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 >> 172.21.15.130:0/3271731650 conn(0x7fbf740fa200 msgr2=0x7fbf740fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:11.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.737+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 shutdown_connections 2023-09-12T16:02:11.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.737+0000 7fbf79982700 1 -- 172.21.15.130:0/3271731650 wait complete. 2023-09-12T16:02:11.739 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 17 2023-09-12T16:02:11.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:11 smithi203 ceph-mon[103431]: osdmap e85: 12 total, 12 up, 12 in 2023-09-12T16:02:11.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:11 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3525192433' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-12T16:02:11.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:11 smithi203 ceph-mon[103431]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:11.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:11 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1280276408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:11.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:11 smithi203 ceph-mon[103431]: pgmap v265: 129 pgs: 104 active+clean, 1 creating+activating, 7 creating+peering, 17 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2023-09-12T16:02:11.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:11 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/103554571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:11.749 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:02:11.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.871+0000 7f655bc5f700 1 -- 172.21.15.130:0/2478840454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 msgr2=0x7f6554108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.871+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2478840454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 0x7f6554108a60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f654c00b0d0 tx=0x7f654c00b3e0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.871+0000 7f655bc5f700 1 -- 172.21.15.130:0/2478840454 shutdown_connections 2023-09-12T16:02:11.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.871+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2478840454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 0x7f6554108a60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.871+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2478840454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6554100620 0x7f6554100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.871+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2478840454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6554101810 0x7f6554101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.871+0000 7f655bc5f700 1 -- 172.21.15.130:0/2478840454 >> 172.21.15.130:0/2478840454 conn(0x7f65540fbb70 msgr2=0x7f65540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:11.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 -- 172.21.15.130:0/2478840454 shutdown_connections 2023-09-12T16:02:11.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 -- 172.21.15.130:0/2478840454 wait complete. 2023-09-12T16:02:11.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 Processor -- start 2023-09-12T16:02:11.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 -- start start 2023-09-12T16:02:11.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6554100620 0x7f6554196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6554101810 0x7f6554196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 0x7f655419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655419ded0 con 0x7f65541021b0 2023-09-12T16:02:11.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655419e010 con 0x7f6554101810 2023-09-12T16:02:11.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655bc5f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655419e1a0 con 0x7f6554100620 2023-09-12T16:02:11.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655a1fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 0x7f655419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655a1fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 0x7f655419d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53390/0 (socket says 172.21.15.130:53390) 2023-09-12T16:02:11.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.872+0000 7f655a1fc700 1 -- 172.21.15.130:0/2286652670 learned_addr learned my addr 172.21.15.130:0/2286652670 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:11.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f65599fb700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6554100620 0x7f6554196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f65591fa700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6554101810 0x7f6554196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f655a1fc700 1 -- 172.21.15.130:0/2286652670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6554100620 msgr2=0x7f6554196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f655a1fc700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6554100620 0x7f6554196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f655a1fc700 1 -- 172.21.15.130:0/2286652670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6554101810 msgr2=0x7f6554196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:11.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f655a1fc700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6554101810 0x7f6554196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:11.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f655a1fc700 1 -- 172.21.15.130:0/2286652670 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c009d00 con 0x7f65541021b0 2023-09-12T16:02:11.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f655a1fc700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 0x7f655419d900 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f654c007a00 tx=0x7f654c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:11.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f654affd700 1 -- 172.21.15.130:0/2286652670 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00e040 con 0x7f65541021b0 2023-09-12T16:02:11.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f654affd700 1 -- 172.21.15.130:0/2286652670 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f654c003e10 con 0x7f65541021b0 2023-09-12T16:02:11.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655419e420 con 0x7f65541021b0 2023-09-12T16:02:11.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f654affd700 1 -- 172.21.15.130:0/2286652670 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c01b9e0 con 0x7f65541021b0 2023-09-12T16:02:11.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.873+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655419e840 con 0x7f65541021b0 2023-09-12T16:02:11.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.874+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655410be40 con 0x7f65541021b0 2023-09-12T16:02:11.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.875+0000 7f654affd700 1 -- 172.21.15.130:0/2286652670 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f654c005630 con 0x7f65541021b0 2023-09-12T16:02:11.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.876+0000 7f654affd700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65400a0690 0x7f65400a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:11.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.876+0000 7f654affd700 1 -- 172.21.15.130:0/2286652670 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f654c0cdb40 con 0x7f65541021b0 2023-09-12T16:02:11.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.876+0000 7f65599fb700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65400a0690 0x7f65400a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:11.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.876+0000 7f65599fb700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65400a0690 0x7f65400a2b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6554101670 tx=0x7f654400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:11.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:11.879+0000 7f654affd700 1 -- 172.21.15.130:0/2286652670 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f654c097510 con 0x7f65541021b0 2023-09-12T16:02:12.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.075+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6554107410 con 0x7f65541021b0 2023-09-12T16:02:12.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.076+0000 7f654affd700 1 -- 172.21.15.130:0/2286652670 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24004 (secure 0 0 0) 0x7f654c09ab30 con 0x7f65541021b0 2023-09-12T16:02:12.076 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:12.076 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":86,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:11.262527+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:12.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65400a0690 msgr2=0x7f65400a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65400a0690 0x7f65400a2b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6554101670 tx=0x7f654400b410 comp rx=0 tx=0).stop 2023-09-12T16:02:12.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 msgr2=0x7f655419d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 0x7f655419d900 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f654c007a00 tx=0x7f654c0094c0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 shutdown_connections 2023-09-12T16:02:12.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f65400a0690 0x7f65400a2b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f65541021b0 0x7f655419d900 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6554100620 0x7f6554196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 --2- 172.21.15.130:0/2286652670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6554101810 0x7f6554196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.078+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 >> 172.21.15.130:0/2286652670 conn(0x7f65540fbb70 msgr2=0x7f65540fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:12.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.079+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 shutdown_connections 2023-09-12T16:02:12.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.079+0000 7f655bc5f700 1 -- 172.21.15.130:0/2286652670 wait complete. 2023-09-12T16:02:12.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-12T16:02:04.422106+0000', 'modified': '2023-09-12T16:02:09.714954+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24593}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6839/3094783118', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 3094783118}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 3094783118}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-09-12T16:02:12.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-12T16:02:12.091 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-12T16:02:12.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 -- 172.21.15.130:0/782194421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 msgr2=0x7fc8e4105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/782194421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 0x7fc8e4105df0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009990 tx=0x7fc8d4009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 -- 172.21.15.130:0/782194421 shutdown_connections 2023-09-12T16:02:12.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/782194421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e4106330 0x7fc8e4108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/782194421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e41010d0 0x7fc8e41034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/782194421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 0x7fc8e4105df0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 -- 172.21.15.130:0/782194421 >> 172.21.15.130:0/782194421 conn(0x7fc8e40faa60 msgr2=0x7fc8e40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:12.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 -- 172.21.15.130:0/782194421 shutdown_connections 2023-09-12T16:02:12.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 -- 172.21.15.130:0/782194421 wait complete. 2023-09-12T16:02:12.218 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.214+0000 7fc8e9d74700 1 Processor -- start 2023-09-12T16:02:12.218 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e9d74700 1 -- start start 2023-09-12T16:02:12.218 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e9d74700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e41010d0 0x7fc8e41962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.218 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e9d74700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 0x7fc8e4196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e9d74700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e4106330 0x7fc8e419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e9d74700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e419de60 con 0x7fc8e4103a00 2023-09-12T16:02:12.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e9d74700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e419dfa0 con 0x7fc8e41010d0 2023-09-12T16:02:12.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e9d74700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e419e130 con 0x7fc8e4106330 2023-09-12T16:02:12.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 0x7fc8e4196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 0x7fc8e4196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53402/0 (socket says 172.21.15.130:53402) 2023-09-12T16:02:12.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/4124001729 learned_addr learned my addr 172.21.15.130:0/4124001729 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:12.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e41010d0 0x7fc8e41962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e4106330 0x7fc8e419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/4124001729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e4106330 msgr2=0x7fc8e419d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e4106330 0x7fc8e419d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/4124001729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e41010d0 msgr2=0x7fc8e41962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e41010d0 0x7fc8e41962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/4124001729 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d40095f0 con 0x7fc8e4103a00 2023-09-12T16:02:12.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e4106330 0x7fc8e419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:12.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.215+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 0x7fc8e4196820 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4000c00 tx=0x7fc8d400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:12.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.216+0000 7fc8e0ff9700 1 -- 172.21.15.130:0/4124001729 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d401d070 con 0x7fc8e4103a00 2023-09-12T16:02:12.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.216+0000 7fc8e9d74700 1 -- 172.21.15.130:0/4124001729 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e419e3b0 con 0x7fc8e4103a00 2023-09-12T16:02:12.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.216+0000 7fc8e0ff9700 1 -- 172.21.15.130:0/4124001729 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8d400fb50 con 0x7fc8e4103a00 2023-09-12T16:02:12.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.216+0000 7fc8e0ff9700 1 -- 172.21.15.130:0/4124001729 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4017650 con 0x7fc8e4103a00 2023-09-12T16:02:12.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.216+0000 7fc8e9d74700 1 -- 172.21.15.130:0/4124001729 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e419e8b0 con 0x7fc8e4103a00 2023-09-12T16:02:12.230 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.218+0000 7fc8e0ff9700 1 -- 172.21.15.130:0/4124001729 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fc8d400fcc0 con 0x7fc8e4103a00 2023-09-12T16:02:12.230 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.218+0000 7fc8e0ff9700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8cc0a0640 0x7fc8cc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.230 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.218+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8cc0a0640 0x7fc8cc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.218+0000 7fc8e0ff9700 1 -- 172.21.15.130:0/4124001729 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc8d40cd1e0 con 0x7fc8e4103a00 2023-09-12T16:02:12.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.219+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8cc0a0640 0x7fc8cc0a2b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005950 tx=0x7fc8d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:12.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.219+0000 7fc8f5f3ab80 1 Processor -- start 2023-09-12T16:02:12.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.219+0000 7fc8f5f3ab80 1 -- start start 2023-09-12T16:02:12.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 0x55de817cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de817ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de817276e0 con 0x55de817d0550 2023-09-12T16:02:12.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de81725000 con 0x55de817cedb0 2023-09-12T16:02:12.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de817208f0 con 0x55de817cfbb0 2023-09-12T16:02:12.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de817ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de817ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53410/0 (socket says 172.21.15.130:53410) 2023-09-12T16:02:12.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/3029369191 learned_addr learned my addr 172.21.15.130:0/3029369191 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:12.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 0x55de817cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/3029369191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 msgr2=0x55de817cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:02:12.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 0x55de817cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/3029369191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 msgr2=0x55de817cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:02:12.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/3029369191 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de8154cd90 con 0x55de817d0550 2023-09-12T16:02:12.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.220+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:12.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.221+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de817ce7b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40a2440 tx=0x7fc8d400b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1857bded6f8dc9c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:12.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.221+0000 7fc8c9ffb700 1 -- 172.21.15.130:0/3029369191 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d409a990 con 0x55de817d0550 2023-09-12T16:02:12.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.221+0000 7fc8c9ffb700 1 -- 172.21.15.130:0/3029369191 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8d409e7c0 con 0x55de817d0550 2023-09-12T16:02:12.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.221+0000 7fc8c9ffb700 1 -- 172.21.15.130:0/3029369191 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40a24e0 con 0x55de817d0550 2023-09-12T16:02:12.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/3029369191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 msgr2=0x55de817ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de817ce7b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40a2440 tx=0x7fc8d400b540 comp rx=0 tx=0).stop 2023-09-12T16:02:12.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/3029369191 shutdown_connections 2023-09-12T16:02:12.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 0x55de817cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/3029369191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de817ce7b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/3029369191 >> 172.21.15.130:0/3029369191 conn(0x55de81743040 msgr2=0x55de817c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:12.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/3029369191 shutdown_connections 2023-09-12T16:02:12.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/3029369191 wait complete. 2023-09-12T16:02:12.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 -- start start 2023-09-12T16:02:12.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.222+0000 7fc8f5f3ab80 1 Processor -- start 2023-09-12T16:02:12.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817d8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cfbb0 0x55de817d87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817df800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de816beb20 con 0x55de817cfbb0 2023-09-12T16:02:12.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de816be820 con 0x55de817cedb0 2023-09-12T16:02:12.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de815a28d0 con 0x55de817d0550 2023-09-12T16:02:12.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 10 client.0 _ll_register_callbacks cb 0x55de81608a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:02:12.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cfbb0 0x55de817d87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cfbb0 0x55de817d87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53416/0 (socket says 172.21.15.130:53416) 2023-09-12T16:02:12.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/743987519 learned_addr learned my addr 172.21.15.130:0/743987519 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:12.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817d8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817df800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/743987519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 msgr2=0x55de817df800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:02:12.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817df800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/743987519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 msgr2=0x55de817d8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:02:12.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817d8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e2ffd700 1 -- 172.21.15.130:0/743987519 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e419e8b0 con 0x55de817cfbb0 2023-09-12T16:02:12.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817df800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:12.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cfbb0 0x55de817d87c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400ff70 tx=0x7fc8d4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2728529ef3efea6d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:12.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:02:12.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/743987519 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4096780 con 0x55de817cfbb0 2023-09-12T16:02:12.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/743987519 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de816784a0 con 0x55de817cfbb0 2023-09-12T16:02:12.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/743987519 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc8d40968e0 con 0x55de817cfbb0 2023-09-12T16:02:12.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/743987519 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8d4096aa0 con 0x55de817cfbb0 2023-09-12T16:02:12.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 10 client.14904 fetch_fsmap learned FSMap version 17 2023-09-12T16:02:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.223+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/743987519 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55de81535c10 con 0x55de817cfbb0 2023-09-12T16:02:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/743987519 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8d407f020 con 0x55de817cfbb0 2023-09-12T16:02:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/743987519 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc8d409e120 con 0x55de817cfbb0 2023-09-12T16:02:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8f5f3ab80 10 client.14904 fetch_fsmap finished waiting for FSMap version 17 2023-09-12T16:02:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8f5f3ab80 10 client.14904 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-12T16:02:12.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8f5f3ab80 20 client.14904 populate_metadata read hostname 'smithi130' 2023-09-12T16:02:12.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55de817e33e0 0x55de817e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8f5f3ab80 4 client.14904 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:02:12.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/743987519 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de81735850 con 0x55de817e33e0 2023-09-12T16:02:12.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55de817e33e0 0x55de817e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8c2ffd700 10 client.14904 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:02:12.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.224+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55de817e33e0 0x55de817e26a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171d26a11931e6e6 server_cookie=3420490216b1d958 in_seq=0 out_seq=0 2023-09-12T16:02:12.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/743987519 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55de81735850 con 0x55de817e33e0 2023-09-12T16:02:12.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8c2ffd700 10 client.14904 handle_command_reply: tid=0 2023-09-12T16:02:12.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 client.14904 shutdown 2023-09-12T16:02:12.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/743987519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cfbb0 msgr2=0x55de817d87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cfbb0 0x55de817d87c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400ff70 tx=0x7fc8d4009310 comp rx=0 tx=0).stop 2023-09-12T16:02:12.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/743987519 shutdown_connections 2023-09-12T16:02:12.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55de817e33e0 0x55de817e26a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817df800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817d8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/743987519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cfbb0 0x55de817d87c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8c2ffd700 0 client.14904 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:02:12.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/743987519 >> 172.21.15.130:0/743987519 conn(0x55de81743040 msgr2=0x55de817c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:12.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/743987519 shutdown_connections 2023-09-12T16:02:12.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.225+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/743987519 wait complete. 2023-09-12T16:02:12.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.226+0000 7fc8f5f3ab80 20 client.14904 trim_cache size 0 max 16384 2023-09-12T16:02:12.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8f5f3ab80 1 Processor -- start 2023-09-12T16:02:12.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8f5f3ab80 1 -- start start 2023-09-12T16:02:12.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cedb0 0x55de817ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cfbb0 0x55de817d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817e0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de815aede0 con 0x55de817cedb0 2023-09-12T16:02:12.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de816e9f70 con 0x55de817cfbb0 2023-09-12T16:02:12.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de817e4850 con 0x55de817d0550 2023-09-12T16:02:12.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8e37fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cedb0 0x55de817ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8e37fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cedb0 0x55de817ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53422/0 (socket says 172.21.15.130:53422) 2023-09-12T16:02:12.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8e37fe700 1 -- 172.21.15.130:0/2364871890 learned_addr learned my addr 172.21.15.130:0/2364871890 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:12.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.243+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817e0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cfbb0 0x55de817d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8e37fe700 1 -- 172.21.15.130:0/2364871890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 msgr2=0x55de817e0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:02:12.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817e0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8e37fe700 1 -- 172.21.15.130:0/2364871890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cfbb0 msgr2=0x55de817d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:02:12.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cfbb0 0x55de817d6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8e37fe700 1 -- 172.21.15.130:0/2364871890 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de81573200 con 0x55de817cedb0 2023-09-12T16:02:12.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817e0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:12.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cfbb0 0x55de817d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:12.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cedb0 0x55de817ce6c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005d90 tx=0x7fc8d8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88452778c76ce18 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:12.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8ca7fc700 1 -- 172.21.15.130:0/2364871890 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8024070 con 0x55de817cedb0 2023-09-12T16:02:12.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8ca7fc700 1 -- 172.21.15.130:0/2364871890 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8d8015070 con 0x55de817cedb0 2023-09-12T16:02:12.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8ca7fc700 1 -- 172.21.15.130:0/2364871890 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8026e60 con 0x55de817cedb0 2023-09-12T16:02:12.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2364871890 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cedb0 msgr2=0x55de817ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cedb0 0x55de817ce6c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005d90 tx=0x7fc8d8013000 comp rx=0 tx=0).stop 2023-09-12T16:02:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2364871890 shutdown_connections 2023-09-12T16:02:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817d0550 0x55de817e0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817cedb0 0x55de817ce6c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2364871890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cfbb0 0x55de817d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2364871890 >> 172.21.15.130:0/2364871890 conn(0x55de81743040 msgr2=0x55de817c9640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:12.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.244+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2364871890 shutdown_connections 2023-09-12T16:02:12.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2364871890 wait complete. 2023-09-12T16:02:12.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 1 -- start start 2023-09-12T16:02:12.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 1 Processor -- start 2023-09-12T16:02:12.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 0x55de817dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de8183b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de817db420 con 0x55de817d0550 2023-09-12T16:02:12.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de817db590 con 0x55de817cedb0 2023-09-12T16:02:12.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8183b990 con 0x55de817cfbb0 2023-09-12T16:02:12.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8e3fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de8183b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8f5f3ab80 10 client.0 _ll_register_callbacks cb 0x7fc8d409a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:02:12.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8e3fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de8183b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53432/0 (socket says 172.21.15.130:53432) 2023-09-12T16:02:12.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.245+0000 7fc8e3fff700 1 -- 172.21.15.130:0/2323080455 learned_addr learned my addr 172.21.15.130:0/2323080455 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:12.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8e2ffd700 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 0x55de817dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8e3fff700 1 -- 172.21.15.130:0/2323080455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 msgr2=0x55de817dac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:02:12.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 0x55de817dac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8e3fff700 1 -- 172.21.15.130:0/2323080455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 msgr2=0x55de817da740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:02:12.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817da740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8e3fff700 1 -- 172.21.15.130:0/2323080455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d800bdb0 con 0x55de817d0550 2023-09-12T16:02:12.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:12.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8e3fff700 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de8183b450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc007b20 tx=0x7fc8dc007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bc4c012f563706 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:12.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:02:12.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/2323080455 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00e040 con 0x55de817d0550 2023-09-12T16:02:12.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/2323080455 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc8dc012730 con 0x55de817d0550 2023-09-12T16:02:12.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2323080455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8d409e7c0 con 0x55de817d0550 2023-09-12T16:02:12.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/2323080455 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8dc0128f0 con 0x55de817d0550 2023-09-12T16:02:12.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8f5f3ab80 10 client.14910 fetch_fsmap learned FSMap version 17 2023-09-12T16:02:12.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2323080455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55de8154b260 con 0x55de817d0550 2023-09-12T16:02:12.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/2323080455 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8d409e7c0 con 0x55de817d0550 2023-09-12T16:02:12.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/2323080455 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc8dc014040 con 0x55de817d0550 2023-09-12T16:02:12.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.246+0000 7fc8f5f3ab80 10 client.14910 fetch_fsmap finished waiting for FSMap version 17 2023-09-12T16:02:12.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8f5f3ab80 10 client.14910 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-12T16:02:12.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8f5f3ab80 20 client.14910 populate_metadata read hostname 'smithi130' 2023-09-12T16:02:12.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55de81840d10 0x55de8183d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:12.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8f5f3ab80 4 client.14910 mds_command: new command op to 24593 tid=0[{"prefix": "status"}] 2023-09-12T16:02:12.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2323080455 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55de817198d0 con 0x55de81840d10 2023-09-12T16:02:12.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55de81840d10 0x55de8183d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:12.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8e37fe700 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55de81840d10 0x55de8183d170 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b8c29614fcaede server_cookie=98f3ead507af4901 in_seq=0 out_seq=0 2023-09-12T16:02:12.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8c2ffd700 10 client.14910 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:02:12.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8c2ffd700 1 -- 172.21.15.130:0/2323080455 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55de817198d0 con 0x55de81840d10 2023-09-12T16:02:12.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8c2ffd700 10 client.14910 handle_command_reply: tid=0 2023-09-12T16:02:12.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.247+0000 7fc8f5f3ab80 1 client.14910 shutdown 2023-09-12T16:02:12.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2323080455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 msgr2=0x55de8183b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de8183b450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc007b20 tx=0x7fc8dc007ee0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2323080455 shutdown_connections 2023-09-12T16:02:12.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55de81840d10 0x55de8183d170 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55de817d0550 0x55de8183b450 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de817cedb0 0x55de817da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8c2ffd700 0 client.14910 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:02:12.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 --2- 172.21.15.130:0/2323080455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de817cfbb0 0x55de817dac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2323080455 >> 172.21.15.130:0/2323080455 conn(0x55de81743040 msgr2=0x55de817d6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:12.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2323080455 shutdown_connections 2023-09-12T16:02:12.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 1 -- 172.21.15.130:0/2323080455 wait complete. 2023-09-12T16:02:12.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8f5f3ab80 20 client.14910 trim_cache size 0 max 16384 2023-09-12T16:02:12.275 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:02:12.275 INFO:teuthology.orchestra.run.smithi130.stdout: "cluster_fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T16:02:12.275 INFO:teuthology.orchestra.run.smithi130.stdout: "whoami": 0, 2023-09-12T16:02:12.275 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 24593, 2023-09-12T16:02:12.276 INFO:teuthology.orchestra.run.smithi130.stdout: "want_state": "up:active", 2023-09-12T16:02:12.276 INFO:teuthology.orchestra.run.smithi130.stdout: "state": "up:active", 2023-09-12T16:02:12.276 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs", 2023-09-12T16:02:12.276 INFO:teuthology.orchestra.run.smithi130.stdout: "rank_uptime": 7.2220748160000001, 2023-09-12T16:02:12.276 INFO:teuthology.orchestra.run.smithi130.stdout: "mdsmap_epoch": 17, 2023-09-12T16:02:12.277 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch": 86, 2023-09-12T16:02:12.277 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch_barrier": 85, 2023-09-12T16:02:12.277 INFO:teuthology.orchestra.run.smithi130.stdout: "uptime": 12.691424789999999 2023-09-12T16:02:12.277 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:02:12.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 -- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8cc0a0640 msgr2=0x7fc8cc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8cc0a0640 0x7fc8cc0a2b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005950 tx=0x7fc8d800b410 comp rx=0 tx=0).stop 2023-09-12T16:02:12.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 -- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 msgr2=0x7fc8e4196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:12.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 0x7fc8e4196820 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4000c00 tx=0x7fc8d400f710 comp rx=0 tx=0).stop 2023-09-12T16:02:12.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 -- 172.21.15.130:0/4124001729 shutdown_connections 2023-09-12T16:02:12.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8cc0a0640 0x7fc8cc0a2b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e4106330 0x7fc8e419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e41010d0 0x7fc8e41962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 --2- 172.21.15.130:0/4124001729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8e4103a00 0x7fc8e4196820 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:12.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 -- 172.21.15.130:0/4124001729 >> 172.21.15.130:0/4124001729 conn(0x7fc8e40faa60 msgr2=0x7fc8e4101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:12.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 -- 172.21.15.130:0/4124001729 shutdown_connections 2023-09-12T16:02:12.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:12.248+0000 7fc8e9d74700 1 -- 172.21.15.130:0/4124001729 wait complete. 2023-09-12T16:02:12.281 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-12T16:02:12.282 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-12T16:02:12.282 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph fs ls 2023-09-12T16:02:12.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:12 smithi130 ceph-mon[98670]: osdmap e86: 12 total, 12 up, 12 in 2023-09-12T16:02:12.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:12 smithi130 ceph-mon[98670]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-12T16:02:12.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:12 smithi130 ceph-mon[98670]: Cluster is now healthy 2023-09-12T16:02:12.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:12 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/517918706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:12.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:12 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3271731650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:12.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:12 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2286652670' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:12.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:12 smithi159 ceph-mon[102522]: osdmap e86: 12 total, 12 up, 12 in 2023-09-12T16:02:12.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:12 smithi159 ceph-mon[102522]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-12T16:02:12.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:12 smithi159 ceph-mon[102522]: Cluster is now healthy 2023-09-12T16:02:12.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:12 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/517918706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:12.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:12 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3271731650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:12.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:12 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2286652670' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:12.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:12 smithi203 ceph-mon[103431]: osdmap e86: 12 total, 12 up, 12 in 2023-09-12T16:02:12.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:12 smithi203 ceph-mon[103431]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-12T16:02:12.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:12 smithi203 ceph-mon[103431]: Cluster is now healthy 2023-09-12T16:02:12.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:12 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/517918706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:12.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:12 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3271731650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:12.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:12 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2286652670' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:12.835 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:13.724 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:13 smithi130 ceph-mon[98670]: pgmap v267: 129 pgs: 121 active+clean, 1 creating+activating, 7 creating+peering; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:13.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:13 smithi159 ceph-mon[102522]: pgmap v267: 129 pgs: 121 active+clean, 1 creating+activating, 7 creating+peering; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:13.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:13 smithi203 ceph-mon[103431]: pgmap v267: 129 pgs: 121 active+clean, 1 creating+activating, 7 creating+peering; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:14.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 -- 172.21.15.130:0/1215694008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 msgr2=0x7f0a58106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:14.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 --2- 172.21.15.130:0/1215694008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 0x7f0a58106b40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009990 tx=0x7f0a54009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 -- 172.21.15.130:0/1215694008 shutdown_connections 2023-09-12T16:02:14.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 --2- 172.21.15.130:0/1215694008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a58101250 0x7f0a5810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 --2- 172.21.15.130:0/1215694008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 0x7f0a58106b40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 --2- 172.21.15.130:0/1215694008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a58100890 0x7f0a58100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 -- 172.21.15.130:0/1215694008 >> 172.21.15.130:0/1215694008 conn(0x7f0a58078620 msgr2=0x7f0a58078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:14.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 -- 172.21.15.130:0/1215694008 shutdown_connections 2023-09-12T16:02:14.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 -- 172.21.15.130:0/1215694008 wait complete. 2023-09-12T16:02:14.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 Processor -- start 2023-09-12T16:02:14.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.029+0000 7f0a6015b700 1 -- start start 2023-09-12T16:02:14.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a6015b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a58100890 0x7f0a5819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:14.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a6015b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a58101250 0x7f0a5819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:14.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a6015b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 0x7f0a581965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:14.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a6015b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a58196c60 con 0x7f0a58106760 2023-09-12T16:02:14.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a6015b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a58196dd0 con 0x7f0a58100890 2023-09-12T16:02:14.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a6015b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a58196f90 con 0x7f0a58101250 2023-09-12T16:02:14.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5e6f8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 0x7f0a581965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:14.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5e6f8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 0x7f0a581965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53446/0 (socket says 172.21.15.130:53446) 2023-09-12T16:02:14.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5e6f8700 1 -- 172.21.15.130:0/2253479322 learned_addr learned my addr 172.21.15.130:0/2253479322 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:14.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5d6f6700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a58101250 0x7f0a5819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:14.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5def7700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a58100890 0x7f0a5819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:14.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5e6f8700 1 -- 172.21.15.130:0/2253479322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a58101250 msgr2=0x7f0a5819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:14.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5e6f8700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a58101250 0x7f0a5819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5e6f8700 1 -- 172.21.15.130:0/2253479322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a58100890 msgr2=0x7f0a5819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:14.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5e6f8700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a58100890 0x7f0a5819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5e6f8700 1 -- 172.21.15.130:0/2253479322 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a540095f0 con 0x7f0a58106760 2023-09-12T16:02:14.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5def7700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a58100890 0x7f0a5819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:14.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a5e6f8700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 0x7f0a581965f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0a50007ee0 tx=0x7f0a5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:14.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a4effd700 1 -- 172.21.15.130:0/2253479322 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50017070 con 0x7f0a58106760 2023-09-12T16:02:14.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a4effd700 1 -- 172.21.15.130:0/2253479322 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a5000f040 con 0x7f0a58106760 2023-09-12T16:02:14.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a58197270 con 0x7f0a58106760 2023-09-12T16:02:14.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.030+0000 7f0a4effd700 1 -- 172.21.15.130:0/2253479322 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50005740 con 0x7f0a58106760 2023-09-12T16:02:14.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.031+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a581a3560 con 0x7f0a58106760 2023-09-12T16:02:14.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.031+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5804e300 con 0x7f0a58106760 2023-09-12T16:02:14.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.033+0000 7f0a4effd700 1 -- 172.21.15.130:0/2253479322 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f0a5000dc60 con 0x7f0a58106760 2023-09-12T16:02:14.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.033+0000 7f0a4effd700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a440a0640 0x7f0a440a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:14.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.033+0000 7f0a5def7700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a440a0640 0x7f0a440a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:14.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.034+0000 7f0a4effd700 1 -- 172.21.15.130:0/2253479322 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0a500cb6e0 con 0x7f0a58106760 2023-09-12T16:02:14.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.034+0000 7f0a5def7700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a440a0640 0x7f0a440a2b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400b5c0 tx=0x7f0a54005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:14.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.036+0000 7f0a4effd700 1 -- 172.21.15.130:0/2253479322 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0a50094fc0 con 0x7f0a58106760 2023-09-12T16:02:14.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.266+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a58197b00 con 0x7f0a58106760 2023-09-12T16:02:14.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.267+0000 7f0a4effd700 1 -- 172.21.15.130:0/2253479322 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0a50021070 con 0x7f0a58106760 2023-09-12T16:02:14.465 INFO:teuthology.orchestra.run.smithi130.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-12T16:02:14.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a440a0640 msgr2=0x7f0a440a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:14.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a440a0640 0x7f0a440a2b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400b5c0 tx=0x7f0a54005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 msgr2=0x7f0a581965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:14.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 0x7f0a581965f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0a50007ee0 tx=0x7f0a5000d3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 shutdown_connections 2023-09-12T16:02:14.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a440a0640 0x7f0a440a2b00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a58106760 0x7f0a581965f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a58100890 0x7f0a5819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 --2- 172.21.15.130:0/2253479322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a58101250 0x7f0a5819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:14.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 >> 172.21.15.130:0/2253479322 conn(0x7f0a58078620 msgr2=0x7f0a58104380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:14.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 shutdown_connections 2023-09-12T16:02:14.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:14.269+0000 7f0a6015b700 1 -- 172.21.15.130:0/2253479322 wait complete. 2023-09-12T16:02:14.600 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:14 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2253479322' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-12T16:02:14.690 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-09-12T16:02:14.691 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-12T16:02:14.692 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi130.front.sepia.ceph.com 2023-09-12T16:02:14.692 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-09-12T16:02:14.693 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-12T16:02:14.693 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-09-12T16:02:14.694 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-12T16:02:14.694 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-12T16:02:14.695 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-09-12T16:02:14.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:14.696 DEBUG:teuthology.orchestra.run.smithi130:> ip addr 2023-09-12T16:02:14.717 INFO:teuthology.orchestra.run.smithi130.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T16:02:14.717 INFO:teuthology.orchestra.run.smithi130.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T16:02:14.717 INFO:teuthology.orchestra.run.smithi130.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T16:02:14.717 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft forever preferred_lft forever 2023-09-12T16:02:14.718 INFO:teuthology.orchestra.run.smithi130.stdout: inet6 ::1/128 scope host 2023-09-12T16:02:14.718 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft forever preferred_lft forever 2023-09-12T16:02:14.718 INFO:teuthology.orchestra.run.smithi130.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-12T16:02:14.718 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:d9:37:96 brd ff:ff:ff:ff:ff:ff 2023-09-12T16:02:14.718 INFO:teuthology.orchestra.run.smithi130.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-12T16:02:14.719 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:d9:37:97 brd ff:ff:ff:ff:ff:ff 2023-09-12T16:02:14.719 INFO:teuthology.orchestra.run.smithi130.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-12T16:02:14.719 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:88:7f:36 brd ff:ff:ff:ff:ff:ff 2023-09-12T16:02:14.719 INFO:teuthology.orchestra.run.smithi130.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-12T16:02:14.719 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:88:7f:37 brd ff:ff:ff:ff:ff:ff 2023-09-12T16:02:14.720 INFO:teuthology.orchestra.run.smithi130.stdout: inet 172.21.15.130/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-12T16:02:14.720 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft 41945sec preferred_lft 41945sec 2023-09-12T16:02:14.720 INFO:teuthology.orchestra.run.smithi130.stdout: inet6 fe80::13c4:484c:d749:6616/64 scope link noprefixroute 2023-09-12T16:02:14.720 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft forever preferred_lft forever 2023-09-12T16:02:14.721 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-12T16:02:14.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:14.722 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:02:14.722 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:02:14.722 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link add name ceph-brx type bridge 2023-09-12T16:02:14.723 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip addr flush dev ceph-brx 2023-09-12T16:02:14.725 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link set ceph-brx up 2023-09-12T16:02:14.726 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-12T16:02:14.726 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:02:14.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:14 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2253479322' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-12T16:02:14.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:14 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2253479322' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-12T16:02:14.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:14.903 DEBUG:teuthology.orchestra.run.smithi130:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-12T16:02:14.932 INFO:teuthology.orchestra.run.smithi130.stdout:1 2023-09-12T16:02:14.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:14.934 DEBUG:teuthology.orchestra.run.smithi130:> route 2023-09-12T16:02:15.009 INFO:teuthology.orchestra.run.smithi130.stdout:Kernel IP routing table 2023-09-12T16:02:15.010 INFO:teuthology.orchestra.run.smithi130.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-12T16:02:15.010 INFO:teuthology.orchestra.run.smithi130.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-12T16:02:15.011 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-12T16:02:15.011 INFO:teuthology.orchestra.run.smithi130.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-12T16:02:15.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:15.013 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:02:15.013 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:02:15.014 DEBUG:teuthology.orchestra.run.smithi130:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-12T16:02:15.014 DEBUG:teuthology.orchestra.run.smithi130:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-12T16:02:15.015 DEBUG:teuthology.orchestra.run.smithi130:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-12T16:02:15.016 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:02:15.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:15.280 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list 2023-09-12T16:02:15.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:15.298 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list-id 2023-09-12T16:02:15.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:15.355 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:02:15.355 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:02:15.356 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-12T16:02:15.356 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-09-12T16:02:15.356 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:02:15.449 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-09-12T16:02:15.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:15.450 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:02:15.450 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:02:15.451 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-09-12T16:02:15.451 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-12T16:02:15.451 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-09-12T16:02:15.452 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-09-12T16:02:15.452 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-09-12T16:02:15.452 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:02:15.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:15 smithi130 ceph-mon[98670]: pgmap v268: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:15.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:15.652 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:02:15.653 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:02:15.653 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link set brx.0 up 2023-09-12T16:02:15.653 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link set dev brx.0 master ceph-brx 2023-09-12T16:02:15.654 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:02:15.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:15 smithi159 ceph-mon[102522]: pgmap v268: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:15.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:15 smithi203 ceph-mon[103431]: pgmap v268: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:15.789 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-09-12T16:02:15.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T16:02:15.790 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-09-12T16:02:15.807 INFO:teuthology.orchestra.run.smithi130.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-09-12T16:02:15.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T16:02:15.808 DEBUG:teuthology.orchestra.run.smithi130:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-09-12T16:02:15.863 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2023-09-12T16:02:15.931 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T16:02:15.932 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T16:02:15.997 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T16:02:15.998 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T16:02:15.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:15.999 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2023-09-12T16:02:16.056 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-12T16:02:16.056 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-09-12T16:02:16.098 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2023-09-12T16:02:16.124 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T16:02:16.125 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T16:02:16.191 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T16:02:16.192 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T16:02:16.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:16.193 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2023-09-12T16:02:16.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi130.stderr:2023-09-12T16:02:16.215+0000 7fd20ee6f5c0 -1 init, newargv = 0x55c48fd1f210 newargc=15 2023-09-12T16:02:16.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi130.stderr:ceph-fuse[156795]: starting ceph client 2023-09-12T16:02:16.222 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi130.stderr:ceph-fuse[156795]: starting fuse 2023-09-12T16:02:16.233 INFO:teuthology.orchestra.run.smithi130.stdout:124 2023-09-12T16:02:16.234 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-09-12T16:02:16.234 DEBUG:teuthology.orchestra.run.smithi130:> sudo stdin-killer -- python3 -c ' 2023-09-12T16:02:16.235 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2023-09-12T16:02:16.235 DEBUG:teuthology.orchestra.run.smithi130:> import re 2023-09-12T16:02:16.235 DEBUG:teuthology.orchestra.run.smithi130:> import os 2023-09-12T16:02:16.236 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2023-09-12T16:02:16.236 DEBUG:teuthology.orchestra.run.smithi130:> 2023-09-12T16:02:16.236 DEBUG:teuthology.orchestra.run.smithi130:> def _find_admin_socket(client_name): 2023-09-12T16:02:16.237 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-09-12T16:02:16.237 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2023-09-12T16:02:16.237 DEBUG:teuthology.orchestra.run.smithi130:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-09-12T16:02:16.238 DEBUG:teuthology.orchestra.run.smithi130:> 2023-09-12T16:02:16.238 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2023-09-12T16:02:16.238 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2023-09-12T16:02:16.239 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2023-09-12T16:02:16.239 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2023-09-12T16:02:16.239 DEBUG:teuthology.orchestra.run.smithi130:> 2023-09-12T16:02:16.240 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2023-09-12T16:02:16.240 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-12T16:02:16.240 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-12T16:02:16.241 DEBUG:teuthology.orchestra.run.smithi130:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-12T16:02:16.241 DEBUG:teuthology.orchestra.run.smithi130:> contents = proc_f.read() 2023-09-12T16:02:16.241 DEBUG:teuthology.orchestra.run.smithi130:> if mountpoint in contents: 2023-09-12T16:02:16.242 DEBUG:teuthology.orchestra.run.smithi130:> return f 2023-09-12T16:02:16.242 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-12T16:02:16.242 DEBUG:teuthology.orchestra.run.smithi130:> 2023-09-12T16:02:16.243 DEBUG:teuthology.orchestra.run.smithi130:> print(_find_admin_socket("client.admin")) 2023-09-12T16:02:16.243 DEBUG:teuthology.orchestra.run.smithi130:> ' 2023-09-12T16:02:16.356 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.admin.156795.asok 2023-09-12T16:02:16.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-12T16:02:16.366 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.156795.asok 2023-09-12T16:02:16.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:16.367 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.156795.asok status 2023-09-12T16:02:16.523 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:02:16.523 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2023-09-12T16:02:16.523 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "2110e00747b31a0d73768c9e5229da18b26b8aa0", 2023-09-12T16:02:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T16:02:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "admin", 2023-09-12T16:02:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2023-09-12T16:02:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-09-12T16:02:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "156795", 2023-09-12T16:02:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2023-09-12T16:02:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:02:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2023-09-12T16:02:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2023-09-12T16:02:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 14922, 2023-09-12T16:02:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2023-09-12T16:02:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2023-09-12T16:02:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2023-09-12T16:02:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 14922 2023-09-12T16:02:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:02:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2023-09-12T16:02:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2023-09-12T16:02:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "192.168.0.1:0", 2023-09-12T16:02:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1922670139 2023-09-12T16:02:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:02:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:02:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2023-09-12T16:02:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2023-09-12T16:02:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "192.168.0.1:0", 2023-09-12T16:02:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1922670139 2023-09-12T16:02:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:02:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.14922 192.168.0.1:0/1922670139", 2023-09-12T16:02:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "192.168.0.1:0/1922670139", 2023-09-12T16:02:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2023-09-12T16:02:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 17, 2023-09-12T16:02:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 86, 2023-09-12T16:02:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2023-09-12T16:02:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "blocklisted": false, 2023-09-12T16:02:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2023-09-12T16:02:16.529 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:02:16.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:16.533 DEBUG:teuthology.orchestra.run.smithi130:> stat --file-system '--printf=%T 2023-09-12T16:02:16.534 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-09-12T16:02:16.551 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2023-09-12T16:02:16.551 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-09-12T16:02:16.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:16.552 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-09-12T16:02:16.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:16.620 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-09-12T16:02:16.721 INFO:teuthology.orchestra.run.smithi130.stdout:# file: . 2023-09-12T16:02:16.722 INFO:teuthology.orchestra.run.smithi130.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-09-12T16:02:16.722 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:16.722 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T16:02:16.723 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T16:02:16.741 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:02:16.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 -- 172.21.15.130:0/3282898684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 msgr2=0x7fa9b40ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:16.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 --2- 172.21.15.130:0/3282898684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 0x7fa9b40ff900 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40099c0 tx=0x7fa9a4009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:16.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 -- 172.21.15.130:0/3282898684 shutdown_connections 2023-09-12T16:02:16.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 --2- 172.21.15.130:0/3282898684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9b4101070 0x7fa9b4107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:16.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 --2- 172.21.15.130:0/3282898684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 0x7fa9b40ff900 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:16.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 --2- 172.21.15.130:0/3282898684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b41006d0 0x7fa9b4100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:16.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 -- 172.21.15.130:0/3282898684 >> 172.21.15.130:0/3282898684 conn(0x7fa9b40faa30 msgr2=0x7fa9b40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:16.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 -- 172.21.15.130:0/3282898684 shutdown_connections 2023-09-12T16:02:16.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 -- 172.21.15.130:0/3282898684 wait complete. 2023-09-12T16:02:16.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.904+0000 7fa9bb804700 1 Processor -- start 2023-09-12T16:02:16.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9bb804700 1 -- start start 2023-09-12T16:02:16.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9bb804700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 0x7fa9b4103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:16.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9bb804700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9b41006d0 0x7fa9b4101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:16.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9bb804700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4101070 0x7fa9b4101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:16.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9bb804700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4102790 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9bb804700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4102900 con 0x7fa9b4101070 2023-09-12T16:02:16.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9bb804700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41a02f0 con 0x7fa9b41006d0 2023-09-12T16:02:16.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b95a0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 0x7fa9b4103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:16.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b95a0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 0x7fa9b4103470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53468/0 (socket says 172.21.15.130:53468) 2023-09-12T16:02:16.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b95a0700 1 -- 172.21.15.130:0/3892588982 learned_addr learned my addr 172.21.15.130:0/3892588982 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:16.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b9da1700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4101070 0x7fa9b4101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:16.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b8d9f700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9b41006d0 0x7fa9b4101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:16.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b95a0700 1 -- 172.21.15.130:0/3892588982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9b41006d0 msgr2=0x7fa9b4101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:16.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b95a0700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9b41006d0 0x7fa9b4101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:16.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b95a0700 1 -- 172.21.15.130:0/3892588982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4101070 msgr2=0x7fa9b4101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:16.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b95a0700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4101070 0x7fa9b4101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:16.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b95a0700 1 -- 172.21.15.130:0/3892588982 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a40095f0 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9b95a0700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 0x7fa9b4103470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009990 tx=0x7fa9a4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:16.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9aa7fc700 1 -- 172.21.15.130:0/3892588982 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a401d070 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9aa7fc700 1 -- 172.21.15.130:0/3892588982 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a4004200 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9bb804700 1 -- 172.21.15.130:0/3892588982 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b41a0570 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.905+0000 7fa9aa7fc700 1 -- 172.21.15.130:0/3892588982 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4022e30 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.906+0000 7fa9bb804700 1 -- 172.21.15.130:0/3892588982 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b41a0a20 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.906+0000 7fa99ffff700 1 -- 172.21.15.130:0/3892588982 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9940052f0 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.907+0000 7fa9aa7fc700 1 -- 172.21.15.130:0/3892588982 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fa9a4003d40 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.908+0000 7fa9aa7fc700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa9a00a0690 0x7fa9a00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:16.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.908+0000 7fa9aa7fc700 1 -- 172.21.15.130:0/3892588982 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa9a40cd050 con 0x7fa9b40ff4e0 2023-09-12T16:02:16.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.908+0000 7fa9b8d9f700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa9a00a0690 0x7fa9a00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:16.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.908+0000 7fa9b8d9f700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa9a00a0690 0x7fa9a00a2b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005fd0 tx=0x7fa9b000a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:16.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:16.912+0000 7fa9aa7fc700 1 -- 172.21.15.130:0/3892588982 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa9a4096a20 con 0x7fa9b40ff4e0 2023-09-12T16:02:17.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.108+0000 7fa99ffff700 1 -- 172.21.15.130:0/3892588982 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa994005160 con 0x7fa9b40ff4e0 2023-09-12T16:02:17.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.109+0000 7fa9aa7fc700 1 -- 172.21.15.130:0/3892588982 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24004 (secure 0 0 0) 0x7fa9a409a040 con 0x7fa9b40ff4e0 2023-09-12T16:02:17.110 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:17.111 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":86,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:11.262527+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:17.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 -- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa9a00a0690 msgr2=0x7fa9a00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa9a00a0690 0x7fa9a00a2b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005fd0 tx=0x7fa9b000a560 comp rx=0 tx=0).stop 2023-09-12T16:02:17.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 -- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 msgr2=0x7fa9b4103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 0x7fa9b4103470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009990 tx=0x7fa9a4005e80 comp rx=0 tx=0).stop 2023-09-12T16:02:17.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 -- 172.21.15.130:0/3892588982 shutdown_connections 2023-09-12T16:02:17.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa9a00a0690 0x7fa9a00a2b50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4101070 0x7fa9b4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9b40ff4e0 0x7fa9b4103470 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 --2- 172.21.15.130:0/3892588982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9b41006d0 0x7fa9b4101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 -- 172.21.15.130:0/3892588982 >> 172.21.15.130:0/3892588982 conn(0x7fa9b40faa30 msgr2=0x7fa9b40677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:17.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 -- 172.21.15.130:0/3892588982 shutdown_connections 2023-09-12T16:02:17.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.112+0000 7fa99ffff700 1 -- 172.21.15.130:0/3892588982 wait complete. 2023-09-12T16:02:17.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:02:17.126 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:02:17.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.249+0000 7fe65cd83700 1 -- 172.21.15.130:0/3268938704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe658101800 msgr2=0x7fe658101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.249+0000 7fe65cd83700 1 --2- 172.21.15.130:0/3268938704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe658101800 0x7fe658101c60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe6480099c0 tx=0x7fe648009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.249+0000 7fe65cd83700 1 -- 172.21.15.130:0/3268938704 shutdown_connections 2023-09-12T16:02:17.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.249+0000 7fe65cd83700 1 --2- 172.21.15.130:0/3268938704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6581021a0 0x7fe658108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.249+0000 7fe65cd83700 1 --2- 172.21.15.130:0/3268938704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe658100610 0x7fe658100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.249+0000 7fe65cd83700 1 --2- 172.21.15.130:0/3268938704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe658101800 0x7fe658101c60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.249+0000 7fe65cd83700 1 -- 172.21.15.130:0/3268938704 >> 172.21.15.130:0/3268938704 conn(0x7fe6580fbb80 msgr2=0x7fe6580fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:17.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.249+0000 7fe65cd83700 1 -- 172.21.15.130:0/3268938704 shutdown_connections 2023-09-12T16:02:17.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.249+0000 7fe65cd83700 1 -- 172.21.15.130:0/3268938704 wait complete. 2023-09-12T16:02:17.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe65cd83700 1 Processor -- start 2023-09-12T16:02:17.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe65cd83700 1 -- start start 2023-09-12T16:02:17.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe65cd83700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe658100610 0x7fe658196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe65cd83700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe658101800 0x7fe658196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe65cd83700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6581021a0 0x7fe65819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe65cd83700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65819de80 con 0x7fe6581021a0 2023-09-12T16:02:17.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe65cd83700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65819dfc0 con 0x7fe658100610 2023-09-12T16:02:17.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe65cd83700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65819e150 con 0x7fe658101800 2023-09-12T16:02:17.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe656d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6581021a0 0x7fe65819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe656d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6581021a0 0x7fe65819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53486/0 (socket says 172.21.15.130:53486) 2023-09-12T16:02:17.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe656d9d700 1 -- 172.21.15.130:0/2492205697 learned_addr learned my addr 172.21.15.130:0/2492205697 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:17.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe65659c700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe658100610 0x7fe658196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.250+0000 7fe655d9b700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe658101800 0x7fe658196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe656d9d700 1 -- 172.21.15.130:0/2492205697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe658101800 msgr2=0x7fe658196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe656d9d700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe658101800 0x7fe658196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe656d9d700 1 -- 172.21.15.130:0/2492205697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe658100610 msgr2=0x7fe658196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe656d9d700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe658100610 0x7fe658196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe656d9d700 1 -- 172.21.15.130:0/2492205697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6480095f0 con 0x7fe6581021a0 2023-09-12T16:02:17.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe656d9d700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6581021a0 0x7fe65819d8b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe650007ee0 tx=0x7fe65000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:17.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe6477fe700 1 -- 172.21.15.130:0/2492205697 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650017070 con 0x7fe6581021a0 2023-09-12T16:02:17.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe6477fe700 1 -- 172.21.15.130:0/2492205697 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe65000f040 con 0x7fe6581021a0 2023-09-12T16:02:17.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65819e430 con 0x7fe6581021a0 2023-09-12T16:02:17.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe6477fe700 1 -- 172.21.15.130:0/2492205697 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650005580 con 0x7fe6581021a0 2023-09-12T16:02:17.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.251+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65819e8f0 con 0x7fe6581021a0 2023-09-12T16:02:17.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.252+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe658107590 con 0x7fe6581021a0 2023-09-12T16:02:17.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.253+0000 7fe6477fe700 1 -- 172.21.15.130:0/2492205697 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fe65000db20 con 0x7fe6581021a0 2023-09-12T16:02:17.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.253+0000 7fe6477fe700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe6400a0640 0x7fe6400a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.254+0000 7fe65659c700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe6400a0640 0x7fe6400a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.254+0000 7fe6477fe700 1 -- 172.21.15.130:0/2492205697 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe6500cb570 con 0x7fe6581021a0 2023-09-12T16:02:17.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.254+0000 7fe65659c700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe6400a0640 0x7fe6400a2b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe658101660 tx=0x7fe64c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:17.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.258+0000 7fe6477fe700 1 -- 172.21.15.130:0/2492205697 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe650094e50 con 0x7fe6581021a0 2023-09-12T16:02:17.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.450+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe65802cea0 con 0x7fe6581021a0 2023-09-12T16:02:17.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.451+0000 7fe6477fe700 1 -- 172.21.15.130:0/2492205697 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe650098470 con 0x7fe6581021a0 2023-09-12T16:02:17.452 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:02:17.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe6400a0640 msgr2=0x7fe6400a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe6400a0640 0x7fe6400a2b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe658101660 tx=0x7fe64c00b410 comp rx=0 tx=0).stop 2023-09-12T16:02:17.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6581021a0 msgr2=0x7fe65819d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6581021a0 0x7fe65819d8b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe650007ee0 tx=0x7fe65000d3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 shutdown_connections 2023-09-12T16:02:17.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe6400a0640 0x7fe6400a2b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6581021a0 0x7fe65819d8b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe658100610 0x7fe658196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 --2- 172.21.15.130:0/2492205697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe658101800 0x7fe658196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 >> 172.21.15.130:0/2492205697 conn(0x7fe6580fbb80 msgr2=0x7fe6580fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:17.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 shutdown_connections 2023-09-12T16:02:17.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.453+0000 7fe65cd83700 1 -- 172.21.15.130:0/2492205697 wait complete. 2023-09-12T16:02:17.465 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:02:17.592 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.589+0000 7fb720095700 1 -- 172.21.15.130:0/775315948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7180ff4f0 msgr2=0x7fb7180ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.589+0000 7fb720095700 1 --2- 172.21.15.130:0/775315948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7180ff4f0 0x7fb7180ff910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb7140099c0 tx=0x7fb714009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.589+0000 7fb720095700 1 -- 172.21.15.130:0/775315948 shutdown_connections 2023-09-12T16:02:17.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.589+0000 7fb720095700 1 --2- 172.21.15.130:0/775315948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb718101080 0x7fb718107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.589+0000 7fb720095700 1 --2- 172.21.15.130:0/775315948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7180ff4f0 0x7fb7180ff910 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.589+0000 7fb720095700 1 --2- 172.21.15.130:0/775315948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7181006e0 0x7fb718100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.589+0000 7fb720095700 1 -- 172.21.15.130:0/775315948 >> 172.21.15.130:0/775315948 conn(0x7fb7180faa60 msgr2=0x7fb7180fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:17.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 -- 172.21.15.130:0/775315948 shutdown_connections 2023-09-12T16:02:17.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 -- 172.21.15.130:0/775315948 wait complete. 2023-09-12T16:02:17.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 Processor -- start 2023-09-12T16:02:17.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 -- start start 2023-09-12T16:02:17.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7180ff4f0 0x7fb718196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7181006e0 0x7fb718196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb718101080 0x7fb71819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71819dea0 con 0x7fb7181006e0 2023-09-12T16:02:17.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71819dfe0 con 0x7fb718101080 2023-09-12T16:02:17.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb720095700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71819e170 con 0x7fb7180ff4f0 2023-09-12T16:02:17.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb71d630700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7181006e0 0x7fb718196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb71d630700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7181006e0 0x7fb718196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53506/0 (socket says 172.21.15.130:53506) 2023-09-12T16:02:17.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.590+0000 7fb71d630700 1 -- 172.21.15.130:0/2390177474 learned_addr learned my addr 172.21.15.130:0/2390177474 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:17.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb71e632700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb718101080 0x7fb71819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb71de31700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7180ff4f0 0x7fb718196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb71d630700 1 -- 172.21.15.130:0/2390177474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7180ff4f0 msgr2=0x7fb718196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb71d630700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7180ff4f0 0x7fb718196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb71d630700 1 -- 172.21.15.130:0/2390177474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb718101080 msgr2=0x7fb71819d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb71d630700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb718101080 0x7fb71819d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb71d630700 1 -- 172.21.15.130:0/2390177474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140095f0 con 0x7fb7181006e0 2023-09-12T16:02:17.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb71de31700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7180ff4f0 0x7fb718196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:17.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb71d630700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7181006e0 0x7fb718196860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c920 tx=0x7fb70800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:17.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb70effd700 1 -- 172.21.15.130:0/2390177474 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708007b00 con 0x7fb7181006e0 2023-09-12T16:02:17.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb70effd700 1 -- 172.21.15.130:0/2390177474 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb70800ce70 con 0x7fb7181006e0 2023-09-12T16:02:17.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71819e450 con 0x7fb7181006e0 2023-09-12T16:02:17.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb70effd700 1 -- 172.21.15.130:0/2390177474 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7080054e0 con 0x7fb7181006e0 2023-09-12T16:02:17.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.591+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71819e960 con 0x7fb7181006e0 2023-09-12T16:02:17.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.592+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71810ad00 con 0x7fb7181006e0 2023-09-12T16:02:17.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.593+0000 7fb70effd700 1 -- 172.21.15.130:0/2390177474 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fb708004750 con 0x7fb7181006e0 2023-09-12T16:02:17.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.594+0000 7fb70effd700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7040a0640 0x7fb7040a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.594+0000 7fb70effd700 1 -- 172.21.15.130:0/2390177474 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb7080cb940 con 0x7fb7181006e0 2023-09-12T16:02:17.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.594+0000 7fb71de31700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7040a0640 0x7fb7040a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.594+0000 7fb71de31700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7040a0640 0x7fb7040a2b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb71400b580 tx=0x7fb714005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:17.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.598+0000 7fb70effd700 1 -- 172.21.15.130:0/2390177474 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb708095280 con 0x7fb7181006e0 2023-09-12T16:02:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:17 smithi130 ceph-mon[98670]: pgmap v269: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:17 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3892588982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:17.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:17 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2492205697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:02:17.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.791+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb71819ebe0 con 0x7fb7181006e0 2023-09-12T16:02:17.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.792+0000 7fb70effd700 1 -- 172.21.15.130:0/2390177474 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb70801d020 con 0x7fb7181006e0 2023-09-12T16:02:17.794 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:02:17.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7040a0640 msgr2=0x7fb7040a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7040a0640 0x7fb7040a2b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb71400b580 tx=0x7fb714005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7181006e0 msgr2=0x7fb718196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7181006e0 0x7fb718196860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c920 tx=0x7fb70800cc30 comp rx=0 tx=0).stop 2023-09-12T16:02:17.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 shutdown_connections 2023-09-12T16:02:17.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7040a0640 0x7fb7040a2b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb718101080 0x7fb71819d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7180ff4f0 0x7fb718196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 --2- 172.21.15.130:0/2390177474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7181006e0 0x7fb718196860 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 >> 172.21.15.130:0/2390177474 conn(0x7fb7180faa60 msgr2=0x7fb7180677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:17.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 shutdown_connections 2023-09-12T16:02:17.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.794+0000 7fb720095700 1 -- 172.21.15.130:0/2390177474 wait complete. 2023-09-12T16:02:17.806 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:02:17.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.931+0000 7fbcfa101700 1 -- 172.21.15.130:0/3207070788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf4100ba0 msgr2=0x7fbcf4100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.931+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3207070788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf4100ba0 0x7fbcf4100fc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009990 tx=0x7fbce8009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.931+0000 7fbcfa101700 1 -- 172.21.15.130:0/3207070788 shutdown_connections 2023-09-12T16:02:17.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.931+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3207070788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcf4100120 0x7fbcf41079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.931+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3207070788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf4100ba0 0x7fbcf4100fc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.931+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3207070788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf40ff760 0x7fbcf40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.931+0000 7fbcfa101700 1 -- 172.21.15.130:0/3207070788 >> 172.21.15.130:0/3207070788 conn(0x7fbcf40fa1f0 msgr2=0x7fbcf40fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:17.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.931+0000 7fbcfa101700 1 -- 172.21.15.130:0/3207070788 shutdown_connections 2023-09-12T16:02:17.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.931+0000 7fbcfa101700 1 -- 172.21.15.130:0/3207070788 wait complete. 2023-09-12T16:02:17.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcfa101700 1 Processor -- start 2023-09-12T16:02:17.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcfa101700 1 -- start start 2023-09-12T16:02:17.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcfa101700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf40ff760 0x7fbcf40ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcfa101700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf4100120 0x7fbcf419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcfa101700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcf4100ba0 0x7fbcf419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcfa101700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf419dd80 con 0x7fbcf40ff760 2023-09-12T16:02:17.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcfa101700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf419def0 con 0x7fbcf4100120 2023-09-12T16:02:17.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcfa101700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf419e0b0 con 0x7fbcf4100ba0 2023-09-12T16:02:17.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf37fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf40ff760 0x7fbcf40ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf37fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf40ff760 0x7fbcf40ff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53518/0 (socket says 172.21.15.130:53518) 2023-09-12T16:02:17.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf37fe700 1 -- 172.21.15.130:0/3769348 learned_addr learned my addr 172.21.15.130:0/3769348 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:17.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf2ffd700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf4100120 0x7fbcf419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf3fff700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcf4100ba0 0x7fbcf419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf37fe700 1 -- 172.21.15.130:0/3769348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcf4100ba0 msgr2=0x7fbcf419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf37fe700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcf4100ba0 0x7fbcf419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf37fe700 1 -- 172.21.15.130:0/3769348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf4100120 msgr2=0x7fbcf419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:17.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf37fe700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf4100120 0x7fbcf419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:17.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.932+0000 7fbcf37fe700 1 -- 172.21.15.130:0/3769348 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce80095f0 con 0x7fbcf40ff760 2023-09-12T16:02:17.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.933+0000 7fbcf37fe700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf40ff760 0x7fbcf40ff1e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbce8000c00 tx=0x7fbce800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:17.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.933+0000 7fbcf0ff9700 1 -- 172.21.15.130:0/3769348 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce801d070 con 0x7fbcf40ff760 2023-09-12T16:02:17.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.933+0000 7fbcf0ff9700 1 -- 172.21.15.130:0/3769348 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbce800fb50 con 0x7fbcf40ff760 2023-09-12T16:02:17.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.933+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf419e330 con 0x7fbcf40ff760 2023-09-12T16:02:17.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.933+0000 7fbcf0ff9700 1 -- 172.21.15.130:0/3769348 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce80173f0 con 0x7fbcf40ff760 2023-09-12T16:02:17.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.933+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf419e7e0 con 0x7fbcf40ff760 2023-09-12T16:02:17.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.934+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf410ad30 con 0x7fbcf40ff760 2023-09-12T16:02:17.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.935+0000 7fbcf0ff9700 1 -- 172.21.15.130:0/3769348 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fbce800fcc0 con 0x7fbcf40ff760 2023-09-12T16:02:17.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.935+0000 7fbcf0ff9700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbcdc0a06c0 0x7fbcdc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:17.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.935+0000 7fbcf0ff9700 1 -- 172.21.15.130:0/3769348 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbce80cd480 con 0x7fbcf40ff760 2023-09-12T16:02:17.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.935+0000 7fbcf2ffd700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbcdc0a06c0 0x7fbcdc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:17.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.936+0000 7fbcf2ffd700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbcdc0a06c0 0x7fbcdc0a2b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40fef50 tx=0x7fbce400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:17.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:17.939+0000 7fbcf0ff9700 1 -- 172.21.15.130:0/3769348 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbce8096df0 con 0x7fbcf40ff760 2023-09-12T16:02:17.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:17 smithi159 ceph-mon[102522]: pgmap v269: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:17.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:17 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3892588982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:17.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:17 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2492205697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:02:17.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:17 smithi203 ceph-mon[103431]: pgmap v269: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:17.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:17 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3892588982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:17.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:17 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2492205697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:02:18.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.128+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbcf41064b0 con 0x7fbcf40ff760 2023-09-12T16:02:18.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.128+0000 7fbcf0ff9700 1 -- 172.21.15.130:0/3769348 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbce809a410 con 0x7fbcf40ff760 2023-09-12T16:02:18.129 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:02:18.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbcdc0a06c0 msgr2=0x7fbcdc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbcdc0a06c0 0x7fbcdc0a2b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40fef50 tx=0x7fbce400b410 comp rx=0 tx=0).stop 2023-09-12T16:02:18.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf40ff760 msgr2=0x7fbcf40ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf40ff760 0x7fbcf40ff1e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbce8000c00 tx=0x7fbce800f710 comp rx=0 tx=0).stop 2023-09-12T16:02:18.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 shutdown_connections 2023-09-12T16:02:18.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbcdc0a06c0 0x7fbcdc0a2b80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcf4100ba0 0x7fbcf419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcf40ff760 0x7fbcf40ff1e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 --2- 172.21.15.130:0/3769348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf4100120 0x7fbcf419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.130+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 >> 172.21.15.130:0/3769348 conn(0x7fbcf40fa1f0 msgr2=0x7fbcf40fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:18.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.131+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 shutdown_connections 2023-09-12T16:02:18.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.131+0000 7fbcfa101700 1 -- 172.21.15.130:0/3769348 wait complete. 2023-09-12T16:02:18.143 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:02:18.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.267+0000 7f9311ee3700 1 -- 172.21.15.130:0/3351754533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c0ff4f0 msgr2=0x7f930c0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.267+0000 7f9311ee3700 1 --2- 172.21.15.130:0/3351754533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c0ff4f0 0x7f930c0ff910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0099c0 tx=0x7f92fc009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.267+0000 7f9311ee3700 1 -- 172.21.15.130:0/3351754533 shutdown_connections 2023-09-12T16:02:18.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.267+0000 7f9311ee3700 1 --2- 172.21.15.130:0/3351754533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f930c101080 0x7f930c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.267+0000 7f9311ee3700 1 --2- 172.21.15.130:0/3351754533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c0ff4f0 0x7f930c0ff910 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.267+0000 7f9311ee3700 1 --2- 172.21.15.130:0/3351754533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f930c1006e0 0x7f930c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.267+0000 7f9311ee3700 1 -- 172.21.15.130:0/3351754533 >> 172.21.15.130:0/3351754533 conn(0x7f930c0faa60 msgr2=0x7f930c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:18.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.267+0000 7f9311ee3700 1 -- 172.21.15.130:0/3351754533 shutdown_connections 2023-09-12T16:02:18.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.267+0000 7f9311ee3700 1 -- 172.21.15.130:0/3351754533 wait complete. 2023-09-12T16:02:18.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.268+0000 7f9311ee3700 1 Processor -- start 2023-09-12T16:02:18.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.268+0000 7f9311ee3700 1 -- start start 2023-09-12T16:02:18.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.268+0000 7f9311ee3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f930c0ff4f0 0x7f930c195190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:18.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.268+0000 7f9311ee3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f930c1006e0 0x7f930c1956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:18.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.268+0000 7f9311ee3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c101080 0x7f930c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:18.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.268+0000 7f9311ee3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c19cd10 con 0x7f930c101080 2023-09-12T16:02:18.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.268+0000 7f9311ee3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c19ce50 con 0x7f930c1006e0 2023-09-12T16:02:18.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.268+0000 7f9311ee3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c19cfe0 con 0x7f930c0ff4f0 2023-09-12T16:02:18.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.268+0000 7f930bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c101080 0x7f930c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:18.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c101080 0x7f930c19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53536/0 (socket says 172.21.15.130:53536) 2023-09-12T16:02:18.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930bfff700 1 -- 172.21.15.130:0/364397414 learned_addr learned my addr 172.21.15.130:0/364397414 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:18.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930b7fe700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f930c0ff4f0 0x7f930c195190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:18.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930affd700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f930c1006e0 0x7f930c1956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:18.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930bfff700 1 -- 172.21.15.130:0/364397414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f930c0ff4f0 msgr2=0x7f930c195190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930bfff700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f930c0ff4f0 0x7f930c195190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930bfff700 1 -- 172.21.15.130:0/364397414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f930c1006e0 msgr2=0x7f930c1956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930bfff700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f930c1006e0 0x7f930c1956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930bfff700 1 -- 172.21.15.130:0/364397414 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc0095f0 con 0x7f930c101080 2023-09-12T16:02:18.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930affd700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f930c1006e0 0x7f930c1956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:18.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f930bfff700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c101080 0x7f930c19c740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9304007d70 tx=0x7f930400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:18.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f9308ff9700 1 -- 172.21.15.130:0/364397414 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930400ad40 con 0x7f930c101080 2023-09-12T16:02:18.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f9308ff9700 1 -- 172.21.15.130:0/364397414 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93040128e0 con 0x7f930c101080 2023-09-12T16:02:18.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f9308ff9700 1 -- 172.21.15.130:0/364397414 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040116b0 con 0x7f930c101080 2023-09-12T16:02:18.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930c19d2c0 con 0x7f930c101080 2023-09-12T16:02:18.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.269+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c19d780 con 0x7f930c101080 2023-09-12T16:02:18.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.270+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930c10ad00 con 0x7f930c101080 2023-09-12T16:02:18.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.272+0000 7f9308ff9700 1 -- 172.21.15.130:0/364397414 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f9304012420 con 0x7f930c101080 2023-09-12T16:02:18.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.272+0000 7f9308ff9700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f92f40a0640 0x7f92f40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:18.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.272+0000 7f930b7fe700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f92f40a0640 0x7f92f40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:18.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.272+0000 7f9308ff9700 1 -- 172.21.15.130:0/364397414 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f93040cc420 con 0x7f930c101080 2023-09-12T16:02:18.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.273+0000 7f930b7fe700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f92f40a0640 0x7f92f40a2b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009990 tx=0x7f92fc0052c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:18.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.276+0000 7f9308ff9700 1 -- 172.21.15.130:0/364397414 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9304095d00 con 0x7f930c101080 2023-09-12T16:02:18.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.466+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f930c1063f0 con 0x7f930c101080 2023-09-12T16:02:18.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.466+0000 7f9308ff9700 1 -- 172.21.15.130:0/364397414 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9304099320 con 0x7f930c101080 2023-09-12T16:02:18.467 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:02:18.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f92f40a0640 msgr2=0x7f92f40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f92f40a0640 0x7f92f40a2b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009990 tx=0x7f92fc0052c0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c101080 msgr2=0x7f930c19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c101080 0x7f930c19c740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9304007d70 tx=0x7f930400a340 comp rx=0 tx=0).stop 2023-09-12T16:02:18.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 shutdown_connections 2023-09-12T16:02:18.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f92f40a0640 0x7f92f40a2b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f930c101080 0x7f930c19c740 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f930c0ff4f0 0x7f930c195190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 --2- 172.21.15.130:0/364397414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f930c1006e0 0x7f930c1956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 >> 172.21.15.130:0/364397414 conn(0x7f930c0faa60 msgr2=0x7f930c0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:18.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 shutdown_connections 2023-09-12T16:02:18.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.469+0000 7f9311ee3700 1 -- 172.21.15.130:0/364397414 wait complete. 2023-09-12T16:02:18.482 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T16:02:18.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.604+0000 7feaa283f700 1 -- 172.21.15.130:0/2555431886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 msgr2=0x7fea9c108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.604+0000 7feaa283f700 1 --2- 172.21.15.130:0/2555431886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 0x7fea9c108bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea9400b0a0 tx=0x7fea9400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.604+0000 7feaa283f700 1 -- 172.21.15.130:0/2555431886 shutdown_connections 2023-09-12T16:02:18.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.604+0000 7feaa283f700 1 --2- 172.21.15.130:0/2555431886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 0x7fea9c108bc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.604+0000 7feaa283f700 1 --2- 172.21.15.130:0/2555431886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea9c0fe900 0x7fea9c0fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.604+0000 7feaa283f700 1 --2- 172.21.15.130:0/2555431886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9c101970 0x7fea9c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.604+0000 7feaa283f700 1 -- 172.21.15.130:0/2555431886 >> 172.21.15.130:0/2555431886 conn(0x7fea9c0fa1f0 msgr2=0x7fea9c0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:18.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.604+0000 7feaa283f700 1 -- 172.21.15.130:0/2555431886 shutdown_connections 2023-09-12T16:02:18.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.604+0000 7feaa283f700 1 -- 172.21.15.130:0/2555431886 wait complete. 2023-09-12T16:02:18.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa283f700 1 Processor -- start 2023-09-12T16:02:18.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa283f700 1 -- start start 2023-09-12T16:02:18.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa283f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea9c0fe900 0x7fea9c1012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:18.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa283f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9c101970 0x7fea9c0ff920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:18.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa283f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 0x7fea9c0ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:18.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa283f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c100600 con 0x7fea9c102310 2023-09-12T16:02:18.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa283f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c100770 con 0x7fea9c0fe900 2023-09-12T16:02:18.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa283f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c1a0430 con 0x7fea9c101970 2023-09-12T16:02:18.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa0ddc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 0x7fea9c0ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:18.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa0ddc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 0x7fea9c0ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53550/0 (socket says 172.21.15.130:53550) 2023-09-12T16:02:18.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7feaa0ddc700 1 -- 172.21.15.130:0/1124527326 learned_addr learned my addr 172.21.15.130:0/1124527326 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:18.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.605+0000 7fea9bfff700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea9c0fe900 0x7fea9c1012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:18.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.606+0000 7fea9b7fe700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9c101970 0x7fea9c0ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:18.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.606+0000 7feaa0ddc700 1 -- 172.21.15.130:0/1124527326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9c101970 msgr2=0x7fea9c0ff920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.606+0000 7feaa0ddc700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9c101970 0x7fea9c0ff920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.606+0000 7feaa0ddc700 1 -- 172.21.15.130:0/1124527326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea9c0fe900 msgr2=0x7fea9c1012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.606+0000 7feaa0ddc700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea9c0fe900 0x7fea9c1012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.606+0000 7feaa0ddc700 1 -- 172.21.15.130:0/1124527326 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94009d00 con 0x7fea9c102310 2023-09-12T16:02:18.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.606+0000 7feaa0ddc700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 0x7fea9c0ffe60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fea940062a0 tx=0x7fea94004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:18.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.607+0000 7fea997fa700 1 -- 172.21.15.130:0/1124527326 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9400e040 con 0x7fea9c102310 2023-09-12T16:02:18.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.607+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c1a06b0 con 0x7fea9c102310 2023-09-12T16:02:18.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.607+0000 7fea997fa700 1 -- 172.21.15.130:0/1124527326 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea9400cec0 con 0x7fea9c102310 2023-09-12T16:02:18.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.607+0000 7fea997fa700 1 -- 172.21.15.130:0/1124527326 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9401baf0 con 0x7fea9c102310 2023-09-12T16:02:18.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.607+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c1a0b30 con 0x7fea9c102310 2023-09-12T16:02:18.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.608+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c107cc0 con 0x7fea9c102310 2023-09-12T16:02:18.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.609+0000 7fea997fa700 1 -- 172.21.15.130:0/1124527326 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fea9401bc50 con 0x7fea9c102310 2023-09-12T16:02:18.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.610+0000 7fea997fa700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fea840a0640 0x7fea840a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:18.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.610+0000 7fea9bfff700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fea840a0640 0x7fea840a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:18.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.610+0000 7fea997fa700 1 -- 172.21.15.130:0/1124527326 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fea940ce0d0 con 0x7fea9c102310 2023-09-12T16:02:18.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.610+0000 7fea9bfff700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fea840a0640 0x7fea840a2b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fea9c068690 tx=0x7fea8c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:18.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.613+0000 7fea997fa700 1 -- 172.21.15.130:0/1124527326 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fea94097a40 con 0x7fea9c102310 2023-09-12T16:02:18.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.823+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fea9c10c070 con 0x7fea9c102310 2023-09-12T16:02:18.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.823+0000 7fea997fa700 1 -- 172.21.15.130:0/1124527326 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+598 (secure 0 0 0) 0x7fea9409b060 con 0x7fea9c102310 2023-09-12T16:02:18.824 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/1706265745 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:18.824 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6800/3042728256 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:18.825 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/3742416190 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:18.825 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/2502139426 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:18.825 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6801/3042728256 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:18.825 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6800/840569029 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:18.825 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6801/840569029 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:18.826 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/410531865 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:18.826 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/288931411 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:18.826 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/1303099884 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:18.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fea840a0640 msgr2=0x7fea840a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fea840a0640 0x7fea840a2b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fea9c068690 tx=0x7fea8c00a380 comp rx=0 tx=0).stop 2023-09-12T16:02:18.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 msgr2=0x7fea9c0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:18.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 0x7fea9c0ffe60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fea940062a0 tx=0x7fea94004d00 comp rx=0 tx=0).stop 2023-09-12T16:02:18.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 shutdown_connections 2023-09-12T16:02:18.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fea840a0640 0x7fea840a2b00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea9c102310 0x7fea9c0ffe60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea9c0fe900 0x7fea9c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 --2- 172.21.15.130:0/1124527326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9c101970 0x7fea9c0ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:18.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 >> 172.21.15.130:0/1124527326 conn(0x7fea9c0fa1f0 msgr2=0x7fea9c104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:18.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 shutdown_connections 2023-09-12T16:02:18.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:18.825+0000 7feaa283f700 1 -- 172.21.15.130:0/1124527326 wait complete. 2023-09-12T16:02:18.830 INFO:teuthology.orchestra.run.smithi130.stderr:listed 10 entries 2023-09-12T16:02:18.837 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:18 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2390177474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:02:18.837 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:18 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3769348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:02:18.837 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:18 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/364397414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:02:18.838 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T16:02:18.839 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T16:02:18.857 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:02:18.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:18 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2390177474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:02:18.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:18 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3769348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:02:18.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:18 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/364397414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:02:18.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:18 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2390177474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:02:18.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:18 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3769348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:02:18.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:18 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/364397414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:02:19.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.017+0000 7fbe6e391700 1 -- 172.21.15.130:0/1074184139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 msgr2=0x7fbe68100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.017+0000 7fbe6e391700 1 --2- 172.21.15.130:0/1074184139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 0x7fbe68100f90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009990 tx=0x7fbe58009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.017+0000 7fbe6e391700 1 -- 172.21.15.130:0/1074184139 shutdown_connections 2023-09-12T16:02:19.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.017+0000 7fbe6e391700 1 --2- 172.21.15.130:0/1074184139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe681000f0 0x7fbe68107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.017+0000 7fbe6e391700 1 --2- 172.21.15.130:0/1074184139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 0x7fbe68100f90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.017+0000 7fbe6e391700 1 --2- 172.21.15.130:0/1074184139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe680ff730 0x7fbe680ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.017+0000 7fbe6e391700 1 -- 172.21.15.130:0/1074184139 >> 172.21.15.130:0/1074184139 conn(0x7fbe680fa1c0 msgr2=0x7fbe680fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:19.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.017+0000 7fbe6e391700 1 -- 172.21.15.130:0/1074184139 shutdown_connections 2023-09-12T16:02:19.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.017+0000 7fbe6e391700 1 -- 172.21.15.130:0/1074184139 wait complete. 2023-09-12T16:02:19.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6e391700 1 Processor -- start 2023-09-12T16:02:19.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6e391700 1 -- start start 2023-09-12T16:02:19.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6e391700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe680ff730 0x7fbe680ff190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6e391700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe681000f0 0x7fbe6819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6e391700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 0x7fbe6819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6e391700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819dda0 con 0x7fbe68100b70 2023-09-12T16:02:19.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6e391700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819df10 con 0x7fbe680ff730 2023-09-12T16:02:19.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6e391700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819e0d0 con 0x7fbe681000f0 2023-09-12T16:02:19.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6c92e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 0x7fbe6819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6c92e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 0x7fbe6819d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53562/0 (socket says 172.21.15.130:53562) 2023-09-12T16:02:19.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.018+0000 7fbe6c92e700 1 -- 172.21.15.130:0/4106541562 learned_addr learned my addr 172.21.15.130:0/4106541562 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:19.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe677fe700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe681000f0 0x7fbe6819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe67fff700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe680ff730 0x7fbe680ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe6c92e700 1 -- 172.21.15.130:0/4106541562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe681000f0 msgr2=0x7fbe6819afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe6c92e700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe681000f0 0x7fbe6819afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe6c92e700 1 -- 172.21.15.130:0/4106541562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe680ff730 msgr2=0x7fbe680ff190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe6c92e700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe680ff730 0x7fbe680ff190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe6c92e700 1 -- 172.21.15.130:0/4106541562 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580095f0 con 0x7fbe68100b70 2023-09-12T16:02:19.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe67fff700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe680ff730 0x7fbe680ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:19.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe6c92e700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 0x7fbe6819d600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbe60007d70 tx=0x7fbe6000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:19.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe657fa700 1 -- 172.21.15.130:0/4106541562 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6000ad40 con 0x7fbe68100b70 2023-09-12T16:02:19.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6819e3b0 con 0x7fbe68100b70 2023-09-12T16:02:19.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe657fa700 1 -- 172.21.15.130:0/4106541562 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe600128e0 con 0x7fbe68100b70 2023-09-12T16:02:19.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.019+0000 7fbe657fa700 1 -- 172.21.15.130:0/4106541562 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe600116b0 con 0x7fbe68100b70 2023-09-12T16:02:19.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.020+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6819e8c0 con 0x7fbe68100b70 2023-09-12T16:02:19.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.021+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6810ad50 con 0x7fbe68100b70 2023-09-12T16:02:19.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.021+0000 7fbe657fa700 1 -- 172.21.15.130:0/4106541562 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fbe60012420 con 0x7fbe68100b70 2023-09-12T16:02:19.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.022+0000 7fbe657fa700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbe500a0690 0x7fbe500a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.022+0000 7fbe67fff700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbe500a0690 0x7fbe500a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.022+0000 7fbe657fa700 1 -- 172.21.15.130:0/4106541562 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbe600cbcf0 con 0x7fbe68100b70 2023-09-12T16:02:19.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.022+0000 7fbe67fff700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbe500a0690 0x7fbe500a2b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe58000c00 tx=0x7fbe58005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:19.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.026+0000 7fbe657fa700 1 -- 172.21.15.130:0/4106541562 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbe600955d0 con 0x7fbe68100b70 2023-09-12T16:02:19.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.219+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe681063e0 con 0x7fbe68100b70 2023-09-12T16:02:19.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.221+0000 7fbe657fa700 1 -- 172.21.15.130:0/4106541562 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24004 (secure 0 0 0) 0x7fbe60098bf0 con 0x7fbe68100b70 2023-09-12T16:02:19.222 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:19.222 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":86,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:11.262527+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:19.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbe500a0690 msgr2=0x7fbe500a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbe500a0690 0x7fbe500a2b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe58000c00 tx=0x7fbe58005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 msgr2=0x7fbe6819d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 0x7fbe6819d600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbe60007d70 tx=0x7fbe6000a340 comp rx=0 tx=0).stop 2023-09-12T16:02:19.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 shutdown_connections 2023-09-12T16:02:19.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbe500a0690 0x7fbe500a2b50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe68100b70 0x7fbe6819d600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe680ff730 0x7fbe680ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 --2- 172.21.15.130:0/4106541562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe681000f0 0x7fbe6819afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 >> 172.21.15.130:0/4106541562 conn(0x7fbe680fa1c0 msgr2=0x7fbe680fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:19.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 shutdown_connections 2023-09-12T16:02:19.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.223+0000 7fbe6e391700 1 -- 172.21.15.130:0/4106541562 wait complete. 2023-09-12T16:02:19.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:02:19.237 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:02:19.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.361+0000 7f45867ae700 1 -- 172.21.15.130:0/3514424761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4580101070 msgr2=0x7f4580103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.361+0000 7f45867ae700 1 --2- 172.21.15.130:0/3514424761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4580101070 0x7f4580103460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4570009990 tx=0x7f4570009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.361+0000 7f45867ae700 1 -- 172.21.15.130:0/3514424761 shutdown_connections 2023-09-12T16:02:19.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.361+0000 7f45867ae700 1 --2- 172.21.15.130:0/3514424761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45801062d0 0x7f4580108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.361+0000 7f45867ae700 1 --2- 172.21.15.130:0/3514424761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4580101070 0x7f4580103460 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.361+0000 7f45867ae700 1 --2- 172.21.15.130:0/3514424761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45801039a0 0x7f4580105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.361+0000 7f45867ae700 1 -- 172.21.15.130:0/3514424761 >> 172.21.15.130:0/3514424761 conn(0x7f45800faa60 msgr2=0x7f45800fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:19.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.361+0000 7f45867ae700 1 -- 172.21.15.130:0/3514424761 shutdown_connections 2023-09-12T16:02:19.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.361+0000 7f45867ae700 1 -- 172.21.15.130:0/3514424761 wait complete. 2023-09-12T16:02:19.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f45867ae700 1 Processor -- start 2023-09-12T16:02:19.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f45867ae700 1 -- start start 2023-09-12T16:02:19.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f45867ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4580101070 0x7f4580196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f45867ae700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45801039a0 0x7f4580196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f45867ae700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45801062d0 0x7f458019d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f45867ae700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458019de80 con 0x7f45801039a0 2023-09-12T16:02:19.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f45867ae700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458019dfc0 con 0x7f45801062d0 2023-09-12T16:02:19.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f45867ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458019e150 con 0x7f4580101070 2023-09-12T16:02:19.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4577fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45801039a0 0x7f4580196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4577fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45801039a0 0x7f4580196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53574/0 (socket says 172.21.15.130:53574) 2023-09-12T16:02:19.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4577fff700 1 -- 172.21.15.130:0/1762579566 learned_addr learned my addr 172.21.15.130:0/1762579566 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:19.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f457ffff700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4580101070 0x7f4580196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4584d4b700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45801062d0 0x7f458019d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4577fff700 1 -- 172.21.15.130:0/1762579566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4580101070 msgr2=0x7f4580196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4577fff700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4580101070 0x7f4580196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4577fff700 1 -- 172.21.15.130:0/1762579566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45801062d0 msgr2=0x7f458019d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4577fff700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45801062d0 0x7f458019d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4577fff700 1 -- 172.21.15.130:0/1762579566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45700095f0 con 0x7f45801039a0 2023-09-12T16:02:19.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4584d4b700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45801062d0 0x7f458019d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:19.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.362+0000 7f4577fff700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45801039a0 0x7f4580196840 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f456800e9a0 tx=0x7f456800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:19.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.363+0000 7f457dffb700 1 -- 172.21.15.130:0/1762579566 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568009790 con 0x7f45801039a0 2023-09-12T16:02:19.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.363+0000 7f457dffb700 1 -- 172.21.15.130:0/1762579566 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f456800cbd0 con 0x7f45801039a0 2023-09-12T16:02:19.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.363+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458019e430 con 0x7f45801039a0 2023-09-12T16:02:19.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.363+0000 7f457dffb700 1 -- 172.21.15.130:0/1762579566 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568018680 con 0x7f45801039a0 2023-09-12T16:02:19.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.363+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458019e940 con 0x7f45801039a0 2023-09-12T16:02:19.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.364+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45800fc5e0 con 0x7f45801039a0 2023-09-12T16:02:19.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.365+0000 7f457dffb700 1 -- 172.21.15.130:0/1762579566 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f456800cd40 con 0x7f45801039a0 2023-09-12T16:02:19.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.365+0000 7f457dffb700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f456c0a0640 0x7f456c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.365+0000 7f457ffff700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f456c0a0640 0x7f456c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.365+0000 7f457dffb700 1 -- 172.21.15.130:0/1762579566 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4568014070 con 0x7f45801039a0 2023-09-12T16:02:19.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.366+0000 7f457ffff700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f456c0a0640 0x7f456c0a2b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45700098e0 tx=0x7f4570000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:19.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.369+0000 7f457dffb700 1 -- 172.21.15.130:0/1762579566 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4568051e10 con 0x7f45801039a0 2023-09-12T16:02:19.559 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.559+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f458002cfa0 con 0x7f45801039a0 2023-09-12T16:02:19.560 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.559+0000 7f457dffb700 1 -- 172.21.15.130:0/1762579566 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45680959f0 con 0x7f45801039a0 2023-09-12T16:02:19.560 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:02:19.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f456c0a0640 msgr2=0x7f456c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f456c0a0640 0x7f456c0a2b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45700098e0 tx=0x7f4570000bc0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45801039a0 msgr2=0x7f4580196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45801039a0 0x7f4580196840 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f456800e9a0 tx=0x7f456800ed60 comp rx=0 tx=0).stop 2023-09-12T16:02:19.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 shutdown_connections 2023-09-12T16:02:19.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f456c0a0640 0x7f456c0a2b00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45801062d0 0x7f458019d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4580101070 0x7f4580196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 --2- 172.21.15.130:0/1762579566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45801039a0 0x7f4580196840 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 >> 172.21.15.130:0/1762579566 conn(0x7f45800faa60 msgr2=0x7f4580101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:19.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 shutdown_connections 2023-09-12T16:02:19.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.561+0000 7f45867ae700 1 -- 172.21.15.130:0/1762579566 wait complete. 2023-09-12T16:02:19.574 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:02:19.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 -- 172.21.15.130:0/3702590972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db0101080 msgr2=0x7f8db0107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 --2- 172.21.15.130:0/3702590972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db0101080 0x7f8db0107930 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8da800b0d0 tx=0x7f8da800b3e0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 -- 172.21.15.130:0/3702590972 shutdown_connections 2023-09-12T16:02:19.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 --2- 172.21.15.130:0/3702590972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db0101080 0x7f8db0107930 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 --2- 172.21.15.130:0/3702590972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8db00ff4f0 0x7f8db00ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 --2- 172.21.15.130:0/3702590972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db01006e0 0x7f8db0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 -- 172.21.15.130:0/3702590972 >> 172.21.15.130:0/3702590972 conn(0x7f8db00faa60 msgr2=0x7f8db00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:19.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 -- 172.21.15.130:0/3702590972 shutdown_connections 2023-09-12T16:02:19.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 -- 172.21.15.130:0/3702590972 wait complete. 2023-09-12T16:02:19.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 Processor -- start 2023-09-12T16:02:19.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.699+0000 7f8db630c700 1 -- start start 2023-09-12T16:02:19.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db630c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db00ff4f0 0x7f8db01963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db630c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db01006e0 0x7f8db0196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db630c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8db0101080 0x7f8db019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db630c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db019df60 con 0x7f8db00ff4f0 2023-09-12T16:02:19.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db630c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db019e0a0 con 0x7f8db01006e0 2023-09-12T16:02:19.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db630c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db019e230 con 0x7f8db0101080 2023-09-12T16:02:19.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db00ff4f0 0x7f8db01963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db00ff4f0 0x7f8db01963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53586/0 (socket says 172.21.15.130:53586) 2023-09-12T16:02:19.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daffff700 1 -- 172.21.15.130:0/2204330870 learned_addr learned my addr 172.21.15.130:0/2204330870 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:19.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db48a9700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8db0101080 0x7f8db019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daf7fe700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db01006e0 0x7f8db0196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daffff700 1 -- 172.21.15.130:0/2204330870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8db0101080 msgr2=0x7f8db019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daffff700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8db0101080 0x7f8db019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daffff700 1 -- 172.21.15.130:0/2204330870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db01006e0 msgr2=0x7f8db0196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daffff700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db01006e0 0x7f8db0196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daffff700 1 -- 172.21.15.130:0/2204330870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da8009d00 con 0x7f8db00ff4f0 2023-09-12T16:02:19.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db48a9700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8db0101080 0x7f8db019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:19.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daf7fe700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db01006e0 0x7f8db0196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:19.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8daffff700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db00ff4f0 0x7f8db01963e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8da400d980 tx=0x7f8da400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:19.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8dad7fa700 1 -- 172.21.15.130:0/2204330870 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400bb60 con 0x7f8db00ff4f0 2023-09-12T16:02:19.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db019e510 con 0x7f8db00ff4f0 2023-09-12T16:02:19.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8dad7fa700 1 -- 172.21.15.130:0/2204330870 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8da4007620 con 0x7f8db00ff4f0 2023-09-12T16:02:19.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8dad7fa700 1 -- 172.21.15.130:0/2204330870 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400f620 con 0x7f8db00ff4f0 2023-09-12T16:02:19.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.700+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db019ea70 con 0x7f8db00ff4f0 2023-09-12T16:02:19.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.701+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db010ad60 con 0x7f8db00ff4f0 2023-09-12T16:02:19.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.702+0000 7f8dad7fa700 1 -- 172.21.15.130:0/2204330870 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f8da4010460 con 0x7f8db00ff4f0 2023-09-12T16:02:19.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.703+0000 7f8dad7fa700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d980a0640 0x7f8d980a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:19.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.703+0000 7f8dad7fa700 1 -- 172.21.15.130:0/2204330870 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8da40cbdb0 con 0x7f8db00ff4f0 2023-09-12T16:02:19.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.703+0000 7f8daf7fe700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d980a0640 0x7f8d980a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:19.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.703+0000 7f8daf7fe700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d980a0640 0x7f8d980a2b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009bc0 tx=0x7f8da0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:19.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.707+0000 7f8dad7fa700 1 -- 172.21.15.130:0/2204330870 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8da40956f0 con 0x7f8db00ff4f0 2023-09-12T16:02:19.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.896+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8db019ec90 con 0x7f8db00ff4f0 2023-09-12T16:02:19.897 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.896+0000 7f8dad7fa700 1 -- 172.21.15.130:0/2204330870 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8da400de60 con 0x7f8db00ff4f0 2023-09-12T16:02:19.897 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:02:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d980a0640 msgr2=0x7f8d980a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d980a0640 0x7f8d980a2b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009bc0 tx=0x7f8da0009260 comp rx=0 tx=0).stop 2023-09-12T16:02:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db00ff4f0 msgr2=0x7f8db01963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db00ff4f0 0x7f8db01963e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8da400d980 tx=0x7f8da400dd40 comp rx=0 tx=0).stop 2023-09-12T16:02:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 shutdown_connections 2023-09-12T16:02:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d980a0640 0x7f8d980a2b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8db0101080 0x7f8db019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8db00ff4f0 0x7f8db01963e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 --2- 172.21.15.130:0/2204330870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db01006e0 0x7f8db0196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:19.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.898+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 >> 172.21.15.130:0/2204330870 conn(0x7f8db00faa60 msgr2=0x7f8db0067780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:19.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.899+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 shutdown_connections 2023-09-12T16:02:19.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:19.899+0000 7f8db630c700 1 -- 172.21.15.130:0/2204330870 wait complete. 2023-09-12T16:02:19.910 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:19 smithi130 ceph-mon[98670]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:19.910 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:19 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1124527326' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-12T16:02:19.910 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:19 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4106541562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:19.910 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:19 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1762579566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:02:19.912 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:02:19.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:19 smithi159 ceph-mon[102522]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:19.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:19 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1124527326' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-12T16:02:19.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:19 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4106541562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:19.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:19 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1762579566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:02:19.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:19 smithi203 ceph-mon[103431]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:19.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:19 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1124527326' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-12T16:02:19.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:19 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4106541562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:19.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:19 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1762579566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:02:20.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.037+0000 7f08be745700 1 -- 172.21.15.130:0/698751999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 msgr2=0x7f08b8100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.037+0000 7f08be745700 1 --2- 172.21.15.130:0/698751999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 0x7f08b8100b40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0099c0 tx=0x7f08ac009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.037+0000 7f08be745700 1 -- 172.21.15.130:0/698751999 shutdown_connections 2023-09-12T16:02:20.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.037+0000 7f08be745700 1 --2- 172.21.15.130:0/698751999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08b8101080 0x7f08b8107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.037+0000 7f08be745700 1 --2- 172.21.15.130:0/698751999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b80ff4f0 0x7f08b80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.037+0000 7f08be745700 1 --2- 172.21.15.130:0/698751999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 0x7f08b8100b40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.037+0000 7f08be745700 1 -- 172.21.15.130:0/698751999 >> 172.21.15.130:0/698751999 conn(0x7f08b80faa60 msgr2=0x7f08b80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:20.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.037+0000 7f08be745700 1 -- 172.21.15.130:0/698751999 shutdown_connections 2023-09-12T16:02:20.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.037+0000 7f08be745700 1 -- 172.21.15.130:0/698751999 wait complete. 2023-09-12T16:02:20.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08be745700 1 Processor -- start 2023-09-12T16:02:20.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08be745700 1 -- start start 2023-09-12T16:02:20.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08be745700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08b80ff4f0 0x7f08b80682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08be745700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 0x7f08b819b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08be745700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b8101080 0x7f08b819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08be745700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b819de70 con 0x7f08b81006e0 2023-09-12T16:02:20.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08be745700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b819dfe0 con 0x7f08b80ff4f0 2023-09-12T16:02:20.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08be745700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b819e1a0 con 0x7f08b8101080 2023-09-12T16:02:20.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08b77fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 0x7f08b819b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08b77fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 0x7f08b819b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53606/0 (socket says 172.21.15.130:53606) 2023-09-12T16:02:20.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08b77fe700 1 -- 172.21.15.130:0/2314829223 learned_addr learned my addr 172.21.15.130:0/2314829223 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:20.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08bcce2700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b8101080 0x7f08b819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.038+0000 7f08b7fff700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08b80ff4f0 0x7f08b80682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08bcce2700 1 -- 172.21.15.130:0/2314829223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08b80ff4f0 msgr2=0x7f08b80682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08bcce2700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08b80ff4f0 0x7f08b80682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08bcce2700 1 -- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 msgr2=0x7f08b819b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08bcce2700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 0x7f08b819b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08bcce2700 1 -- 172.21.15.130:0/2314829223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0095f0 con 0x7f08b8101080 2023-09-12T16:02:20.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08b77fe700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 0x7f08b819b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:02:20.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08bcce2700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b8101080 0x7f08b819d6a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f08b000efc0 tx=0x7f08b000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:20.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08b57fa700 1 -- 172.21.15.130:0/2314829223 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0018070 con 0x7f08b8101080 2023-09-12T16:02:20.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b819e420 con 0x7f08b8101080 2023-09-12T16:02:20.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08b57fa700 1 -- 172.21.15.130:0/2314829223 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08b000f040 con 0x7f08b8101080 2023-09-12T16:02:20.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b819e930 con 0x7f08b8101080 2023-09-12T16:02:20.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.039+0000 7f08b57fa700 1 -- 172.21.15.130:0/2314829223 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b000cce0 con 0x7f08b8101080 2023-09-12T16:02:20.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.040+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b810ad60 con 0x7f08b8101080 2023-09-12T16:02:20.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.041+0000 7f08b57fa700 1 -- 172.21.15.130:0/2314829223 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f08b0003a00 con 0x7f08b8101080 2023-09-12T16:02:20.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.042+0000 7f08b57fa700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f08a00a0640 0x7f08a00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.042+0000 7f08b7fff700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f08a00a0640 0x7f08a00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.042+0000 7f08b57fa700 1 -- 172.21.15.130:0/2314829223 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f08b00cc300 con 0x7f08b8101080 2023-09-12T16:02:20.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.042+0000 7f08b7fff700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f08a00a0640 0x7f08a00a2b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f08b8100540 tx=0x7f08a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:20.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.046+0000 7f08b57fa700 1 -- 172.21.15.130:0/2314829223 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f08b0095cd0 con 0x7f08b8101080 2023-09-12T16:02:20.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.238+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08b8106440 con 0x7f08b8101080 2023-09-12T16:02:20.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.239+0000 7f08b57fa700 1 -- 172.21.15.130:0/2314829223 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08b00992f0 con 0x7f08b8101080 2023-09-12T16:02:20.240 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:02:20.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f08a00a0640 msgr2=0x7f08a00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f08a00a0640 0x7f08a00a2b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f08b8100540 tx=0x7f08a8009260 comp rx=0 tx=0).stop 2023-09-12T16:02:20.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b8101080 msgr2=0x7f08b819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b8101080 0x7f08b819d6a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f08b000efc0 tx=0x7f08b000c3a0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 shutdown_connections 2023-09-12T16:02:20.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f08a00a0640 0x7f08a00a2b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b8101080 0x7f08b819d6a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08b80ff4f0 0x7f08b80682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 --2- 172.21.15.130:0/2314829223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b81006e0 0x7f08b819b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 >> 172.21.15.130:0/2314829223 conn(0x7f08b80faa60 msgr2=0x7f08b80fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:20.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 shutdown_connections 2023-09-12T16:02:20.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.241+0000 7f08be745700 1 -- 172.21.15.130:0/2314829223 wait complete. 2023-09-12T16:02:20.254 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:02:20.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.378+0000 7f07830fe700 1 -- 172.21.15.130:0/3078151416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c102210 msgr2=0x7f077c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.378+0000 7f07830fe700 1 --2- 172.21.15.130:0/3078151416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c102210 0x7f077c108ac0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f077400b0a0 tx=0x7f077400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.378+0000 7f07830fe700 1 -- 172.21.15.130:0/3078151416 shutdown_connections 2023-09-12T16:02:20.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.378+0000 7f07830fe700 1 --2- 172.21.15.130:0/3078151416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c102210 0x7f077c108ac0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.378+0000 7f07830fe700 1 --2- 172.21.15.130:0/3078151416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f077c0fe8a0 0x7f077c0fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.378+0000 7f07830fe700 1 --2- 172.21.15.130:0/3078151416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f077c101870 0x7f077c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.378+0000 7f07830fe700 1 -- 172.21.15.130:0/3078151416 >> 172.21.15.130:0/3078151416 conn(0x7f077c0fa1d0 msgr2=0x7f077c0fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:20.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.378+0000 7f07830fe700 1 -- 172.21.15.130:0/3078151416 shutdown_connections 2023-09-12T16:02:20.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.378+0000 7f07830fe700 1 -- 172.21.15.130:0/3078151416 wait complete. 2023-09-12T16:02:20.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f07830fe700 1 Processor -- start 2023-09-12T16:02:20.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f07830fe700 1 -- start start 2023-09-12T16:02:20.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f07830fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f077c0fe8a0 0x7f077c1011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f07830fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c101870 0x7f077c0ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f07830fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f077c102210 0x7f077c0ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f07830fe700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c100500 con 0x7f077c101870 2023-09-12T16:02:20.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f07830fe700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c100670 con 0x7f077c102210 2023-09-12T16:02:20.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f07830fe700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c1a0430 con 0x7f077c0fe8a0 2023-09-12T16:02:20.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f077bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c101870 0x7f077c0ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f077bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c101870 0x7f077c0ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53620/0 (socket says 172.21.15.130:53620) 2023-09-12T16:02:20.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f077bfff700 1 -- 172.21.15.130:0/1474214705 learned_addr learned my addr 172.21.15.130:0/1474214705 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:20.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f0780e9a700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f077c0fe8a0 0x7f077c1011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f078169b700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f077c102210 0x7f077c0ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f077bfff700 1 -- 172.21.15.130:0/1474214705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f077c0fe8a0 msgr2=0x7f077c1011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f077bfff700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f077c0fe8a0 0x7f077c1011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f077bfff700 1 -- 172.21.15.130:0/1474214705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f077c102210 msgr2=0x7f077c0ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f077bfff700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f077c102210 0x7f077c0ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f077bfff700 1 -- 172.21.15.130:0/1474214705 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0774009d00 con 0x7f077c101870 2023-09-12T16:02:20.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f078169b700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f077c102210 0x7f077c0ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:20.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f077bfff700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c101870 0x7f077c0ff820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f076800d950 tx=0x7f076800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:20.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.379+0000 7f0779ffb700 1 -- 172.21.15.130:0/1474214705 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768009940 con 0x7f077c101870 2023-09-12T16:02:20.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.380+0000 7f0779ffb700 1 -- 172.21.15.130:0/1474214705 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0768010460 con 0x7f077c101870 2023-09-12T16:02:20.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.380+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c1a0710 con 0x7f077c101870 2023-09-12T16:02:20.387 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.380+0000 7f0779ffb700 1 -- 172.21.15.130:0/1474214705 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076800f620 con 0x7f077c101870 2023-09-12T16:02:20.387 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.380+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c1a0c20 con 0x7f077c101870 2023-09-12T16:02:20.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.381+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c1075b0 con 0x7f077c101870 2023-09-12T16:02:20.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.382+0000 7f0779ffb700 1 -- 172.21.15.130:0/1474214705 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f07680108d0 con 0x7f077c101870 2023-09-12T16:02:20.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.382+0000 7f0779ffb700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f076c0a0640 0x7f076c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.382+0000 7f0779ffb700 1 -- 172.21.15.130:0/1474214705 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f07680cbbe0 con 0x7f077c101870 2023-09-12T16:02:20.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.382+0000 7f0780e9a700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f076c0a0640 0x7f076c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.383+0000 7f0780e9a700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f076c0a0640 0x7f076c0a2b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f077c067560 tx=0x7f077000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:20.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.387+0000 7f0779ffb700 1 -- 172.21.15.130:0/1474214705 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0768095520 con 0x7f077c101870 2023-09-12T16:02:20.576 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.576+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f077c02cf50 con 0x7f077c101870 2023-09-12T16:02:20.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.576+0000 7f0779ffb700 1 -- 172.21.15.130:0/1474214705 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f07680160a0 con 0x7f077c101870 2023-09-12T16:02:20.577 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:02:20.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f076c0a0640 msgr2=0x7f076c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f076c0a0640 0x7f076c0a2b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f077c067560 tx=0x7f077000a380 comp rx=0 tx=0).stop 2023-09-12T16:02:20.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c101870 msgr2=0x7f077c0ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c101870 0x7f077c0ff820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f076800d950 tx=0x7f076800dc60 comp rx=0 tx=0).stop 2023-09-12T16:02:20.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 shutdown_connections 2023-09-12T16:02:20.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f076c0a0640 0x7f076c0a2b00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f077c102210 0x7f077c0ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f077c0fe8a0 0x7f077c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 --2- 172.21.15.130:0/1474214705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f077c101870 0x7f077c0ff820 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.578+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 >> 172.21.15.130:0/1474214705 conn(0x7f077c0fa1d0 msgr2=0x7f077c0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:20.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.579+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 shutdown_connections 2023-09-12T16:02:20.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.579+0000 7f07830fe700 1 -- 172.21.15.130:0/1474214705 wait complete. 2023-09-12T16:02:20.591 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T16:02:20.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.713+0000 7f016b2c3700 1 -- 172.21.15.130:0/441733360 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0164100750 msgr2=0x7f0164100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.713+0000 7f016b2c3700 1 --2- 172.21.15.130:0/441733360 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0164100750 0x7f0164100bb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f01600099c0 tx=0x7f0160009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.713+0000 7f016b2c3700 1 -- 172.21.15.130:0/441733360 shutdown_connections 2023-09-12T16:02:20.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.713+0000 7f016b2c3700 1 --2- 172.21.15.130:0/441733360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01641010f0 0x7f01641079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.713+0000 7f016b2c3700 1 --2- 172.21.15.130:0/441733360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01640ff560 0x7f01640ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.713+0000 7f016b2c3700 1 --2- 172.21.15.130:0/441733360 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0164100750 0x7f0164100bb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.713+0000 7f016b2c3700 1 -- 172.21.15.130:0/441733360 >> 172.21.15.130:0/441733360 conn(0x7f01640faad0 msgr2=0x7f01640fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:20.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 -- 172.21.15.130:0/441733360 shutdown_connections 2023-09-12T16:02:20.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 -- 172.21.15.130:0/441733360 wait complete. 2023-09-12T16:02:20.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 Processor -- start 2023-09-12T16:02:20.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 -- start start 2023-09-12T16:02:20.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01640ff560 0x7f0164069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0164100750 0x7f016419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641010f0 0x7f016419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419ddc0 con 0x7f01640ff560 2023-09-12T16:02:20.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419df30 con 0x7f01641010f0 2023-09-12T16:02:20.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016b2c3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419e0f0 con 0x7f0164100750 2023-09-12T16:02:20.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016905f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01640ff560 0x7f0164069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016905f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01640ff560 0x7f0164069480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53632/0 (socket says 172.21.15.130:53632) 2023-09-12T16:02:20.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016905f700 1 -- 172.21.15.130:0/2209906694 learned_addr learned my addr 172.21.15.130:0/2209906694 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:20.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f016885e700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0164100750 0x7f016419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.714+0000 7f0169860700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641010f0 0x7f016419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f016905f700 1 -- 172.21.15.130:0/2209906694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0164100750 msgr2=0x7f016419afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f016905f700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0164100750 0x7f016419afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f016905f700 1 -- 172.21.15.130:0/2209906694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641010f0 msgr2=0x7f016419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f016905f700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641010f0 0x7f016419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f016905f700 1 -- 172.21.15.130:0/2209906694 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01600095f0 con 0x7f01640ff560 2023-09-12T16:02:20.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f016905f700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01640ff560 0x7f0164069480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f015400e970 tx=0x7f015400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:20.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f015a7fc700 1 -- 172.21.15.130:0/2209906694 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154009790 con 0x7f01640ff560 2023-09-12T16:02:20.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f015a7fc700 1 -- 172.21.15.130:0/2209906694 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f015400c7e0 con 0x7f01640ff560 2023-09-12T16:02:20.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016419e3d0 con 0x7f01640ff560 2023-09-12T16:02:20.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f015a7fc700 1 -- 172.21.15.130:0/2209906694 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154003c70 con 0x7f01640ff560 2023-09-12T16:02:20.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.715+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016419e8e0 con 0x7f01640ff560 2023-09-12T16:02:20.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.716+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0164106b00 con 0x7f01640ff560 2023-09-12T16:02:20.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.717+0000 7f015a7fc700 1 -- 172.21.15.130:0/2209906694 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f015400cc50 con 0x7f01640ff560 2023-09-12T16:02:20.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.717+0000 7f015a7fc700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01500a0640 0x7f01500a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:20.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.718+0000 7f016885e700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01500a0640 0x7f01500a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:20.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.718+0000 7f015a7fc700 1 -- 172.21.15.130:0/2209906694 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0154014070 con 0x7f01640ff560 2023-09-12T16:02:20.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.718+0000 7f016885e700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01500a0640 0x7f01500a2b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f016000b580 tx=0x7f0160005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:20.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.722+0000 7f015a7fc700 1 -- 172.21.15.130:0/2209906694 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0154095880 con 0x7f01640ff560 2023-09-12T16:02:20.929 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.929+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f016410ad20 con 0x7f01640ff560 2023-09-12T16:02:20.930 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.929+0000 7f015a7fc700 1 -- 172.21.15.130:0/2209906694 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+598 (secure 0 0 0) 0x7f0154003dd0 con 0x7f01640ff560 2023-09-12T16:02:20.930 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/1706265745 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:20.930 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6800/3042728256 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:20.931 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/3742416190 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:20.931 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/2502139426 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:20.931 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6801/3042728256 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:20.931 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6800/840569029 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:20.931 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6801/840569029 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:20.931 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/410531865 2023-09-13T15:55:50.680182+0000 2023-09-12T16:02:20.932 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/288931411 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:20.932 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/1303099884 2023-09-13T15:55:28.728403+0000 2023-09-12T16:02:20.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01500a0640 msgr2=0x7f01500a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01500a0640 0x7f01500a2b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f016000b580 tx=0x7f0160005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01640ff560 msgr2=0x7f0164069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:20.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01640ff560 0x7f0164069480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f015400e970 tx=0x7f015400ec80 comp rx=0 tx=0).stop 2023-09-12T16:02:20.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 shutdown_connections 2023-09-12T16:02:20.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f01500a0640 0x7f01500a2b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641010f0 0x7f016419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01640ff560 0x7f0164069480 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 --2- 172.21.15.130:0/2209906694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0164100750 0x7f016419afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:20.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.931+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 >> 172.21.15.130:0/2209906694 conn(0x7f01640faad0 msgr2=0x7f0164103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:20.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.932+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 shutdown_connections 2023-09-12T16:02:20.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:20.932+0000 7f016b2c3700 1 -- 172.21.15.130:0/2209906694 wait complete. 2023-09-12T16:02:20.938 INFO:teuthology.orchestra.run.smithi130.stderr:listed 10 entries 2023-09-12T16:02:20.943 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:20 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2204330870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:02:20.943 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:20 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2314829223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:02:20.943 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:20 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1474214705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:02:20.944 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2023-09-12T16:02:20.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:20.945 DEBUG:teuthology.orchestra.run.smithi130:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-09-12T16:02:20.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:20 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2204330870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:02:20.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:20 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2314829223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:02:20.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:20 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1474214705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:02:20.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi130.stderr:ceph-fuse[156795]: fuse finished with error 0 and tester_r 0 2023-09-12T16:02:20.984 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T16:02:20.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:20 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2204330870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:02:20.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:20 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2314829223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:02:20.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:20 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1474214705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:02:21.001 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi130.front.sepia.ceph.com 2023-09-12T16:02:21.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:02:21.002 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-09-12T16:02:21.070 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:02:21.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.231+0000 7fa6790c6700 1 -- 172.21.15.130:0/3697150547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6740ff730 msgr2=0x7fa6740ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.231+0000 7fa6790c6700 1 --2- 172.21.15.130:0/3697150547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6740ff730 0x7fa6740ffbb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa668009990 tx=0x7fa668009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.231+0000 7fa6790c6700 1 -- 172.21.15.130:0/3697150547 shutdown_connections 2023-09-12T16:02:21.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.231+0000 7fa6790c6700 1 --2- 172.21.15.130:0/3697150547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6741000f0 0x7fa674107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.231+0000 7fa6790c6700 1 --2- 172.21.15.130:0/3697150547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa674100b70 0x7fa674100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.231+0000 7fa6790c6700 1 --2- 172.21.15.130:0/3697150547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6740ff730 0x7fa6740ffbb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.231+0000 7fa6790c6700 1 -- 172.21.15.130:0/3697150547 >> 172.21.15.130:0/3697150547 conn(0x7fa6740fa1c0 msgr2=0x7fa6740fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:21.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.231+0000 7fa6790c6700 1 -- 172.21.15.130:0/3697150547 shutdown_connections 2023-09-12T16:02:21.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.231+0000 7fa6790c6700 1 -- 172.21.15.130:0/3697150547 wait complete. 2023-09-12T16:02:21.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.232+0000 7fa6790c6700 1 Processor -- start 2023-09-12T16:02:21.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.232+0000 7fa6790c6700 1 -- start start 2023-09-12T16:02:21.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa6790c6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6740ff730 0x7fa674195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa6790c6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6741000f0 0x7fa674195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa6790c6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa674100b70 0x7fa67419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa6790c6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67419ccc0 con 0x7fa674100b70 2023-09-12T16:02:21.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa6790c6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67419ce00 con 0x7fa6740ff730 2023-09-12T16:02:21.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa6790c6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67419cf90 con 0x7fa6741000f0 2023-09-12T16:02:21.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa674100b70 0x7fa67419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa674100b70 0x7fa67419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53650/0 (socket says 172.21.15.130:53650) 2023-09-12T16:02:21.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67359e700 1 -- 172.21.15.130:0/2342929045 learned_addr learned my addr 172.21.15.130:0/2342929045 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:21.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa672d9d700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6740ff730 0x7fa674195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67259c700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6741000f0 0x7fa674195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67359e700 1 -- 172.21.15.130:0/2342929045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6741000f0 msgr2=0x7fa674195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67359e700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6741000f0 0x7fa674195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67359e700 1 -- 172.21.15.130:0/2342929045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6740ff730 msgr2=0x7fa674195140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67359e700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6740ff730 0x7fa674195140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67359e700 1 -- 172.21.15.130:0/2342929045 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6680095f0 con 0x7fa674100b70 2023-09-12T16:02:21.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa67359e700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa674100b70 0x7fa67419c6f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa66c007ee0 tx=0x7fa66c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:21.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa65bfff700 1 -- 172.21.15.130:0/2342929045 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c017070 con 0x7fa674100b70 2023-09-12T16:02:21.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa65bfff700 1 -- 172.21.15.130:0/2342929045 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa66c00f040 con 0x7fa674100b70 2023-09-12T16:02:21.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6740fe680 con 0x7fa674100b70 2023-09-12T16:02:21.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa65bfff700 1 -- 172.21.15.130:0/2342929045 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c005580 con 0x7fa674100b70 2023-09-12T16:02:21.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.233+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6740feb40 con 0x7fa674100b70 2023-09-12T16:02:21.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.234+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa674066b60 con 0x7fa674100b70 2023-09-12T16:02:21.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.235+0000 7fa65bfff700 1 -- 172.21.15.130:0/2342929045 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fa66c00db20 con 0x7fa674100b70 2023-09-12T16:02:21.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.236+0000 7fa65bfff700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa65c0a0690 0x7fa65c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.236+0000 7fa672d9d700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa65c0a0690 0x7fa65c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.236+0000 7fa65bfff700 1 -- 172.21.15.130:0/2342929045 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa66c0cbc10 con 0x7fa674100b70 2023-09-12T16:02:21.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.236+0000 7fa672d9d700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa65c0a0690 0x7fa65c0a2b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa66400cb10 tx=0x7fa66400e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:21.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.239+0000 7fa65bfff700 1 -- 172.21.15.130:0/2342929045 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa66c0954f0 con 0x7fa674100b70 2023-09-12T16:02:21.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.433+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa67410aed0 con 0x7fa674100b70 2023-09-12T16:02:21.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.435+0000 7fa65bfff700 1 -- 172.21.15.130:0/2342929045 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24004 (secure 0 0 0) 0x7fa66c098b10 con 0x7fa674100b70 2023-09-12T16:02:21.435 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:21.436 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":86,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:11.262527+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:21.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa65c0a0690 msgr2=0x7fa65c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa65c0a0690 0x7fa65c0a2b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa66400cb10 tx=0x7fa66400e040 comp rx=0 tx=0).stop 2023-09-12T16:02:21.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa674100b70 msgr2=0x7fa67419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa674100b70 0x7fa67419c6f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa66c007ee0 tx=0x7fa66c00d3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 shutdown_connections 2023-09-12T16:02:21.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa65c0a0690 0x7fa65c0a2b50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa674100b70 0x7fa67419c6f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6740ff730 0x7fa674195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 --2- 172.21.15.130:0/2342929045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6741000f0 0x7fa674195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 >> 172.21.15.130:0/2342929045 conn(0x7fa6740fa1c0 msgr2=0x7fa674103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:21.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 shutdown_connections 2023-09-12T16:02:21.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.437+0000 7fa6790c6700 1 -- 172.21.15.130:0/2342929045 wait complete. 2023-09-12T16:02:21.450 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:02:21.573 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.571+0000 7f547edf1700 1 -- 172.21.15.130:0/1607044815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54780ff4f0 msgr2=0x7f54780ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.571+0000 7f547edf1700 1 --2- 172.21.15.130:0/1607044815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54780ff4f0 0x7f54780ff910 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f546c0099c0 tx=0x7f546c009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.571+0000 7f547edf1700 1 -- 172.21.15.130:0/1607044815 shutdown_connections 2023-09-12T16:02:21.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.571+0000 7f547edf1700 1 --2- 172.21.15.130:0/1607044815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5478101080 0x7f5478107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.571+0000 7f547edf1700 1 --2- 172.21.15.130:0/1607044815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54780ff4f0 0x7f54780ff910 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.575 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.571+0000 7f547edf1700 1 --2- 172.21.15.130:0/1607044815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54781006e0 0x7f5478100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.575 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.571+0000 7f547edf1700 1 -- 172.21.15.130:0/1607044815 >> 172.21.15.130:0/1607044815 conn(0x7f54780faa60 msgr2=0x7f54780fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:21.575 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.571+0000 7f547edf1700 1 -- 172.21.15.130:0/1607044815 shutdown_connections 2023-09-12T16:02:21.575 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.571+0000 7f547edf1700 1 -- 172.21.15.130:0/1607044815 wait complete. 2023-09-12T16:02:21.575 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547edf1700 1 Processor -- start 2023-09-12T16:02:21.576 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547edf1700 1 -- start start 2023-09-12T16:02:21.576 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547edf1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54780ff4f0 0x7f5478103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.576 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547edf1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54781006e0 0x7f5478101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.576 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547edf1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5478101080 0x7f5478102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.576 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547edf1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54781027a0 con 0x7f54781006e0 2023-09-12T16:02:21.576 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547edf1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5478102910 con 0x7f54780ff4f0 2023-09-12T16:02:21.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547edf1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54781a0370 con 0x7f5478101080 2023-09-12T16:02:21.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f5477fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54781006e0 0x7f5478101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547d38e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5478101080 0x7f5478102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f547cb8d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54780ff4f0 0x7f5478103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f5477fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54781006e0 0x7f5478101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53664/0 (socket says 172.21.15.130:53664) 2023-09-12T16:02:21.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f5477fff700 1 -- 172.21.15.130:0/3669452388 learned_addr learned my addr 172.21.15.130:0/3669452388 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:21.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f5477fff700 1 -- 172.21.15.130:0/3669452388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5478101080 msgr2=0x7f5478102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f5477fff700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5478101080 0x7f5478102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f5477fff700 1 -- 172.21.15.130:0/3669452388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54780ff4f0 msgr2=0x7f5478103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f5477fff700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54780ff4f0 0x7f5478103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.572+0000 7f5477fff700 1 -- 172.21.15.130:0/3669452388 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c0095f0 con 0x7f54781006e0 2023-09-12T16:02:21.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.573+0000 7f5477fff700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54781006e0 0x7f5478101ac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f546400d980 tx=0x7f546400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:21.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.573+0000 7f5475ffb700 1 -- 172.21.15.130:0/3669452388 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464004220 con 0x7f54781006e0 2023-09-12T16:02:21.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.573+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54781a0650 con 0x7f54781006e0 2023-09-12T16:02:21.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.573+0000 7f5475ffb700 1 -- 172.21.15.130:0/3669452388 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5464007640 con 0x7f54781006e0 2023-09-12T16:02:21.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.573+0000 7f5475ffb700 1 -- 172.21.15.130:0/3669452388 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464005690 con 0x7f54781006e0 2023-09-12T16:02:21.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.573+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54781a0bb0 con 0x7f54781006e0 2023-09-12T16:02:21.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.574+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547810ad60 con 0x7f54781006e0 2023-09-12T16:02:21.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.575+0000 7f5475ffb700 1 -- 172.21.15.130:0/3669452388 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f5464004750 con 0x7f54781006e0 2023-09-12T16:02:21.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.575+0000 7f5475ffb700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54680a0690 0x7f54680a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.576+0000 7f547cb8d700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54680a0690 0x7f54680a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.576+0000 7f5475ffb700 1 -- 172.21.15.130:0/3669452388 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f54640cbc10 con 0x7f54781006e0 2023-09-12T16:02:21.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.576+0000 7f547cb8d700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54680a0690 0x7f54680a2b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f546c009990 tx=0x7f546c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:21.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.580+0000 7f5475ffb700 1 -- 172.21.15.130:0/3669452388 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f54640954f0 con 0x7f54781006e0 2023-09-12T16:02:21.724 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:21 smithi130 ceph-mon[98670]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:21.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:21 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2209906694' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-12T16:02:21.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:21 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2342929045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:21.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.773+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54781a10a0 con 0x7f54781006e0 2023-09-12T16:02:21.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.775+0000 7f5475ffb700 1 -- 172.21.15.130:0/3669452388 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24004 (secure 0 0 0) 0x7f5464016020 con 0x7f54781006e0 2023-09-12T16:02:21.776 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:21.776 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":86,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:11.262527+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:21.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54680a0690 msgr2=0x7f54680a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54680a0690 0x7f54680a2b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f546c009990 tx=0x7f546c005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54781006e0 msgr2=0x7f5478101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54781006e0 0x7f5478101ac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f546400d980 tx=0x7f546400dc90 comp rx=0 tx=0).stop 2023-09-12T16:02:21.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 shutdown_connections 2023-09-12T16:02:21.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54680a0690 0x7f54680a2b50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5478101080 0x7f5478102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54780ff4f0 0x7f5478103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 --2- 172.21.15.130:0/3669452388 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54781006e0 0x7f5478101ac0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 >> 172.21.15.130:0/3669452388 conn(0x7f54780faa60 msgr2=0x7f5478068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:21.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 shutdown_connections 2023-09-12T16:02:21.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.777+0000 7f547edf1700 1 -- 172.21.15.130:0/3669452388 wait complete. 2023-09-12T16:02:21.791 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-12T16:02:21.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 -- 172.21.15.130:0/2843602974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8228103a00 msgr2=0x7f8228105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 --2- 172.21.15.130:0/2843602974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8228103a00 0x7f8228105df0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8218009990 tx=0x7f8218009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 -- 172.21.15.130:0/2843602974 shutdown_connections 2023-09-12T16:02:21.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 --2- 172.21.15.130:0/2843602974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8228106330 0x7f8228108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 --2- 172.21.15.130:0/2843602974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82281010d0 0x7f82281034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 --2- 172.21.15.130:0/2843602974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8228103a00 0x7f8228105df0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 -- 172.21.15.130:0/2843602974 >> 172.21.15.130:0/2843602974 conn(0x7f82280faa60 msgr2=0x7f82280fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:21.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 -- 172.21.15.130:0/2843602974 shutdown_connections 2023-09-12T16:02:21.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 -- 172.21.15.130:0/2843602974 wait complete. 2023-09-12T16:02:21.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 Processor -- start 2023-09-12T16:02:21.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.915+0000 7f822e908700 1 -- start start 2023-09-12T16:02:21.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822e908700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82281010d0 0x7f82281962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822e908700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8228103a00 0x7f8228196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822e908700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8228106330 0x7f822819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822e908700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822819de50 con 0x7f82281010d0 2023-09-12T16:02:21.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822e908700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822819df90 con 0x7f8228106330 2023-09-12T16:02:21.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822e908700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822819e120 con 0x7f8228103a00 2023-09-12T16:02:21.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f8227fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82281010d0 0x7f82281962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f8227fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82281010d0 0x7f82281962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53672/0 (socket says 172.21.15.130:53672) 2023-09-12T16:02:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f8227fff700 1 -- 172.21.15.130:0/1831404233 learned_addr learned my addr 172.21.15.130:0/1831404233 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f82277fe700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8228103a00 0x7f8228196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822cea5700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8228106330 0x7f822819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f8227fff700 1 -- 172.21.15.130:0/1831404233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8228103a00 msgr2=0x7f8228196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f8227fff700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8228103a00 0x7f8228196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f8227fff700 1 -- 172.21.15.130:0/1831404233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8228106330 msgr2=0x7f822819d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f8227fff700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8228106330 0x7f822819d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f8227fff700 1 -- 172.21.15.130:0/1831404233 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82180095f0 con 0x7f82281010d0 2023-09-12T16:02:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822cea5700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8228106330 0x7f822819d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f8227fff700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82281010d0 0x7f82281962d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9a0 tx=0x7f821c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f82257fa700 1 -- 172.21.15.130:0/1831404233 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c009790 con 0x7f82281010d0 2023-09-12T16:02:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f82257fa700 1 -- 172.21.15.130:0/1831404233 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f821c00cbd0 con 0x7f82281010d0 2023-09-12T16:02:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822819e400 con 0x7f82281010d0 2023-09-12T16:02:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f82257fa700 1 -- 172.21.15.130:0/1831404233 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c018680 con 0x7f82281010d0 2023-09-12T16:02:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.916+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822819e910 con 0x7f82281010d0 2023-09-12T16:02:21.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.917+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82280fc5e0 con 0x7f82281010d0 2023-09-12T16:02:21.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.919+0000 7f82257fa700 1 -- 172.21.15.130:0/1831404233 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f821c00cd40 con 0x7f82281010d0 2023-09-12T16:02:21.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.919+0000 7f82257fa700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f82100a0640 0x7f82100a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:21.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.919+0000 7f82257fa700 1 -- 172.21.15.130:0/1831404233 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f821c014070 con 0x7f82281010d0 2023-09-12T16:02:21.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.919+0000 7f82277fe700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f82100a0640 0x7f82100a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:21.928 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.920+0000 7f82277fe700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f82100a0640 0x7f82100a2b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f821800b2e0 tx=0x7f8218005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:21.928 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:21.922+0000 7f82257fa700 1 -- 172.21.15.130:0/1831404233 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f821c096260 con 0x7f82281010d0 2023-09-12T16:02:21.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:21 smithi159 ceph-mon[102522]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:21.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:21 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2209906694' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-12T16:02:21.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:21 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2342929045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:21.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:21 smithi203 ceph-mon[103431]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-12T16:02:21.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:21 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2209906694' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-12T16:02:21.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:21 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2342929045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:22.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.159+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f822810c0c0 con 0x7f82281010d0 2023-09-12T16:02:22.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.620+0000 7f82257fa700 1 -- 172.21.15.130:0/1831404233 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f821c0b9070 con 0x7f82281010d0 2023-09-12T16:02:22.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f82100a0640 msgr2=0x7f82100a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:22.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f82100a0640 0x7f82100a2b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f821800b2e0 tx=0x7f8218005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82281010d0 msgr2=0x7f82281962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:22.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82281010d0 0x7f82281962d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9a0 tx=0x7f821c00ecb0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 shutdown_connections 2023-09-12T16:02:22.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f82100a0640 0x7f82100a2b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8228106330 0x7f822819d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82281010d0 0x7f82281962d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 --2- 172.21.15.130:0/1831404233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8228103a00 0x7f8228196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 >> 172.21.15.130:0/1831404233 conn(0x7f82280faa60 msgr2=0x7f8228101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:22.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 shutdown_connections 2023-09-12T16:02:22.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.623+0000 7f822e908700 1 -- 172.21.15.130:0/1831404233 wait complete. 2023-09-12T16:02:22.637 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-09-12T16:02:22.638 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-09-12T16:02:22.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.763+0000 7ff42423f700 1 -- 172.21.15.130:0/1742109437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c100660 msgr2=0x7ff41c100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:22.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.763+0000 7ff42423f700 1 --2- 172.21.15.130:0/1742109437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c100660 0x7ff41c100a80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0099c0 tx=0x7ff40c009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.763+0000 7ff42423f700 1 -- 172.21.15.130:0/1742109437 shutdown_connections 2023-09-12T16:02:22.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.763+0000 7ff42423f700 1 --2- 172.21.15.130:0/1742109437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff41c1021f0 0x7ff41c108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.763+0000 7ff42423f700 1 --2- 172.21.15.130:0/1742109437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c100660 0x7ff41c100a80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.763+0000 7ff42423f700 1 --2- 172.21.15.130:0/1742109437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41c101850 0x7ff41c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.763+0000 7ff42423f700 1 -- 172.21.15.130:0/1742109437 >> 172.21.15.130:0/1742109437 conn(0x7ff41c0fbbd0 msgr2=0x7ff41c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:22.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.763+0000 7ff42423f700 1 -- 172.21.15.130:0/1742109437 shutdown_connections 2023-09-12T16:02:22.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.763+0000 7ff42423f700 1 -- 172.21.15.130:0/1742109437 wait complete. 2023-09-12T16:02:22.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff42423f700 1 Processor -- start 2023-09-12T16:02:22.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff42423f700 1 -- start start 2023-09-12T16:02:22.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff42423f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41c100660 0x7ff41c0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:22.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff42423f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c101850 0x7ff41c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:22.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff42423f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff41c1021f0 0x7ff41c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:22.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff42423f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0736d0 con 0x7ff41c101850 2023-09-12T16:02:22.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff42423f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c073840 con 0x7ff41c1021f0 2023-09-12T16:02:22.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff42423f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c1a0320 con 0x7ff41c100660 2023-09-12T16:02:22.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff4217da700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c101850 0x7ff41c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:22.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff4217da700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c101850 0x7ff41c0729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53688/0 (socket says 172.21.15.130:53688) 2023-09-12T16:02:22.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff4217da700 1 -- 172.21.15.130:0/4053031182 learned_addr learned my addr 172.21.15.130:0/4053031182 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:22.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.764+0000 7ff4227dc700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff41c1021f0 0x7ff41c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:22.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff421fdb700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41c100660 0x7ff41c0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:22.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff4217da700 1 -- 172.21.15.130:0/4053031182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41c100660 msgr2=0x7ff41c0743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:22.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff4217da700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41c100660 0x7ff41c0743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff4217da700 1 -- 172.21.15.130:0/4053031182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff41c1021f0 msgr2=0x7ff41c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:22.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff4217da700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff41c1021f0 0x7ff41c072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:22.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff4217da700 1 -- 172.21.15.130:0/4053031182 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0095f0 con 0x7ff41c101850 2023-09-12T16:02:22.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff4217da700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c101850 0x7ff41c0729f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff41800d980 tx=0x7ff41800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:22.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff412ffd700 1 -- 172.21.15.130:0/4053031182 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418004220 con 0x7ff41c101850 2023-09-12T16:02:22.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff412ffd700 1 -- 172.21.15.130:0/4053031182 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff418007640 con 0x7ff41c101850 2023-09-12T16:02:22.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c1a0600 con 0x7ff41c101850 2023-09-12T16:02:22.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff412ffd700 1 -- 172.21.15.130:0/4053031182 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418005610 con 0x7ff41c101850 2023-09-12T16:02:22.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.765+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c1a0b10 con 0x7ff41c101850 2023-09-12T16:02:22.773 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.766+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c10be60 con 0x7ff41c101850 2023-09-12T16:02:22.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.768+0000 7ff412ffd700 1 -- 172.21.15.130:0/4053031182 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7ff418004750 con 0x7ff41c101850 2023-09-12T16:02:22.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.768+0000 7ff412ffd700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4080a0640 0x7ff4080a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:22.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.768+0000 7ff421fdb700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4080a0640 0x7ff4080a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:22.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.768+0000 7ff412ffd700 1 -- 172.21.15.130:0/4053031182 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff4180cc120 con 0x7ff41c101850 2023-09-12T16:02:22.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.769+0000 7ff421fdb700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4080a0640 0x7ff4080a2b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00b580 tx=0x7ff40c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:22.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:22.772+0000 7ff412ffd700 1 -- 172.21.15.130:0/4053031182 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff418095a60 con 0x7ff41c101850 2023-09-12T16:02:22.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:22 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3669452388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:22.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:22 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1831404233' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-12T16:02:22.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:22 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3669452388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:22.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:22 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1831404233' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-12T16:02:22.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:22 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3669452388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:22.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:22 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1831404233' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-12T16:02:23.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.008+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7ff41c107560 con 0x7ff41c101850 2023-09-12T16:02:23.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.626+0000 7ff412ffd700 1 -- 172.21.15.130:0/4053031182 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff418099080 con 0x7ff41c101850 2023-09-12T16:02:23.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4080a0640 msgr2=0x7ff4080a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:23.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4080a0640 0x7ff4080a2b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00b580 tx=0x7ff40c005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c101850 msgr2=0x7ff41c0729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:23.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c101850 0x7ff41c0729f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff41800d980 tx=0x7ff41800dc90 comp rx=0 tx=0).stop 2023-09-12T16:02:23.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 shutdown_connections 2023-09-12T16:02:23.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4080a0640 0x7ff4080a2b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff41c1021f0 0x7ff41c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41c100660 0x7ff41c0743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 --2- 172.21.15.130:0/4053031182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff41c101850 0x7ff41c0729f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 >> 172.21.15.130:0/4053031182 conn(0x7ff41c0fbbd0 msgr2=0x7ff41c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:23.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 shutdown_connections 2023-09-12T16:02:23.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.629+0000 7ff42423f700 1 -- 172.21.15.130:0/4053031182 wait complete. 2023-09-12T16:02:23.646 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-09-12T16:02:23.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.773+0000 7f4c8988e700 1 -- 172.21.15.130:0/3802119237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c840ff7a0 msgr2=0x7f4c840ffc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:23.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.773+0000 7f4c8988e700 1 --2- 172.21.15.130:0/3802119237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c840ff7a0 0x7f4c840ffc00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009990 tx=0x7f4c74009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.773+0000 7f4c8988e700 1 -- 172.21.15.130:0/3802119237 shutdown_connections 2023-09-12T16:02:23.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.773+0000 7f4c8988e700 1 --2- 172.21.15.130:0/3802119237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84100140 0x7f4c84107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.773+0000 7f4c8988e700 1 --2- 172.21.15.130:0/3802119237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c84100b00 0x7f4c84100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.773+0000 7f4c8988e700 1 --2- 172.21.15.130:0/3802119237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c840ff7a0 0x7f4c840ffc00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.773+0000 7f4c8988e700 1 -- 172.21.15.130:0/3802119237 >> 172.21.15.130:0/3802119237 conn(0x7f4c840fa1f0 msgr2=0x7f4c840fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:23.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.773+0000 7f4c8988e700 1 -- 172.21.15.130:0/3802119237 shutdown_connections 2023-09-12T16:02:23.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.773+0000 7f4c8988e700 1 -- 172.21.15.130:0/3802119237 wait complete. 2023-09-12T16:02:23.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c8988e700 1 Processor -- start 2023-09-12T16:02:23.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c8988e700 1 -- start start 2023-09-12T16:02:23.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c8988e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c840ff7a0 0x7f4c840ff220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:23.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c8988e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c84100140 0x7f4c8419b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:23.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c8988e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84100b00 0x7f4c8419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:23.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c8988e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8419de60 con 0x7f4c84100140 2023-09-12T16:02:23.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c8988e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8419dfd0 con 0x7f4c84100b00 2023-09-12T16:02:23.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c8988e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8419e190 con 0x7f4c840ff7a0 2023-09-12T16:02:23.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c827fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c84100140 0x7f4c8419b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:23.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c827fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c84100140 0x7f4c8419b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53702/0 (socket says 172.21.15.130:53702) 2023-09-12T16:02:23.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c827fc700 1 -- 172.21.15.130:0/2313024020 learned_addr learned my addr 172.21.15.130:0/2313024020 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:23.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c82ffd700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c840ff7a0 0x7f4c840ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:23.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c837fe700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84100b00 0x7f4c8419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:23.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c827fc700 1 -- 172.21.15.130:0/2313024020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c840ff7a0 msgr2=0x7f4c840ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:23.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c827fc700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c840ff7a0 0x7f4c840ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c827fc700 1 -- 172.21.15.130:0/2313024020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84100b00 msgr2=0x7f4c8419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:23.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c827fc700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84100b00 0x7f4c8419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:23.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.774+0000 7f4c827fc700 1 -- 172.21.15.130:0/2313024020 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740095f0 con 0x7f4c84100140 2023-09-12T16:02:23.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.775+0000 7f4c827fc700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c84100140 0x7f4c8419b040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4c74005910 tx=0x7f4c7400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:23.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.775+0000 7f4c8888c700 1 -- 172.21.15.130:0/2313024020 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7401d070 con 0x7f4c84100140 2023-09-12T16:02:23.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.775+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8419e410 con 0x7f4c84100140 2023-09-12T16:02:23.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.775+0000 7f4c8888c700 1 -- 172.21.15.130:0/2313024020 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c74022470 con 0x7f4c84100140 2023-09-12T16:02:23.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.775+0000 7f4c8888c700 1 -- 172.21.15.130:0/2313024020 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74017650 con 0x7f4c84100140 2023-09-12T16:02:23.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.775+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8419e830 con 0x7f4c84100140 2023-09-12T16:02:23.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.776+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c84106b60 con 0x7f4c84100140 2023-09-12T16:02:23.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.777+0000 7f4c8888c700 1 -- 172.21.15.130:0/2313024020 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f4c7400fad0 con 0x7f4c84100140 2023-09-12T16:02:23.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.777+0000 7f4c8888c700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4c6c0a0640 0x7f4c6c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:23.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.777+0000 7f4c8888c700 1 -- 172.21.15.130:0/2313024020 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4c740cd4e0 con 0x7f4c84100140 2023-09-12T16:02:23.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.777+0000 7f4c82ffd700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4c6c0a0640 0x7f4c6c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:23.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.778+0000 7f4c82ffd700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4c6c0a0640 0x7f4c6c0a2b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4c84068710 tx=0x7f4c7800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:23.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:23.781+0000 7f4c8888c700 1 -- 172.21.15.130:0/2313024020 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4c74096eb0 con 0x7f4c84100140 2023-09-12T16:02:23.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:23 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1831404233' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-12T16:02:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:23 smithi130 ceph-mon[98670]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:23 smithi130 ceph-mon[98670]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 267 B/s wr, 0 op/s 2023-09-12T16:02:23.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:23 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4053031182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-12T16:02:23.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:23 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1831404233' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-12T16:02:23.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:23 smithi159 ceph-mon[102522]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:23.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:23 smithi159 ceph-mon[102522]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 267 B/s wr, 0 op/s 2023-09-12T16:02:23.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:23 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4053031182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-12T16:02:23.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:23 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1831404233' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-12T16:02:23.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:23 smithi203 ceph-mon[103431]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:23.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:23 smithi203 ceph-mon[103431]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 267 B/s wr, 0 op/s 2023-09-12T16:02:23.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:23 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4053031182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-12T16:02:24.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.019+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f4c8402cf70 con 0x7f4c84100140 2023-09-12T16:02:24.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.632+0000 7f4c8888c700 1 -- 172.21.15.130:0/2313024020 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f4c7409a4d0 con 0x7f4c84100140 2023-09-12T16:02:24.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4c6c0a0640 msgr2=0x7f4c6c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:24.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4c6c0a0640 0x7f4c6c0a2b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4c84068710 tx=0x7f4c7800b410 comp rx=0 tx=0).stop 2023-09-12T16:02:24.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c84100140 msgr2=0x7f4c8419b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:24.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c84100140 0x7f4c8419b040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4c74005910 tx=0x7f4c7400f690 comp rx=0 tx=0).stop 2023-09-12T16:02:24.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 shutdown_connections 2023-09-12T16:02:24.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4c6c0a0640 0x7f4c6c0a2b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84100b00 0x7f4c8419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c840ff7a0 0x7f4c840ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 --2- 172.21.15.130:0/2313024020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c84100140 0x7f4c8419b040 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 >> 172.21.15.130:0/2313024020 conn(0x7f4c840fa1f0 msgr2=0x7f4c84103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:24.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 shutdown_connections 2023-09-12T16:02:24.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.635+0000 7f4c8988e700 1 -- 172.21.15.130:0/2313024020 wait complete. 2023-09-12T16:02:24.652 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-09-12T16:02:24.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 -- 172.21.15.130:0/2622470791 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d400ff500 msgr2=0x7f8d400ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:24.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/2622470791 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d400ff500 0x7f8d400ff920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8d300099c0 tx=0x7f8d30009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 -- 172.21.15.130:0/2622470791 shutdown_connections 2023-09-12T16:02:24.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/2622470791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40101090 0x7f8d40107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/2622470791 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d400ff500 0x7f8d400ff920 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/2622470791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d401006f0 0x7f8d40100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 -- 172.21.15.130:0/2622470791 >> 172.21.15.130:0/2622470791 conn(0x7f8d400faa90 msgr2=0x7f8d400fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:24.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 -- 172.21.15.130:0/2622470791 shutdown_connections 2023-09-12T16:02:24.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 -- 172.21.15.130:0/2622470791 wait complete. 2023-09-12T16:02:24.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 Processor -- start 2023-09-12T16:02:24.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.778+0000 7f8d44a5d700 1 -- start start 2023-09-12T16:02:24.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d44a5d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ff500 0x7f8d40195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:24.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d44a5d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d401006f0 0x7f8d401957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:24.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d44a5d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40101090 0x7f8d4019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:24.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d44a5d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4019ce00 con 0x7f8d40101090 2023-09-12T16:02:24.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d44a5d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4019cf40 con 0x7f8d401006f0 2023-09-12T16:02:24.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d44a5d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4019d0d0 con 0x7f8d400ff500 2023-09-12T16:02:24.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3ed9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40101090 0x7f8d4019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:24.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3ed9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40101090 0x7f8d4019c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53716/0 (socket says 172.21.15.130:53716) 2023-09-12T16:02:24.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3ed9d700 1 -- 172.21.15.130:0/3834988212 learned_addr learned my addr 172.21.15.130:0/3834988212 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:24.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3e59c700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ff500 0x7f8d40195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:24.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3dd9b700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d401006f0 0x7f8d401957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:24.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3ed9d700 1 -- 172.21.15.130:0/3834988212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ff500 msgr2=0x7f8d40195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:24.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3ed9d700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ff500 0x7f8d40195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3ed9d700 1 -- 172.21.15.130:0/3834988212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d401006f0 msgr2=0x7f8d401957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:24.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3ed9d700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d401006f0 0x7f8d401957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:24.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3ed9d700 1 -- 172.21.15.130:0/3834988212 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300095f0 con 0x7f8d40101090 2023-09-12T16:02:24.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d3ed9d700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40101090 0x7f8d4019c830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ada0 tx=0x7f8d3800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:24.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d2f7fe700 1 -- 172.21.15.130:0/3834988212 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800f070 con 0x7f8d40101090 2023-09-12T16:02:24.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4019d3b0 con 0x7f8d40101090 2023-09-12T16:02:24.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d2f7fe700 1 -- 172.21.15.130:0/3834988212 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d38007b30 con 0x7f8d40101090 2023-09-12T16:02:24.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d2f7fe700 1 -- 172.21.15.130:0/3834988212 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380053e0 con 0x7f8d40101090 2023-09-12T16:02:24.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.779+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4019d870 con 0x7f8d40101090 2023-09-12T16:02:24.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.780+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4010ad30 con 0x7f8d40101090 2023-09-12T16:02:24.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.781+0000 7f8d2f7fe700 1 -- 172.21.15.130:0/3834988212 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f8d3800c980 con 0x7f8d40101090 2023-09-12T16:02:24.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.782+0000 7f8d2f7fe700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d280a0640 0x7f8d280a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:24.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.782+0000 7f8d2f7fe700 1 -- 172.21.15.130:0/3834988212 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8d380cb310 con 0x7f8d40101090 2023-09-12T16:02:24.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.782+0000 7f8d3e59c700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d280a0640 0x7f8d280a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:24.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.782+0000 7f8d3e59c700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d280a0640 0x7f8d280a2b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009990 tx=0x7f8d300052c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:24.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:24.786+0000 7f8d2f7fe700 1 -- 172.21.15.130:0/3834988212 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8d38094ce0 con 0x7f8d40101090 2023-09-12T16:02:24.942 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:24 smithi130 ceph-mon[98670]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-12T16:02:24.942 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:24 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4053031182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-12T16:02:24.942 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:24 smithi130 ceph-mon[98670]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-12T16:02:24.943 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:24 smithi130 ceph-mon[98670]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:24.943 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:24 smithi130 ceph-mon[98670]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-12T16:02:24.943 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:24 smithi130 ceph-mon[98670]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-12T16:02:24.943 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:24 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2313024020' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-12T16:02:24.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:24 smithi159 ceph-mon[102522]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-12T16:02:24.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:24 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4053031182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-12T16:02:24.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:24 smithi159 ceph-mon[102522]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-12T16:02:24.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:24 smithi159 ceph-mon[102522]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:24.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:24 smithi159 ceph-mon[102522]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-12T16:02:24.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:24 smithi159 ceph-mon[102522]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-12T16:02:24.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:24 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2313024020' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-12T16:02:24.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:24 smithi203 ceph-mon[103431]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-12T16:02:24.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:24 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4053031182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-12T16:02:24.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:24 smithi203 ceph-mon[103431]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-12T16:02:24.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:24 smithi203 ceph-mon[103431]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-12T16:02:24.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:24 smithi203 ceph-mon[103431]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-12T16:02:24.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:24 smithi203 ceph-mon[103431]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-12T16:02:24.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:24 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2313024020' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-12T16:02:25.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.026+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f8d4010aef0 con 0x7f8d40101090 2023-09-12T16:02:25.475 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:02:24 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-b[139180]: 2023-09-12T16:02:24.991+0000 7f005ca0a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-09-12T16:02:25.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.642+0000 7f8d2f7fe700 1 -- 172.21.15.130:0/3834988212 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f8d38098300 con 0x7f8d40101090 2023-09-12T16:02:25.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.644+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d280a0640 msgr2=0x7f8d280a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:25.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.644+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d280a0640 0x7f8d280a2b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009990 tx=0x7f8d300052c0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.644+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40101090 msgr2=0x7f8d4019c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:25.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.644+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40101090 0x7f8d4019c830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ada0 tx=0x7f8d3800c330 comp rx=0 tx=0).stop 2023-09-12T16:02:25.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.645+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 shutdown_connections 2023-09-12T16:02:25.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.645+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8d280a0640 0x7f8d280a2b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.645+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d40101090 0x7f8d4019c830 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.645+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ff500 0x7f8d40195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.645+0000 7f8d44a5d700 1 --2- 172.21.15.130:0/3834988212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d401006f0 0x7f8d401957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.645+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 >> 172.21.15.130:0/3834988212 conn(0x7f8d400faa90 msgr2=0x7f8d400fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:25.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.645+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 shutdown_connections 2023-09-12T16:02:25.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.645+0000 7f8d44a5d700 1 -- 172.21.15.130:0/3834988212 wait complete. 2023-09-12T16:02:25.659 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-09-12T16:02:25.660 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-09-12T16:02:25.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.786+0000 7f263c910700 1 -- 172.21.15.130:0/1597076536 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 msgr2=0x7f26381034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:25.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.786+0000 7f263c910700 1 --2- 172.21.15.130:0/1597076536 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 0x7f26381034c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f262c009990 tx=0x7f262c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.786+0000 7f263c910700 1 -- 172.21.15.130:0/1597076536 shutdown_connections 2023-09-12T16:02:25.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.786+0000 7f263c910700 1 --2- 172.21.15.130:0/1597076536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2638106330 0x7f2638108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.786+0000 7f263c910700 1 --2- 172.21.15.130:0/1597076536 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 0x7f26381034c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.786+0000 7f263c910700 1 --2- 172.21.15.130:0/1597076536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2638103a00 0x7f2638105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.786+0000 7f263c910700 1 -- 172.21.15.130:0/1597076536 >> 172.21.15.130:0/1597076536 conn(0x7f26380faa60 msgr2=0x7f26380fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:25.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.786+0000 7f263c910700 1 -- 172.21.15.130:0/1597076536 shutdown_connections 2023-09-12T16:02:25.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.786+0000 7f263c910700 1 -- 172.21.15.130:0/1597076536 wait complete. 2023-09-12T16:02:25.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263c910700 1 Processor -- start 2023-09-12T16:02:25.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263c910700 1 -- start start 2023-09-12T16:02:25.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263c910700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 0x7f2638196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:25.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263c910700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2638103a00 0x7f2638196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:25.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263c910700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2638106330 0x7f263819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:25.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263c910700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263819de80 con 0x7f26381010d0 2023-09-12T16:02:25.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263c910700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263819dfc0 con 0x7f2638103a00 2023-09-12T16:02:25.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263c910700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263819e150 con 0x7f2638106330 2023-09-12T16:02:25.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 0x7f2638196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:25.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 0x7f2638196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53726/0 (socket says 172.21.15.130:53726) 2023-09-12T16:02:25.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263659c700 1 -- 172.21.15.130:0/521299596 learned_addr learned my addr 172.21.15.130:0/521299596 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:25.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f2636d9d700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2638106330 0x7f263819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:25.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f2635d9b700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2638103a00 0x7f2638196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:25.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263659c700 1 -- 172.21.15.130:0/521299596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2638106330 msgr2=0x7f263819d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:25.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263659c700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2638106330 0x7f263819d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263659c700 1 -- 172.21.15.130:0/521299596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2638103a00 msgr2=0x7f2638196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:25.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.787+0000 7f263659c700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2638103a00 0x7f2638196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:25.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.788+0000 7f263659c700 1 -- 172.21.15.130:0/521299596 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0095f0 con 0x7f26381010d0 2023-09-12T16:02:25.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.788+0000 7f2635d9b700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2638103a00 0x7f2638196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:25.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.788+0000 7f263659c700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 0x7f2638196300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f262c000c00 tx=0x7f262c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:25.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.788+0000 7f26277fe700 1 -- 172.21.15.130:0/521299596 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c01d070 con 0x7f26381010d0 2023-09-12T16:02:25.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.788+0000 7f26277fe700 1 -- 172.21.15.130:0/521299596 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f262c00fb50 con 0x7f26381010d0 2023-09-12T16:02:25.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.788+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263819e3d0 con 0x7f26381010d0 2023-09-12T16:02:25.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.788+0000 7f26277fe700 1 -- 172.21.15.130:0/521299596 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c017650 con 0x7f26381010d0 2023-09-12T16:02:25.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.788+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263819e880 con 0x7f26381010d0 2023-09-12T16:02:25.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.789+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26380fc5e0 con 0x7f26381010d0 2023-09-12T16:02:25.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.790+0000 7f26277fe700 1 -- 172.21.15.130:0/521299596 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f262c00fcc0 con 0x7f26381010d0 2023-09-12T16:02:25.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.790+0000 7f26277fe700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26200a0640 0x7f26200a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:25.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.791+0000 7f2635d9b700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26200a0640 0x7f26200a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:25.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.791+0000 7f26277fe700 1 -- 172.21.15.130:0/521299596 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f262c0cd4c0 con 0x7f26381010d0 2023-09-12T16:02:25.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.791+0000 7f2635d9b700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26200a0640 0x7f26200a2b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2628005950 tx=0x7f26280058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:25.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:25.794+0000 7f26277fe700 1 -- 172.21.15.130:0/521299596 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f262c096e30 con 0x7f26381010d0 2023-09-12T16:02:25.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2313024020' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-12T16:02:25.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] up:active 2023-09-12T16:02:25.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-12T16:02:25.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-12T16:02:25.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: Cluster is now healthy 2023-09-12T16:02:25.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-12T16:02:25.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-09-12T16:02:25.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-09-12T16:02:25.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: pgmap v273: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-12T16:02:25.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:25 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3834988212' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-12T16:02:25.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2313024020' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-12T16:02:25.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] up:active 2023-09-12T16:02:25.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-12T16:02:25.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-12T16:02:25.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: Cluster is now healthy 2023-09-12T16:02:25.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-12T16:02:25.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-09-12T16:02:25.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-09-12T16:02:25.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: pgmap v273: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-12T16:02:25.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:25 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3834988212' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-12T16:02:25.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2313024020' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-12T16:02:25.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] up:active 2023-09-12T16:02:25.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-12T16:02:25.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-12T16:02:25.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: Cluster is now healthy 2023-09-12T16:02:25.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-12T16:02:25.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-09-12T16:02:25.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-09-12T16:02:25.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: pgmap v273: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-12T16:02:25.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:25 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3834988212' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-12T16:02:26.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.016+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f263802cf50 con 0x7f26200a0640 2023-09-12T16:02:26.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.060+0000 7f26277fe700 1 -- 172.21.15.130:0/521299596 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f263802cf50 con 0x7f26200a0640 2023-09-12T16:02:26.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26200a0640 msgr2=0x7f26200a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26200a0640 0x7f26200a2b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2628005950 tx=0x7f26280058e0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 msgr2=0x7f2638196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 0x7f2638196300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f262c000c00 tx=0x7f262c00f710 comp rx=0 tx=0).stop 2023-09-12T16:02:26.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 shutdown_connections 2023-09-12T16:02:26.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26200a0640 0x7f26200a2b00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2638106330 0x7f263819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26381010d0 0x7f2638196300 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 --2- 172.21.15.130:0/521299596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2638103a00 0x7f2638196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 >> 172.21.15.130:0/521299596 conn(0x7f26380faa60 msgr2=0x7f2638101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:26.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 shutdown_connections 2023-09-12T16:02:26.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.062+0000 7f263c910700 1 -- 172.21.15.130:0/521299596 wait complete. 2023-09-12T16:02:26.075 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-09-12T16:02:26.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 -- 172.21.15.130:0/2890213071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed00ff4f0 msgr2=0x7fbed00ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/2890213071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed00ff4f0 0x7fbed00ff910 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0099c0 tx=0x7fbecc009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 -- 172.21.15.130:0/2890213071 shutdown_connections 2023-09-12T16:02:26.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/2890213071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed0101080 0x7fbed0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/2890213071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed00ff4f0 0x7fbed00ff910 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/2890213071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed01006e0 0x7fbed0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 -- 172.21.15.130:0/2890213071 >> 172.21.15.130:0/2890213071 conn(0x7fbed00faa60 msgr2=0x7fbed00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:26.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 -- 172.21.15.130:0/2890213071 shutdown_connections 2023-09-12T16:02:26.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 -- 172.21.15.130:0/2890213071 wait complete. 2023-09-12T16:02:26.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 Processor -- start 2023-09-12T16:02:26.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 -- start start 2023-09-12T16:02:26.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed00ff4f0 0x7fbed0103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed01006e0 0x7fbed0101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed0101080 0x7fbed0102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed01027a0 con 0x7fbed0101080 2023-09-12T16:02:26.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed0102910 con 0x7fbed00ff4f0 2023-09-12T16:02:26.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.199+0000 7fbed7a7b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed01a0380 con 0x7fbed01006e0 2023-09-12T16:02:26.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed6018700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed0101080 0x7fbed0102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed6018700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed0101080 0x7fbed0102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53764/0 (socket says 172.21.15.130:53764) 2023-09-12T16:02:26.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed6018700 1 -- 172.21.15.130:0/936333354 learned_addr learned my addr 172.21.15.130:0/936333354 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:26.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed5016700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed01006e0 0x7fbed0101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed5817700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed00ff4f0 0x7fbed0103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed6018700 1 -- 172.21.15.130:0/936333354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed01006e0 msgr2=0x7fbed0101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed6018700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed01006e0 0x7fbed0101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed6018700 1 -- 172.21.15.130:0/936333354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed00ff4f0 msgr2=0x7fbed0103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed6018700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed00ff4f0 0x7fbed0103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed6018700 1 -- 172.21.15.130:0/936333354 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0095f0 con 0x7fbed0101080 2023-09-12T16:02:26.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed5817700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed00ff4f0 0x7fbed0103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:26.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed6018700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed0101080 0x7fbed0102000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbec8007ee0 tx=0x7fbec800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:26.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbec6ffd700 1 -- 172.21.15.130:0/936333354 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800ddd0 con 0x7fbed0101080 2023-09-12T16:02:26.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed01a0660 con 0x7fbed0101080 2023-09-12T16:02:26.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbec6ffd700 1 -- 172.21.15.130:0/936333354 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbec800f040 con 0x7fbed0101080 2023-09-12T16:02:26.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbec6ffd700 1 -- 172.21.15.130:0/936333354 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80048e0 con 0x7fbed0101080 2023-09-12T16:02:26.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.200+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed01a0b20 con 0x7fbed0101080 2023-09-12T16:02:26.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.201+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed010ad00 con 0x7fbed0101080 2023-09-12T16:02:26.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.202+0000 7fbec6ffd700 1 -- 172.21.15.130:0/936333354 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fbec8003a00 con 0x7fbed0101080 2023-09-12T16:02:26.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.203+0000 7fbec6ffd700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbebc0a0640 0x7fbebc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.203+0000 7fbed5817700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbebc0a0640 0x7fbebc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.203+0000 7fbec6ffd700 1 -- 172.21.15.130:0/936333354 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbec80cb710 con 0x7fbed0101080 2023-09-12T16:02:26.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.203+0000 7fbed5817700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbebc0a0640 0x7fbebc0a2b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009990 tx=0x7fbecc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:26.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.207+0000 7fbec6ffd700 1 -- 172.21.15.130:0/936333354 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbec804ee10 con 0x7fbed0101080 2023-09-12T16:02:26.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.433+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fbed0068ee0 con 0x7fbebc0a0640 2023-09-12T16:02:26.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.437+0000 7fbec6ffd700 1 -- 172.21.15.130:0/936333354 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fbed0068ee0 con 0x7fbebc0a0640 2023-09-12T16:02:26.438 INFO:teuthology.orchestra.run.smithi130.stdout:/volumes/_nogroup/sv_0/317f330f-760b-4a97-a24e-21dab804b73f 2023-09-12T16:02:26.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbebc0a0640 msgr2=0x7fbebc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbebc0a0640 0x7fbebc0a2b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009990 tx=0x7fbecc01a040 comp rx=0 tx=0).stop 2023-09-12T16:02:26.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed0101080 msgr2=0x7fbed0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed0101080 0x7fbed0102000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbec8007ee0 tx=0x7fbec800d3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 shutdown_connections 2023-09-12T16:02:26.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fbebc0a0640 0x7fbebc0a2b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbed0101080 0x7fbed0102000 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed00ff4f0 0x7fbed0103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 --2- 172.21.15.130:0/936333354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed01006e0 0x7fbed0101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 >> 172.21.15.130:0/936333354 conn(0x7fbed00faa60 msgr2=0x7fbed00687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:26.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 shutdown_connections 2023-09-12T16:02:26.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.439+0000 7fbed7a7b700 1 -- 172.21.15.130:0/936333354 wait complete. 2023-09-12T16:02:26.451 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T16:02:26.021+0000 7fca860f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-12T16:02:26.451 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T16:02:26.021+0000 7fca860f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-12T16:02:26.451 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T16:02:26.021+0000 7fca860f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-12T16:02:26.452 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T16:02:26.021+0000 7fca860f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-12T16:02:26.452 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x[98913]: 2023-09-12T16:02:26.021+0000 7fca860f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-12T16:02:26.453 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-09-12T16:02:26.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.577+0000 7f3d78e21700 1 -- 172.21.15.130:0/666307872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 msgr2=0x7f3d74108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.577+0000 7f3d78e21700 1 --2- 172.21.15.130:0/666307872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 0x7f3d74108a60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00b0d0 tx=0x7f3d6c00b3e0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.578+0000 7f3d78e21700 1 -- 172.21.15.130:0/666307872 shutdown_connections 2023-09-12T16:02:26.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.578+0000 7f3d78e21700 1 --2- 172.21.15.130:0/666307872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 0x7f3d74108a60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.578+0000 7f3d78e21700 1 --2- 172.21.15.130:0/666307872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d74100620 0x7f3d74100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.578+0000 7f3d78e21700 1 --2- 172.21.15.130:0/666307872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d74101810 0x7f3d74101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.578+0000 7f3d78e21700 1 -- 172.21.15.130:0/666307872 >> 172.21.15.130:0/666307872 conn(0x7f3d740fbb70 msgr2=0x7f3d740fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:26.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.578+0000 7f3d78e21700 1 -- 172.21.15.130:0/666307872 shutdown_connections 2023-09-12T16:02:26.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.578+0000 7f3d78e21700 1 -- 172.21.15.130:0/666307872 wait complete. 2023-09-12T16:02:26.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.578+0000 7f3d78e21700 1 Processor -- start 2023-09-12T16:02:26.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.578+0000 7f3d78e21700 1 -- start start 2023-09-12T16:02:26.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d78e21700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d74100620 0x7f3d740743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d78e21700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d74101810 0x7f3d740729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d78e21700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 0x7f3d74072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d78e21700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d740736d0 con 0x7f3d741021b0 2023-09-12T16:02:26.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d78e21700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74073840 con 0x7f3d74100620 2023-09-12T16:02:26.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d78e21700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d741a0380 con 0x7f3d74101810 2023-09-12T16:02:26.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d72d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 0x7f3d74072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d72d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 0x7f3d74072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53776/0 (socket says 172.21.15.130:53776) 2023-09-12T16:02:26.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d72d9d700 1 -- 172.21.15.130:0/1080611760 learned_addr learned my addr 172.21.15.130:0/1080611760 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:26.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d7259c700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d74100620 0x7f3d740743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d71d9b700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d74101810 0x7f3d740729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d72d9d700 1 -- 172.21.15.130:0/1080611760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d74101810 msgr2=0x7f3d740729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d72d9d700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d74101810 0x7f3d740729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d72d9d700 1 -- 172.21.15.130:0/1080611760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d74100620 msgr2=0x7f3d740743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d72d9d700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d74100620 0x7f3d740743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d72d9d700 1 -- 172.21.15.130:0/1080611760 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c009d00 con 0x7f3d741021b0 2023-09-12T16:02:26.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d7259c700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d74100620 0x7f3d740743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:26.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d72d9d700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 0x7f3d74072f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c007a00 tx=0x7f3d6c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:26.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d637fe700 1 -- 172.21.15.130:0/1080611760 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00e040 con 0x7f3d741021b0 2023-09-12T16:02:26.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d741a0600 con 0x7f3d741021b0 2023-09-12T16:02:26.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d637fe700 1 -- 172.21.15.130:0/1080611760 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d6c003e10 con 0x7f3d741021b0 2023-09-12T16:02:26.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d637fe700 1 -- 172.21.15.130:0/1080611760 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c005610 con 0x7f3d741021b0 2023-09-12T16:02:26.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.579+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d741a0a80 con 0x7f3d741021b0 2023-09-12T16:02:26.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.580+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7410be40 con 0x7f3d741021b0 2023-09-12T16:02:26.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.581+0000 7f3d637fe700 1 -- 172.21.15.130:0/1080611760 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f3d6c005770 con 0x7f3d741021b0 2023-09-12T16:02:26.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.582+0000 7f3d637fe700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d5c0a0640 0x7f3d5c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.582+0000 7f3d637fe700 1 -- 172.21.15.130:0/1080611760 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3d6c0cd4e0 con 0x7f3d741021b0 2023-09-12T16:02:26.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.582+0000 7f3d7259c700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d5c0a0640 0x7f3d5c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.582+0000 7f3d7259c700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d5c0a0640 0x7f3d5c0a2b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3d74101670 tx=0x7f3d6400a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:26.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.586+0000 7f3d637fe700 1 -- 172.21.15.130:0/1080611760 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3d6c096eb0 con 0x7f3d741021b0 2023-09-12T16:02:26.724 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3834988212' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-12T16:02:26.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] up:active 2023-09-12T16:02:26.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-09-12T16:02:26.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:02:26.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:26 smithi130 ceph-mon[98670]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:26.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.813+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d74060a40 con 0x7f3d5c0a0640 2023-09-12T16:02:26.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.839+0000 7f3d637fe700 1 -- 172.21.15.130:0/1080611760 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3d74060a40 con 0x7f3d5c0a0640 2023-09-12T16:02:26.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d5c0a0640 msgr2=0x7f3d5c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d5c0a0640 0x7f3d5c0a2b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3d74101670 tx=0x7f3d6400a680 comp rx=0 tx=0).stop 2023-09-12T16:02:26.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 msgr2=0x7f3d74072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 0x7f3d74072f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c007a00 tx=0x7f3d6c009570 comp rx=0 tx=0).stop 2023-09-12T16:02:26.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 shutdown_connections 2023-09-12T16:02:26.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3d5c0a0640 0x7f3d5c0a2b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d741021b0 0x7f3d74072f30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d74100620 0x7f3d740743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.844 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 --2- 172.21.15.130:0/1080611760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d74101810 0x7f3d740729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.844 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 >> 172.21.15.130:0/1080611760 conn(0x7f3d740fbb70 msgr2=0x7f3d740fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:26.844 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 shutdown_connections 2023-09-12T16:02:26.844 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.841+0000 7f3d78e21700 1 -- 172.21.15.130:0/1080611760 wait complete. 2023-09-12T16:02:26.854 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-09-12T16:02:26.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:26 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3834988212' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-12T16:02:26.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:26 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] up:active 2023-09-12T16:02:26.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:26 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-09-12T16:02:26.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:26 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:02:26.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:26 smithi159 ceph-mon[102522]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:26.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 -- 172.21.15.130:0/970403918 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1881006e0 msgr2=0x7fe188100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/970403918 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1881006e0 0x7fe188100b40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe1780099c0 tx=0x7fe178009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 -- 172.21.15.130:0/970403918 shutdown_connections 2023-09-12T16:02:26.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/970403918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe188101080 0x7fe188107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/970403918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1880ff4f0 0x7fe1880ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/970403918 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1881006e0 0x7fe188100b40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 -- 172.21.15.130:0/970403918 >> 172.21.15.130:0/970403918 conn(0x7fe1880faa60 msgr2=0x7fe1880fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:26.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 -- 172.21.15.130:0/970403918 shutdown_connections 2023-09-12T16:02:26.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 -- 172.21.15.130:0/970403918 wait complete. 2023-09-12T16:02:26.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.979+0000 7fe18d9c6700 1 Processor -- start 2023-09-12T16:02:26.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe18d9c6700 1 -- start start 2023-09-12T16:02:26.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe18d9c6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1880ff4f0 0x7fe188103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe18d9c6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1881006e0 0x7fe188101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe18d9c6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe188101080 0x7fe188102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe18d9c6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1881027a0 con 0x7fe1880ff4f0 2023-09-12T16:02:26.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe18d9c6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe188102910 con 0x7fe188101080 2023-09-12T16:02:26.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe18d9c6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1881a03e0 con 0x7fe1881006e0 2023-09-12T16:02:26.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe186ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1880ff4f0 0x7fe188103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe186ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1880ff4f0 0x7fe188103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53788/0 (socket says 172.21.15.130:53788) 2023-09-12T16:02:26.986 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe186ffd700 1 -- 172.21.15.130:0/1687133530 learned_addr learned my addr 172.21.15.130:0/1687133530 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:26.986 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe1877fe700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe188101080 0x7fe188102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.986 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe1867fc700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1881006e0 0x7fe188101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.986 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe186ffd700 1 -- 172.21.15.130:0/1687133530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1881006e0 msgr2=0x7fe188101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.986 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe186ffd700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1881006e0 0x7fe188101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe186ffd700 1 -- 172.21.15.130:0/1687133530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe188101080 msgr2=0x7fe188102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:26.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe186ffd700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe188101080 0x7fe188102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:26.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe186ffd700 1 -- 172.21.15.130:0/1687133530 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1780095f0 con 0x7fe1880ff4f0 2023-09-12T16:02:26.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.980+0000 7fe186ffd700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1880ff4f0 0x7fe188103480 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c970 tx=0x7fe17c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:26.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.981+0000 7fe18c9c4700 1 -- 172.21.15.130:0/1687133530 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c004220 con 0x7fe1880ff4f0 2023-09-12T16:02:26.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.981+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1881a06c0 con 0x7fe1880ff4f0 2023-09-12T16:02:26.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.981+0000 7fe18c9c4700 1 -- 172.21.15.130:0/1687133530 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe17c007d60 con 0x7fe1880ff4f0 2023-09-12T16:02:26.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.981+0000 7fe18c9c4700 1 -- 172.21.15.130:0/1687133530 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c003c90 con 0x7fe1880ff4f0 2023-09-12T16:02:26.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.981+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1881a0bd0 con 0x7fe1880ff4f0 2023-09-12T16:02:26.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.982+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18810ad10 con 0x7fe1880ff4f0 2023-09-12T16:02:26.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.983+0000 7fe18c9c4700 1 -- 172.21.15.130:0/1687133530 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fe17c004380 con 0x7fe1880ff4f0 2023-09-12T16:02:26.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.983+0000 7fe18c9c4700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1700a0640 0x7fe1700a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:26.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.983+0000 7fe1867fc700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1700a0640 0x7fe1700a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:26.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.983+0000 7fe18c9c4700 1 -- 172.21.15.130:0/1687133530 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe17c0cb950 con 0x7fe1880ff4f0 2023-09-12T16:02:26.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.984+0000 7fe1867fc700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1700a0640 0x7fe1700a2b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe178003680 tx=0x7fe178005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:26.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:26.987+0000 7fe18c9c4700 1 -- 172.21.15.130:0/1687133530 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe17c095290 con 0x7fe1880ff4f0 2023-09-12T16:02:26.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:26 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3834988212' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-12T16:02:26.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:26 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] up:active 2023-09-12T16:02:26.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:26 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-09-12T16:02:26.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:26 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:02:26.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:26 smithi203 ceph-mon[103431]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:27.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.215+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe188067e70 con 0x7fe1700a0640 2023-09-12T16:02:27.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.219+0000 7fe18c9c4700 1 -- 172.21.15.130:0/1687133530 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fe188067e70 con 0x7fe1700a0640 2023-09-12T16:02:27.219 INFO:teuthology.orchestra.run.smithi130.stdout:/volumes/_nogroup/sv_1/47d66d2e-a22e-4bfa-ac90-69257256aa66 2023-09-12T16:02:27.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1700a0640 msgr2=0x7fe1700a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1700a0640 0x7fe1700a2b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe178003680 tx=0x7fe178005a90 comp rx=0 tx=0).stop 2023-09-12T16:02:27.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1880ff4f0 msgr2=0x7fe188103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1880ff4f0 0x7fe188103480 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c970 tx=0x7fe17c00cc80 comp rx=0 tx=0).stop 2023-09-12T16:02:27.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 shutdown_connections 2023-09-12T16:02:27.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1700a0640 0x7fe1700a2b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe188101080 0x7fe188102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1880ff4f0 0x7fe188103480 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 --2- 172.21.15.130:0/1687133530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1881006e0 0x7fe188101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 >> 172.21.15.130:0/1687133530 conn(0x7fe1880faa60 msgr2=0x7fe188067760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:27.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 shutdown_connections 2023-09-12T16:02:27.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.221+0000 7fe18d9c6700 1 -- 172.21.15.130:0/1687133530 wait complete. 2023-09-12T16:02:27.234 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:02:27.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.356+0000 7fe205959700 1 -- 172.21.15.130:0/434124359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200102300 msgr2=0x7fe200108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.356+0000 7fe205959700 1 --2- 172.21.15.130:0/434124359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200102300 0x7fe200108b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800b0a0 tx=0x7fe1f800b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.356+0000 7fe205959700 1 -- 172.21.15.130:0/434124359 shutdown_connections 2023-09-12T16:02:27.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.356+0000 7fe205959700 1 --2- 172.21.15.130:0/434124359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200102300 0x7fe200108b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.356+0000 7fe205959700 1 --2- 172.21.15.130:0/434124359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2000fe920 0x7fe2000fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.356+0000 7fe205959700 1 --2- 172.21.15.130:0/434124359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200101940 0x7fe200101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.356+0000 7fe205959700 1 -- 172.21.15.130:0/434124359 >> 172.21.15.130:0/434124359 conn(0x7fe2000fa200 msgr2=0x7fe2000fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:27.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.356+0000 7fe205959700 1 -- 172.21.15.130:0/434124359 shutdown_connections 2023-09-12T16:02:27.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.356+0000 7fe205959700 1 -- 172.21.15.130:0/434124359 wait complete. 2023-09-12T16:02:27.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe205959700 1 Processor -- start 2023-09-12T16:02:27.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe205959700 1 -- start start 2023-09-12T16:02:27.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe205959700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2000fe920 0x7fe200196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:27.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe205959700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200101940 0x7fe200196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:27.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe205959700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200102300 0x7fe20019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:27.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe205959700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019ded0 con 0x7fe200101940 2023-09-12T16:02:27.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe205959700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019e010 con 0x7fe200102300 2023-09-12T16:02:27.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe205959700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019e1a0 con 0x7fe2000fe920 2023-09-12T16:02:27.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe1fe7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200101940 0x7fe200196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:27.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe1fe7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200101940 0x7fe200196890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53802/0 (socket says 172.21.15.130:53802) 2023-09-12T16:02:27.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe1fe7fc700 1 -- 172.21.15.130:0/2908679365 learned_addr learned my addr 172.21.15.130:0/2908679365 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:27.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe1feffd700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2000fe920 0x7fe200196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:27.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe1ff7fe700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200102300 0x7fe20019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:27.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe1fe7fc700 1 -- 172.21.15.130:0/2908679365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2000fe920 msgr2=0x7fe200196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe1fe7fc700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2000fe920 0x7fe200196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.357+0000 7fe1fe7fc700 1 -- 172.21.15.130:0/2908679365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200102300 msgr2=0x7fe20019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.358+0000 7fe1fe7fc700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200102300 0x7fe20019d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.358+0000 7fe1fe7fc700 1 -- 172.21.15.130:0/2908679365 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f8009d00 con 0x7fe200101940 2023-09-12T16:02:27.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.358+0000 7fe1ff7fe700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200102300 0x7fe20019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:27.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.358+0000 7fe1fe7fc700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200101940 0x7fe200196890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800d950 tx=0x7fe1e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:27.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.358+0000 7fe204957700 1 -- 172.21.15.130:0/2908679365 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8009940 con 0x7fe200101940 2023-09-12T16:02:27.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.358+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20019e480 con 0x7fe200101940 2023-09-12T16:02:27.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.358+0000 7fe204957700 1 -- 172.21.15.130:0/2908679365 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1e8010460 con 0x7fe200101940 2023-09-12T16:02:27.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.358+0000 7fe204957700 1 -- 172.21.15.130:0/2908679365 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800f620 con 0x7fe200101940 2023-09-12T16:02:27.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.358+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20019e9e0 con 0x7fe200101940 2023-09-12T16:02:27.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.359+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2001075b0 con 0x7fe200101940 2023-09-12T16:02:27.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.360+0000 7fe204957700 1 -- 172.21.15.130:0/2908679365 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fe1e80108d0 con 0x7fe200101940 2023-09-12T16:02:27.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.360+0000 7fe204957700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec0a0690 0x7fe1ec0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:27.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.361+0000 7fe204957700 1 -- 172.21.15.130:0/2908679365 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe1e80cbf90 con 0x7fe200101940 2023-09-12T16:02:27.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.361+0000 7fe1feffd700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec0a0690 0x7fe1ec0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:27.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.361+0000 7fe1feffd700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec0a0690 0x7fe1ec0a2b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe200068740 tx=0x7fe1f000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:27.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.364+0000 7fe204957700 1 -- 172.21.15.130:0/2908679365 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe1e80958d0 con 0x7fe200101940 2023-09-12T16:02:27.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.600+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe20002cf10 con 0x7fe200101940 2023-09-12T16:02:27.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.600+0000 7fe204957700 1 -- 172.21.15.130:0/2908679365 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11342 (secure 0 0 0) 0x7fe1e8098ef0 con 0x7fe200101940 2023-09-12T16:02:27.601 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:27.602 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:25.646573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:02:27.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec0a0690 msgr2=0x7fe1ec0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec0a0690 0x7fe1ec0a2b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe200068740 tx=0x7fe1f000a380 comp rx=0 tx=0).stop 2023-09-12T16:02:27.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200101940 msgr2=0x7fe200196890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200101940 0x7fe200196890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800d950 tx=0x7fe1e800dc60 comp rx=0 tx=0).stop 2023-09-12T16:02:27.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 shutdown_connections 2023-09-12T16:02:27.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe1ec0a0690 0x7fe1ec0a2b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe200102300 0x7fe20019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2000fe920 0x7fe200196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 --2- 172.21.15.130:0/2908679365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe200101940 0x7fe200196890 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 >> 172.21.15.130:0/2908679365 conn(0x7fe2000fa200 msgr2=0x7fe2000fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:27.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 shutdown_connections 2023-09-12T16:02:27.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.603+0000 7fe205959700 1 -- 172.21.15.130:0/2908679365 wait complete. 2023-09-12T16:02:27.607 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 24 2023-09-12T16:02:27.616 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:02:27.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.742+0000 7f6159eba700 1 -- 172.21.15.130:0/3813610075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 msgr2=0x7f61540ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.742+0000 7f6159eba700 1 --2- 172.21.15.130:0/3813610075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 0x7f61540ff900 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f61480099c0 tx=0x7f6148009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.742+0000 7f6159eba700 1 -- 172.21.15.130:0/3813610075 shutdown_connections 2023-09-12T16:02:27.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.742+0000 7f6159eba700 1 --2- 172.21.15.130:0/3813610075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6154101070 0x7f6154107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.742+0000 7f6159eba700 1 --2- 172.21.15.130:0/3813610075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 0x7f61540ff900 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.742+0000 7f6159eba700 1 --2- 172.21.15.130:0/3813610075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61541006d0 0x7f6154100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.742+0000 7f6159eba700 1 -- 172.21.15.130:0/3813610075 >> 172.21.15.130:0/3813610075 conn(0x7f61540faa30 msgr2=0x7f61540fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:27.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.742+0000 7f6159eba700 1 -- 172.21.15.130:0/3813610075 shutdown_connections 2023-09-12T16:02:27.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.742+0000 7f6159eba700 1 -- 172.21.15.130:0/3813610075 wait complete. 2023-09-12T16:02:27.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6159eba700 1 Processor -- start 2023-09-12T16:02:27.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6159eba700 1 -- start start 2023-09-12T16:02:27.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6159eba700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 0x7f61540682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:27.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6159eba700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61541006d0 0x7f615419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:27.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6159eba700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6154101070 0x7f615419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:27.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6159eba700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615419dde0 con 0x7f61540ff4e0 2023-09-12T16:02:27.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6159eba700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615419df50 con 0x7f6154101070 2023-09-12T16:02:27.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6159eba700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615419e110 con 0x7f61541006d0 2023-09-12T16:02:27.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f61537fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 0x7f61540682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:27.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f61537fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 0x7f61540682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53814/0 (socket says 172.21.15.130:53814) 2023-09-12T16:02:27.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f61537fe700 1 -- 172.21.15.130:0/3978389283 learned_addr learned my addr 172.21.15.130:0/3978389283 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:27.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6152ffd700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61541006d0 0x7f615419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:27.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.743+0000 7f6153fff700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6154101070 0x7f615419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:27.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f61537fe700 1 -- 172.21.15.130:0/3978389283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61541006d0 msgr2=0x7f615419b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f61537fe700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61541006d0 0x7f615419b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f61537fe700 1 -- 172.21.15.130:0/3978389283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6154101070 msgr2=0x7f615419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f61537fe700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6154101070 0x7f615419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f61537fe700 1 -- 172.21.15.130:0/3978389283 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61480095f0 con 0x7f61540ff4e0 2023-09-12T16:02:27.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f61537fe700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 0x7f61540682d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6148009990 tx=0x7f6148005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:27.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f6150ff9700 1 -- 172.21.15.130:0/3978389283 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614801d070 con 0x7f61540ff4e0 2023-09-12T16:02:27.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f6150ff9700 1 -- 172.21.15.130:0/3978389283 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6148004200 con 0x7f61540ff4e0 2023-09-12T16:02:27.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615419e390 con 0x7f61540ff4e0 2023-09-12T16:02:27.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f6150ff9700 1 -- 172.21.15.130:0/3978389283 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148022e30 con 0x7f61540ff4e0 2023-09-12T16:02:27.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.744+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615419e840 con 0x7f61540ff4e0 2023-09-12T16:02:27.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.745+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6154106460 con 0x7f61540ff4e0 2023-09-12T16:02:27.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.746+0000 7f6150ff9700 1 -- 172.21.15.130:0/3978389283 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f6148003d40 con 0x7f61540ff4e0 2023-09-12T16:02:27.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.746+0000 7f6150ff9700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f613c0a0710 0x7f613c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:27.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.747+0000 7f6152ffd700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f613c0a0710 0x7f613c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:27.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.747+0000 7f6150ff9700 1 -- 172.21.15.130:0/3978389283 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f61480cd1e0 con 0x7f61540ff4e0 2023-09-12T16:02:27.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.747+0000 7f6152ffd700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f613c0a0710 0x7f613c0a2bd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6144005950 tx=0x7f61440058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:27.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.751+0000 7f6150ff9700 1 -- 172.21.15.130:0/3978389283 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6148096a50 con 0x7f61540ff4e0 2023-09-12T16:02:27.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.944+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f615410af40 con 0x7f61540ff4e0 2023-09-12T16:02:27.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.945+0000 7f6150ff9700 1 -- 172.21.15.130:0/3978389283 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24004 (secure 0 0 0) 0x7f614809a070 con 0x7f61540ff4e0 2023-09-12T16:02:27.946 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:27.946 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":86,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:11.262527+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:27.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.947+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f613c0a0710 msgr2=0x7f613c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.947+0000 7f6159eba700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f613c0a0710 0x7f613c0a2bd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6144005950 tx=0x7f61440058e0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.947+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 msgr2=0x7f61540682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.947+0000 7f6159eba700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 0x7f61540682d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6148009990 tx=0x7f6148005e80 comp rx=0 tx=0).stop 2023-09-12T16:02:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.948+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 shutdown_connections 2023-09-12T16:02:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.948+0000 7f6159eba700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f613c0a0710 0x7f613c0a2bd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.948+0000 7f6159eba700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6154101070 0x7f615419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.948+0000 7f6159eba700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61540ff4e0 0x7f61540682d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.948+0000 7f6159eba700 1 --2- 172.21.15.130:0/3978389283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61541006d0 0x7f615419b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:27.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.948+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 >> 172.21.15.130:0/3978389283 conn(0x7f61540faa30 msgr2=0x7f61540fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:27.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.948+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 shutdown_connections 2023-09-12T16:02:27.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:27.948+0000 7f6159eba700 1 -- 172.21.15.130:0/3978389283 wait complete. 2023-09-12T16:02:27.959 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:27 smithi130 ceph-mon[98670]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:27.960 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:27 smithi130 ceph-mon[98670]: pgmap v274: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.4 KiB/s wr, 4 op/s 2023-09-12T16:02:27.960 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:27 smithi130 ceph-mon[98670]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:27.960 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:27 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2908679365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:27.962 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-12T16:02:27.962 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T16:02:27.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:27 smithi159 ceph-mon[102522]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:27 smithi159 ceph-mon[102522]: pgmap v274: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.4 KiB/s wr, 4 op/s 2023-09-12T16:02:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:27 smithi159 ceph-mon[102522]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:27 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2908679365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:27 smithi203 ceph-mon[103431]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:27 smithi203 ceph-mon[103431]: pgmap v274: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.4 KiB/s wr, 4 op/s 2023-09-12T16:02:27.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:27 smithi203 ceph-mon[103431]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:27.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:27 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2908679365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:02:28.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.085+0000 7fef27fe2700 1 -- 172.21.15.130:0/3689486369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 msgr2=0x7fef20100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.085+0000 7fef27fe2700 1 --2- 172.21.15.130:0/3689486369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 0x7fef20100a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef100099c0 tx=0x7fef10009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.085+0000 7fef27fe2700 1 -- 172.21.15.130:0/3689486369 shutdown_connections 2023-09-12T16:02:28.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.085+0000 7fef27fe2700 1 --2- 172.21.15.130:0/3689486369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef201021a0 0x7fef20108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.085+0000 7fef27fe2700 1 --2- 172.21.15.130:0/3689486369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 0x7fef20100a30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.085+0000 7fef27fe2700 1 --2- 172.21.15.130:0/3689486369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef20101800 0x7fef20101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.085+0000 7fef27fe2700 1 -- 172.21.15.130:0/3689486369 >> 172.21.15.130:0/3689486369 conn(0x7fef200fbb40 msgr2=0x7fef200fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:28.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.085+0000 7fef27fe2700 1 -- 172.21.15.130:0/3689486369 shutdown_connections 2023-09-12T16:02:28.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.085+0000 7fef27fe2700 1 -- 172.21.15.130:0/3689486369 wait complete. 2023-09-12T16:02:28.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef27fe2700 1 Processor -- start 2023-09-12T16:02:28.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef27fe2700 1 -- start start 2023-09-12T16:02:28.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef27fe2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 0x7fef20196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:28.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef27fe2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef20101800 0x7fef20196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:28.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef27fe2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef201021a0 0x7fef2019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:28.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef27fe2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2019ddd0 con 0x7fef20100610 2023-09-12T16:02:28.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef27fe2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2019df10 con 0x7fef20101800 2023-09-12T16:02:28.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef27fe2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2019e0a0 con 0x7fef201021a0 2023-09-12T16:02:28.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef25d7e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 0x7fef20196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:28.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef25d7e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 0x7fef20196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53828/0 (socket says 172.21.15.130:53828) 2023-09-12T16:02:28.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef25d7e700 1 -- 172.21.15.130:0/677772230 learned_addr learned my addr 172.21.15.130:0/677772230 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:28.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef2557d700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef20101800 0x7fef20196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:28.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef25d7e700 1 -- 172.21.15.130:0/677772230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef201021a0 msgr2=0x7fef2019d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef2657f700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef201021a0 0x7fef2019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:28.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef25d7e700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef201021a0 0x7fef2019d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef25d7e700 1 -- 172.21.15.130:0/677772230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef20101800 msgr2=0x7fef20196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef25d7e700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef20101800 0x7fef20196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef25d7e700 1 -- 172.21.15.130:0/677772230 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100095f0 con 0x7fef20100610 2023-09-12T16:02:28.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef2557d700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef20101800 0x7fef20196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:28.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef2657f700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef201021a0 0x7fef2019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:28.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.086+0000 7fef25d7e700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 0x7fef20196250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fef10009990 tx=0x7fef10003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:28.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.087+0000 7fef16ffd700 1 -- 172.21.15.130:0/677772230 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1001d070 con 0x7fef20100610 2023-09-12T16:02:28.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.087+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2019e320 con 0x7fef20100610 2023-09-12T16:02:28.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.087+0000 7fef16ffd700 1 -- 172.21.15.130:0/677772230 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef10004640 con 0x7fef20100610 2023-09-12T16:02:28.096 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.087+0000 7fef16ffd700 1 -- 172.21.15.130:0/677772230 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1000f460 con 0x7fef20100610 2023-09-12T16:02:28.096 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.087+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2019e7d0 con 0x7fef20100610 2023-09-12T16:02:28.096 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.088+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2010be40 con 0x7fef20100610 2023-09-12T16:02:28.097 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.088+0000 7fef16ffd700 1 -- 172.21.15.130:0/677772230 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fef10004050 con 0x7fef20100610 2023-09-12T16:02:28.097 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.089+0000 7fef16ffd700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fef0c0a0710 0x7fef0c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:28.098 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.089+0000 7fef2557d700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fef0c0a0710 0x7fef0c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:28.098 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.089+0000 7fef16ffd700 1 -- 172.21.15.130:0/677772230 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fef100cd460 con 0x7fef20100610 2023-09-12T16:02:28.098 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.089+0000 7fef2557d700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fef0c0a0710 0x7fef0c0a2bd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005fd0 tx=0x7fef1c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:28.098 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.093+0000 7fef16ffd700 1 -- 172.21.15.130:0/677772230 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fef10096cd0 con 0x7fef20100610 2023-09-12T16:02:28.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.281+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef2019e9f0 con 0x7fef0c0a0710 2023-09-12T16:02:28.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.293+0000 7fef16ffd700 1 -- 172.21.15.130:0/677772230 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418770 (secure 0 0 0) 0x7fef2019e9f0 con 0x7fef0c0a0710 2023-09-12T16:02:28.299 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:28.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.297+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fef0c0a0710 msgr2=0x7fef0c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.297+0000 7fef27fe2700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fef0c0a0710 0x7fef0c0a2bd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005fd0 tx=0x7fef1c005e30 comp rx=0 tx=0).stop 2023-09-12T16:02:28.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 msgr2=0x7fef20196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 0x7fef20196250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fef10009990 tx=0x7fef10003ba0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 shutdown_connections 2023-09-12T16:02:28.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fef0c0a0710 0x7fef0c0a2bd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef201021a0 0x7fef2019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20100610 0x7fef20196250 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 --2- 172.21.15.130:0/677772230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef20101800 0x7fef20196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 >> 172.21.15.130:0/677772230 conn(0x7fef200fbb40 msgr2=0x7fef200fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:28.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 shutdown_connections 2023-09-12T16:02:28.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.298+0000 7fef27fe2700 1 -- 172.21.15.130:0/677772230 wait complete. 2023-09-12T16:02:28.305 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2023-09-12T16:02:28.310 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-09-12T16:02:26.760271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595904,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":131,"num_read_kb":116,"num_write":271,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335388,"kb_used_data":11164,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606436,"statfs":{"total":1151940427776,"available":1151596990464,"internally_reserved":0,"allocated":11431936,"data_stored":7374017,"data_compressed":283014,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3126,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":7,"num_write":45,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003104"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892578+0000","last_change":"2023-09-12T16:02:05.039617+0000","last_active":"2023-09-12T16:02:10.892578+0000","last_peered":"2023-09-12T16:02:10.892578+0000","last_clean":"2023-09-12T16:02:10.892578+0000","last_became_active":"2023-09-12T16:02:05.039264+0000","last_became_peered":"2023-09-12T16:02:05.039264+0000","last_unstale":"2023-09-12T16:02:10.892578+0000","last_undegraded":"2023-09-12T16:02:10.892578+0000","last_fullsized":"2023-09-12T16:02:10.892578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:37:01.381406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891691+0000","last_change":"2023-09-12T16:02:05.033151+0000","last_active":"2023-09-12T16:02:10.891691+0000","last_peered":"2023-09-12T16:02:10.891691+0000","last_clean":"2023-09-12T16:02:10.891691+0000","last_became_active":"2023-09-12T16:02:05.032866+0000","last_became_peered":"2023-09-12T16:02:05.032866+0000","last_unstale":"2023-09-12T16:02:10.891691+0000","last_undegraded":"2023-09-12T16:02:10.891691+0000","last_fullsized":"2023-09-12T16:02:10.891691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:35.921428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891781+0000","last_change":"2023-09-12T16:02:05.032478+0000","last_active":"2023-09-12T16:02:10.891781+0000","last_peered":"2023-09-12T16:02:10.891781+0000","last_clean":"2023-09-12T16:02:10.891781+0000","last_became_active":"2023-09-12T16:02:05.032124+0000","last_became_peered":"2023-09-12T16:02:05.032124+0000","last_unstale":"2023-09-12T16:02:10.891781+0000","last_undegraded":"2023-09-12T16:02:10.891781+0000","last_fullsized":"2023-09-12T16:02:10.891781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:58:54.112577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760684+0000","last_change":"2023-09-12T16:02:05.033088+0000","last_active":"2023-09-12T16:02:10.760684+0000","last_peered":"2023-09-12T16:02:10.760684+0000","last_clean":"2023-09-12T16:02:10.760684+0000","last_became_active":"2023-09-12T16:02:05.032332+0000","last_became_peered":"2023-09-12T16:02:05.032332+0000","last_unstale":"2023-09-12T16:02:10.760684+0000","last_undegraded":"2023-09-12T16:02:10.760684+0000","last_fullsized":"2023-09-12T16:02:10.760684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:25:38.751961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305466+0000","last_change":"2023-09-12T16:02:05.036612+0000","last_active":"2023-09-12T16:02:10.305466+0000","last_peered":"2023-09-12T16:02:10.305466+0000","last_clean":"2023-09-12T16:02:10.305466+0000","last_became_active":"2023-09-12T16:02:05.036215+0000","last_became_peered":"2023-09-12T16:02:05.036215+0000","last_unstale":"2023-09-12T16:02:10.305466+0000","last_undegraded":"2023-09-12T16:02:10.305466+0000","last_fullsized":"2023-09-12T16:02:10.305466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:15:04.561751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891896+0000","last_change":"2023-09-12T16:02:05.043816+0000","last_active":"2023-09-12T16:02:10.891896+0000","last_peered":"2023-09-12T16:02:10.891896+0000","last_clean":"2023-09-12T16:02:10.891896+0000","last_became_active":"2023-09-12T16:02:05.043599+0000","last_became_peered":"2023-09-12T16:02:05.043599+0000","last_unstale":"2023-09-12T16:02:10.891896+0000","last_undegraded":"2023-09-12T16:02:10.891896+0000","last_fullsized":"2023-09-12T16:02:10.891896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:32:00.286196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265184+0000","last_change":"2023-09-12T16:02:05.031476+0000","last_active":"2023-09-12T16:02:10.265184+0000","last_peered":"2023-09-12T16:02:10.265184+0000","last_clean":"2023-09-12T16:02:10.265184+0000","last_became_active":"2023-09-12T16:02:05.031150+0000","last_became_peered":"2023-09-12T16:02:05.031150+0000","last_unstale":"2023-09-12T16:02:10.265184+0000","last_undegraded":"2023-09-12T16:02:10.265184+0000","last_fullsized":"2023-09-12T16:02:10.265184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:22:27.405505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.627779+0000","last_change":"2023-09-12T16:02:05.031981+0000","last_active":"2023-09-12T16:02:10.627779+0000","last_peered":"2023-09-12T16:02:10.627779+0000","last_clean":"2023-09-12T16:02:10.627779+0000","last_became_active":"2023-09-12T16:02:05.031692+0000","last_became_peered":"2023-09-12T16:02:05.031692+0000","last_unstale":"2023-09-12T16:02:10.627779+0000","last_undegraded":"2023-09-12T16:02:10.627779+0000","last_fullsized":"2023-09-12T16:02:10.627779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:51:00.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573730+0000","last_change":"2023-09-12T16:02:05.029898+0000","last_active":"2023-09-12T16:02:10.573730+0000","last_peered":"2023-09-12T16:02:10.573730+0000","last_clean":"2023-09-12T16:02:10.573730+0000","last_became_active":"2023-09-12T16:02:05.029731+0000","last_became_peered":"2023-09-12T16:02:05.029731+0000","last_unstale":"2023-09-12T16:02:10.573730+0000","last_undegraded":"2023-09-12T16:02:10.573730+0000","last_fullsized":"2023-09-12T16:02:10.573730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:16:31.194561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892310+0000","last_change":"2023-09-12T16:02:05.044846+0000","last_active":"2023-09-12T16:02:10.892310+0000","last_peered":"2023-09-12T16:02:10.892310+0000","last_clean":"2023-09-12T16:02:10.892310+0000","last_became_active":"2023-09-12T16:02:05.044430+0000","last_became_peered":"2023-09-12T16:02:05.044430+0000","last_unstale":"2023-09-12T16:02:10.892310+0000","last_undegraded":"2023-09-12T16:02:10.892310+0000","last_fullsized":"2023-09-12T16:02:10.892310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:15.999080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265884+0000","last_change":"2023-09-12T16:02:05.030285+0000","last_active":"2023-09-12T16:02:10.265884+0000","last_peered":"2023-09-12T16:02:10.265884+0000","last_clean":"2023-09-12T16:02:10.265884+0000","last_became_active":"2023-09-12T16:02:05.029585+0000","last_became_peered":"2023-09-12T16:02:05.029585+0000","last_unstale":"2023-09-12T16:02:10.265884+0000","last_undegraded":"2023-09-12T16:02:10.265884+0000","last_fullsized":"2023-09-12T16:02:10.265884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:52:27.471412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760652+0000","last_change":"2023-09-12T16:02:05.037258+0000","last_active":"2023-09-12T16:02:10.760652+0000","last_peered":"2023-09-12T16:02:10.760652+0000","last_clean":"2023-09-12T16:02:10.760652+0000","last_became_active":"2023-09-12T16:02:05.036941+0000","last_became_peered":"2023-09-12T16:02:05.036941+0000","last_unstale":"2023-09-12T16:02:10.760652+0000","last_undegraded":"2023-09-12T16:02:10.760652+0000","last_fullsized":"2023-09-12T16:02:10.760652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:04:17.998927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266860+0000","last_change":"2023-09-12T16:02:05.041986+0000","last_active":"2023-09-12T16:02:10.266860+0000","last_peered":"2023-09-12T16:02:10.266860+0000","last_clean":"2023-09-12T16:02:10.266860+0000","last_became_active":"2023-09-12T16:02:05.041584+0000","last_became_peered":"2023-09-12T16:02:05.041584+0000","last_unstale":"2023-09-12T16:02:10.266860+0000","last_undegraded":"2023-09-12T16:02:10.266860+0000","last_fullsized":"2023-09-12T16:02:10.266860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:39.065525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891874+0000","last_change":"2023-09-12T16:02:05.030405+0000","last_active":"2023-09-12T16:02:10.891874+0000","last_peered":"2023-09-12T16:02:10.891874+0000","last_clean":"2023-09-12T16:02:10.891874+0000","last_became_active":"2023-09-12T16:02:05.029488+0000","last_became_peered":"2023-09-12T16:02:05.029488+0000","last_unstale":"2023-09-12T16:02:10.891874+0000","last_undegraded":"2023-09-12T16:02:10.891874+0000","last_fullsized":"2023-09-12T16:02:10.891874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.646069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891352+0000","last_change":"2023-09-12T16:02:05.032576+0000","last_active":"2023-09-12T16:02:10.891352+0000","last_peered":"2023-09-12T16:02:10.891352+0000","last_clean":"2023-09-12T16:02:10.891352+0000","last_became_active":"2023-09-12T16:02:05.032277+0000","last_became_peered":"2023-09-12T16:02:05.032277+0000","last_unstale":"2023-09-12T16:02:10.891352+0000","last_undegraded":"2023-09-12T16:02:10.891352+0000","last_fullsized":"2023-09-12T16:02:10.891352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:47:32.734830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573074+0000","last_change":"2023-09-12T16:02:05.023468+0000","last_active":"2023-09-12T16:02:10.573074+0000","last_peered":"2023-09-12T16:02:10.573074+0000","last_clean":"2023-09-12T16:02:10.573074+0000","last_became_active":"2023-09-12T16:02:05.023132+0000","last_became_peered":"2023-09-12T16:02:05.023132+0000","last_unstale":"2023-09-12T16:02:10.573074+0000","last_undegraded":"2023-09-12T16:02:10.573074+0000","last_fullsized":"2023-09-12T16:02:10.573074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:21:49.378779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892504+0000","last_change":"2023-09-12T16:02:05.039910+0000","last_active":"2023-09-12T16:02:10.892504+0000","last_peered":"2023-09-12T16:02:10.892504+0000","last_clean":"2023-09-12T16:02:10.892504+0000","last_became_active":"2023-09-12T16:02:05.039605+0000","last_became_peered":"2023-09-12T16:02:05.039605+0000","last_unstale":"2023-09-12T16:02:10.892504+0000","last_undegraded":"2023-09-12T16:02:10.892504+0000","last_fullsized":"2023-09-12T16:02:10.892504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:19:58.823471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892624+0000","last_change":"2023-09-12T16:02:05.031667+0000","last_active":"2023-09-12T16:02:10.892624+0000","last_peered":"2023-09-12T16:02:10.892624+0000","last_clean":"2023-09-12T16:02:10.892624+0000","last_became_active":"2023-09-12T16:02:05.030886+0000","last_became_peered":"2023-09-12T16:02:05.030886+0000","last_unstale":"2023-09-12T16:02:10.892624+0000","last_undegraded":"2023-09-12T16:02:10.892624+0000","last_fullsized":"2023-09-12T16:02:10.892624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:24:02.312902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892704+0000","last_change":"2023-09-12T16:02:05.037607+0000","last_active":"2023-09-12T16:02:10.892704+0000","last_peered":"2023-09-12T16:02:10.892704+0000","last_clean":"2023-09-12T16:02:10.892704+0000","last_became_active":"2023-09-12T16:02:05.037095+0000","last_became_peered":"2023-09-12T16:02:05.037095+0000","last_unstale":"2023-09-12T16:02:10.892704+0000","last_undegraded":"2023-09-12T16:02:10.892704+0000","last_fullsized":"2023-09-12T16:02:10.892704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:17:04.886737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266183+0000","last_change":"2023-09-12T16:02:09.292093+0000","last_active":"2023-09-12T16:02:11.266183+0000","last_peered":"2023-09-12T16:02:11.266183+0000","last_clean":"2023-09-12T16:02:11.266183+0000","last_became_active":"2023-09-12T16:02:09.291602+0000","last_became_peered":"2023-09-12T16:02:09.291602+0000","last_unstale":"2023-09-12T16:02:11.266183+0000","last_undegraded":"2023-09-12T16:02:11.266183+0000","last_fullsized":"2023-09-12T16:02:11.266183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:36:23.089498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892761+0000","last_change":"2023-09-12T16:02:04.029311+0000","last_active":"2023-09-12T16:02:10.892761+0000","last_peered":"2023-09-12T16:02:10.892761+0000","last_clean":"2023-09-12T16:02:10.892761+0000","last_became_active":"2023-09-12T16:02:04.028956+0000","last_became_peered":"2023-09-12T16:02:04.028956+0000","last_unstale":"2023-09-12T16:02:10.892761+0000","last_undegraded":"2023-09-12T16:02:10.892761+0000","last_fullsized":"2023-09-12T16:02:10.892761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:51:43.855047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.628092+0000","last_change":"2023-09-12T16:02:05.032191+0000","last_active":"2023-09-12T16:02:10.628092+0000","last_peered":"2023-09-12T16:02:10.628092+0000","last_clean":"2023-09-12T16:02:10.628092+0000","last_became_active":"2023-09-12T16:02:05.031888+0000","last_became_peered":"2023-09-12T16:02:05.031888+0000","last_unstale":"2023-09-12T16:02:10.628092+0000","last_undegraded":"2023-09-12T16:02:10.628092+0000","last_fullsized":"2023-09-12T16:02:10.628092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:50:15.775078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792837+0000","last_change":"2023-09-12T16:02:09.290650+0000","last_active":"2023-09-12T16:02:11.792837+0000","last_peered":"2023-09-12T16:02:11.792837+0000","last_clean":"2023-09-12T16:02:11.792837+0000","last_became_active":"2023-09-12T16:02:09.290205+0000","last_became_peered":"2023-09-12T16:02:09.290205+0000","last_unstale":"2023-09-12T16:02:11.792837+0000","last_undegraded":"2023-09-12T16:02:11.792837+0000","last_fullsized":"2023-09-12T16:02:11.792837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:44:04.930063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.265957+0000","last_change":"2023-09-12T16:02:04.032506+0000","last_active":"2023-09-12T16:02:11.265957+0000","last_peered":"2023-09-12T16:02:11.265957+0000","last_clean":"2023-09-12T16:02:11.265957+0000","last_became_active":"2023-09-12T16:02:04.032047+0000","last_became_peered":"2023-09-12T16:02:04.032047+0000","last_unstale":"2023-09-12T16:02:11.265957+0000","last_undegraded":"2023-09-12T16:02:11.265957+0000","last_fullsized":"2023-09-12T16:02:11.265957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:30:22.664465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891413+0000","last_change":"2023-09-12T16:02:05.025692+0000","last_active":"2023-09-12T16:02:10.891413+0000","last_peered":"2023-09-12T16:02:10.891413+0000","last_clean":"2023-09-12T16:02:10.891413+0000","last_became_active":"2023-09-12T16:02:05.025370+0000","last_became_peered":"2023-09-12T16:02:05.025370+0000","last_unstale":"2023-09-12T16:02:10.891413+0000","last_undegraded":"2023-09-12T16:02:10.891413+0000","last_fullsized":"2023-09-12T16:02:10.891413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.305375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793714+0000","last_change":"2023-09-12T16:02:09.290547+0000","last_active":"2023-09-12T16:02:11.793714+0000","last_peered":"2023-09-12T16:02:11.793714+0000","last_clean":"2023-09-12T16:02:11.793714+0000","last_became_active":"2023-09-12T16:02:09.290120+0000","last_became_peered":"2023-09-12T16:02:09.290120+0000","last_unstale":"2023-09-12T16:02:11.793714+0000","last_undegraded":"2023-09-12T16:02:11.793714+0000","last_fullsized":"2023-09-12T16:02:11.793714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:31:25.364487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.649918+0000","last_change":"2023-09-12T16:02:04.032079+0000","last_active":"2023-09-12T16:02:24.649918+0000","last_peered":"2023-09-12T16:02:24.649918+0000","last_clean":"2023-09-12T16:02:24.649918+0000","last_became_active":"2023-09-12T16:02:04.031835+0000","last_became_peered":"2023-09-12T16:02:04.031835+0000","last_unstale":"2023-09-12T16:02:24.649918+0000","last_undegraded":"2023-09-12T16:02:24.649918+0000","last_fullsized":"2023-09-12T16:02:24.649918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:48:33.935012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573472+0000","last_change":"2023-09-12T16:02:05.037671+0000","last_active":"2023-09-12T16:02:10.573472+0000","last_peered":"2023-09-12T16:02:10.573472+0000","last_clean":"2023-09-12T16:02:10.573472+0000","last_became_active":"2023-09-12T16:02:05.037307+0000","last_became_peered":"2023-09-12T16:02:05.037307+0000","last_unstale":"2023-09-12T16:02:10.573472+0000","last_undegraded":"2023-09-12T16:02:10.573472+0000","last_fullsized":"2023-09-12T16:02:10.573472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:47:23.836612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792372+0000","last_change":"2023-09-12T16:02:09.289165+0000","last_active":"2023-09-12T16:02:11.792372+0000","last_peered":"2023-09-12T16:02:11.792372+0000","last_clean":"2023-09-12T16:02:11.792372+0000","last_became_active":"2023-09-12T16:02:09.288903+0000","last_became_peered":"2023-09-12T16:02:09.288903+0000","last_unstale":"2023-09-12T16:02:11.792372+0000","last_undegraded":"2023-09-12T16:02:11.792372+0000","last_fullsized":"2023-09-12T16:02:11.792372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:27:09.160822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305747+0000","last_change":"2023-09-12T16:02:04.027279+0000","last_active":"2023-09-12T16:02:10.305747+0000","last_peered":"2023-09-12T16:02:10.305747+0000","last_clean":"2023-09-12T16:02:10.305747+0000","last_became_active":"2023-09-12T16:02:04.026975+0000","last_became_peered":"2023-09-12T16:02:04.026975+0000","last_unstale":"2023-09-12T16:02:10.305747+0000","last_undegraded":"2023-09-12T16:02:10.305747+0000","last_fullsized":"2023-09-12T16:02:10.305747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:45.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265479+0000","last_change":"2023-09-12T16:02:05.027582+0000","last_active":"2023-09-12T16:02:10.265479+0000","last_peered":"2023-09-12T16:02:10.265479+0000","last_clean":"2023-09-12T16:02:10.265479+0000","last_became_active":"2023-09-12T16:02:05.027236+0000","last_became_peered":"2023-09-12T16:02:05.027236+0000","last_unstale":"2023-09-12T16:02:10.265479+0000","last_undegraded":"2023-09-12T16:02:10.265479+0000","last_fullsized":"2023-09-12T16:02:10.265479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:06:24.812731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266605+0000","last_change":"2023-09-12T16:02:09.284381+0000","last_active":"2023-09-12T16:02:11.266605+0000","last_peered":"2023-09-12T16:02:11.266605+0000","last_clean":"2023-09-12T16:02:11.266605+0000","last_became_active":"2023-09-12T16:02:09.283947+0000","last_became_peered":"2023-09-12T16:02:09.283947+0000","last_unstale":"2023-09-12T16:02:11.266605+0000","last_undegraded":"2023-09-12T16:02:11.266605+0000","last_fullsized":"2023-09-12T16:02:11.266605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:05:24.803781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"80'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265192+0000","last_change":"2023-09-12T16:02:04.027693+0000","last_active":"2023-09-12T16:02:10.265192+0000","last_peered":"2023-09-12T16:02:10.265192+0000","last_clean":"2023-09-12T16:02:10.265192+0000","last_became_active":"2023-09-12T16:02:04.027315+0000","last_became_peered":"2023-09-12T16:02:04.027315+0000","last_unstale":"2023-09-12T16:02:10.265192+0000","last_undegraded":"2023-09-12T16:02:10.265192+0000","last_fullsized":"2023-09-12T16:02:10.265192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:32:21.756802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305697+0000","last_change":"2023-09-12T16:02:05.031673+0000","last_active":"2023-09-12T16:02:10.305697+0000","last_peered":"2023-09-12T16:02:10.305697+0000","last_clean":"2023-09-12T16:02:10.305697+0000","last_became_active":"2023-09-12T16:02:05.031364+0000","last_became_peered":"2023-09-12T16:02:05.031364+0000","last_unstale":"2023-09-12T16:02:10.305697+0000","last_undegraded":"2023-09-12T16:02:10.305697+0000","last_fullsized":"2023-09-12T16:02:10.305697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:48:05.222530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266486+0000","last_change":"2023-09-12T16:02:09.288536+0000","last_active":"2023-09-12T16:02:11.266486+0000","last_peered":"2023-09-12T16:02:11.266486+0000","last_clean":"2023-09-12T16:02:11.266486+0000","last_became_active":"2023-09-12T16:02:09.288062+0000","last_became_peered":"2023-09-12T16:02:09.288062+0000","last_unstale":"2023-09-12T16:02:11.266486+0000","last_undegraded":"2023-09-12T16:02:11.266486+0000","last_fullsized":"2023-09-12T16:02:11.266486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:39:28.935563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.649762+0000","last_change":"2023-09-12T16:02:04.026298+0000","last_active":"2023-09-12T16:02:24.649762+0000","last_peered":"2023-09-12T16:02:24.649762+0000","last_clean":"2023-09-12T16:02:24.649762+0000","last_became_active":"2023-09-12T16:02:04.025975+0000","last_became_peered":"2023-09-12T16:02:04.025975+0000","last_unstale":"2023-09-12T16:02:24.649762+0000","last_undegraded":"2023-09-12T16:02:24.649762+0000","last_fullsized":"2023-09-12T16:02:24.649762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:12:04.338497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265378+0000","last_change":"2023-09-12T16:02:05.028337+0000","last_active":"2023-09-12T16:02:10.265378+0000","last_peered":"2023-09-12T16:02:10.265378+0000","last_clean":"2023-09-12T16:02:10.265378+0000","last_became_active":"2023-09-12T16:02:05.028036+0000","last_became_peered":"2023-09-12T16:02:05.028036+0000","last_unstale":"2023-09-12T16:02:10.265378+0000","last_undegraded":"2023-09-12T16:02:10.265378+0000","last_fullsized":"2023-09-12T16:02:10.265378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:39:09.387191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793737+0000","last_change":"2023-09-12T16:02:09.290950+0000","last_active":"2023-09-12T16:02:11.793737+0000","last_peered":"2023-09-12T16:02:11.793737+0000","last_clean":"2023-09-12T16:02:11.793737+0000","last_became_active":"2023-09-12T16:02:09.290477+0000","last_became_peered":"2023-09-12T16:02:09.290477+0000","last_unstale":"2023-09-12T16:02:11.793737+0000","last_undegraded":"2023-09-12T16:02:11.793737+0000","last_fullsized":"2023-09-12T16:02:11.793737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:53:45.907575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266543+0000","last_change":"2023-09-12T16:02:04.026025+0000","last_active":"2023-09-12T16:02:10.266543+0000","last_peered":"2023-09-12T16:02:10.266543+0000","last_clean":"2023-09-12T16:02:10.266543+0000","last_became_active":"2023-09-12T16:02:04.025722+0000","last_became_peered":"2023-09-12T16:02:04.025722+0000","last_unstale":"2023-09-12T16:02:10.266543+0000","last_undegraded":"2023-09-12T16:02:10.266543+0000","last_fullsized":"2023-09-12T16:02:10.266543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:29:58.357960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.628351+0000","last_change":"2023-09-12T16:02:05.028461+0000","last_active":"2023-09-12T16:02:10.628351+0000","last_peered":"2023-09-12T16:02:10.628351+0000","last_clean":"2023-09-12T16:02:10.628351+0000","last_became_active":"2023-09-12T16:02:05.028176+0000","last_became_peered":"2023-09-12T16:02:05.028176+0000","last_unstale":"2023-09-12T16:02:10.628351+0000","last_undegraded":"2023-09-12T16:02:10.628351+0000","last_fullsized":"2023-09-12T16:02:10.628351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:16:48.652513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:12.221469+0000","last_change":"2023-09-12T16:02:09.295245+0000","last_active":"2023-09-12T16:02:12.221469+0000","last_peered":"2023-09-12T16:02:12.221469+0000","last_clean":"2023-09-12T16:02:12.221469+0000","last_became_active":"2023-09-12T16:02:09.294872+0000","last_became_peered":"2023-09-12T16:02:09.294872+0000","last_unstale":"2023-09-12T16:02:12.221469+0000","last_undegraded":"2023-09-12T16:02:12.221469+0000","last_fullsized":"2023-09-12T16:02:12.221469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:43:49.462317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.264897+0000","last_change":"2023-09-12T16:02:04.030815+0000","last_active":"2023-09-12T16:02:10.264897+0000","last_peered":"2023-09-12T16:02:10.264897+0000","last_clean":"2023-09-12T16:02:10.264897+0000","last_became_active":"2023-09-12T16:02:04.030435+0000","last_became_peered":"2023-09-12T16:02:04.030435+0000","last_unstale":"2023-09-12T16:02:10.264897+0000","last_undegraded":"2023-09-12T16:02:10.264897+0000","last_fullsized":"2023-09-12T16:02:10.264897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:39:59.661143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891762+0000","last_change":"2023-09-12T16:02:05.031155+0000","last_active":"2023-09-12T16:02:10.891762+0000","last_peered":"2023-09-12T16:02:10.891762+0000","last_clean":"2023-09-12T16:02:10.891762+0000","last_became_active":"2023-09-12T16:02:05.030803+0000","last_became_peered":"2023-09-12T16:02:05.030803+0000","last_unstale":"2023-09-12T16:02:10.891762+0000","last_undegraded":"2023-09-12T16:02:10.891762+0000","last_fullsized":"2023-09-12T16:02:10.891762+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:55:21.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.265570+0000","last_change":"2023-09-12T16:02:09.286445+0000","last_active":"2023-09-12T16:02:11.265570+0000","last_peered":"2023-09-12T16:02:11.265570+0000","last_clean":"2023-09-12T16:02:11.265570+0000","last_became_active":"2023-09-12T16:02:09.286049+0000","last_became_peered":"2023-09-12T16:02:09.286049+0000","last_unstale":"2023-09-12T16:02:11.265570+0000","last_undegraded":"2023-09-12T16:02:11.265570+0000","last_fullsized":"2023-09-12T16:02:11.265570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:43:13.943984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:23.645323+0000","last_change":"2023-09-12T16:02:04.035341+0000","last_active":"2023-09-12T16:02:23.645323+0000","last_peered":"2023-09-12T16:02:23.645323+0000","last_clean":"2023-09-12T16:02:23.645323+0000","last_became_active":"2023-09-12T16:02:04.035092+0000","last_became_peered":"2023-09-12T16:02:04.035092+0000","last_unstale":"2023-09-12T16:02:23.645323+0000","last_undegraded":"2023-09-12T16:02:23.645323+0000","last_fullsized":"2023-09-12T16:02:23.645323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:33:00.784401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573712+0000","last_change":"2023-09-12T16:02:05.025229+0000","last_active":"2023-09-12T16:02:10.573712+0000","last_peered":"2023-09-12T16:02:10.573712+0000","last_clean":"2023-09-12T16:02:10.573712+0000","last_became_active":"2023-09-12T16:02:05.024963+0000","last_became_peered":"2023-09-12T16:02:05.024963+0000","last_unstale":"2023-09-12T16:02:10.573712+0000","last_undegraded":"2023-09-12T16:02:10.573712+0000","last_fullsized":"2023-09-12T16:02:10.573712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:46:04.318082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792883+0000","last_change":"2023-09-12T16:02:09.295011+0000","last_active":"2023-09-12T16:02:11.792883+0000","last_peered":"2023-09-12T16:02:11.792883+0000","last_clean":"2023-09-12T16:02:11.792883+0000","last_became_active":"2023-09-12T16:02:09.294617+0000","last_became_peered":"2023-09-12T16:02:09.294617+0000","last_unstale":"2023-09-12T16:02:11.792883+0000","last_undegraded":"2023-09-12T16:02:11.792883+0000","last_fullsized":"2023-09-12T16:02:11.792883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:57:53.902946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:23.643545+0000","last_change":"2023-09-12T16:02:04.028086+0000","last_active":"2023-09-12T16:02:23.643545+0000","last_peered":"2023-09-12T16:02:23.643545+0000","last_clean":"2023-09-12T16:02:23.643545+0000","last_became_active":"2023-09-12T16:02:04.027798+0000","last_became_peered":"2023-09-12T16:02:04.027798+0000","last_unstale":"2023-09-12T16:02:23.643545+0000","last_undegraded":"2023-09-12T16:02:23.643545+0000","last_fullsized":"2023-09-12T16:02:23.643545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:11:16.921824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891728+0000","last_change":"2023-09-12T16:02:05.030691+0000","last_active":"2023-09-12T16:02:10.891728+0000","last_peered":"2023-09-12T16:02:10.891728+0000","last_clean":"2023-09-12T16:02:10.891728+0000","last_became_active":"2023-09-12T16:02:05.030379+0000","last_became_peered":"2023-09-12T16:02:05.030379+0000","last_unstale":"2023-09-12T16:02:10.891728+0000","last_undegraded":"2023-09-12T16:02:10.891728+0000","last_fullsized":"2023-09-12T16:02:10.891728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:15:05.221078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793996+0000","last_change":"2023-09-12T16:02:09.290703+0000","last_active":"2023-09-12T16:02:11.793996+0000","last_peered":"2023-09-12T16:02:11.793996+0000","last_clean":"2023-09-12T16:02:11.793996+0000","last_became_active":"2023-09-12T16:02:09.290227+0000","last_became_peered":"2023-09-12T16:02:09.290227+0000","last_unstale":"2023-09-12T16:02:11.793996+0000","last_undegraded":"2023-09-12T16:02:11.793996+0000","last_fullsized":"2023-09-12T16:02:11.793996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:07:06.182295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"80'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265450+0000","last_change":"2023-09-12T16:02:04.028630+0000","last_active":"2023-09-12T16:02:10.265450+0000","last_peered":"2023-09-12T16:02:10.265450+0000","last_clean":"2023-09-12T16:02:10.265450+0000","last_became_active":"2023-09-12T16:02:04.028319+0000","last_became_peered":"2023-09-12T16:02:04.028319+0000","last_unstale":"2023-09-12T16:02:10.265450+0000","last_undegraded":"2023-09-12T16:02:10.265450+0000","last_fullsized":"2023-09-12T16:02:10.265450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:13:43.349425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573598+0000","last_change":"2023-09-12T16:02:05.032392+0000","last_active":"2023-09-12T16:02:10.573598+0000","last_peered":"2023-09-12T16:02:10.573598+0000","last_clean":"2023-09-12T16:02:10.573598+0000","last_became_active":"2023-09-12T16:02:05.032099+0000","last_became_peered":"2023-09-12T16:02:05.032099+0000","last_unstale":"2023-09-12T16:02:10.573598+0000","last_undegraded":"2023-09-12T16:02:10.573598+0000","last_fullsized":"2023-09-12T16:02:10.573598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:46:03.108459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792388+0000","last_change":"2023-09-12T16:02:09.289573+0000","last_active":"2023-09-12T16:02:11.792388+0000","last_peered":"2023-09-12T16:02:11.792388+0000","last_clean":"2023-09-12T16:02:11.792388+0000","last_became_active":"2023-09-12T16:02:09.289179+0000","last_became_peered":"2023-09-12T16:02:09.289179+0000","last_unstale":"2023-09-12T16:02:11.792388+0000","last_undegraded":"2023-09-12T16:02:11.792388+0000","last_fullsized":"2023-09-12T16:02:11.792388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:52:35.905346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'12","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:25.657479+0000","last_change":"2023-09-12T16:02:04.025347+0000","last_active":"2023-09-12T16:02:25.657479+0000","last_peered":"2023-09-12T16:02:25.657479+0000","last_clean":"2023-09-12T16:02:25.657479+0000","last_became_active":"2023-09-12T16:02:04.025017+0000","last_became_peered":"2023-09-12T16:02:04.025017+0000","last_unstale":"2023-09-12T16:02:25.657479+0000","last_undegraded":"2023-09-12T16:02:25.657479+0000","last_fullsized":"2023-09-12T16:02:25.657479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:03:46.502387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305835+0000","last_change":"2023-09-12T16:02:05.031409+0000","last_active":"2023-09-12T16:02:10.305835+0000","last_peered":"2023-09-12T16:02:10.305835+0000","last_clean":"2023-09-12T16:02:10.305835+0000","last_became_active":"2023-09-12T16:02:05.031110+0000","last_became_peered":"2023-09-12T16:02:05.031110+0000","last_unstale":"2023-09-12T16:02:10.305835+0000","last_undegraded":"2023-09-12T16:02:10.305835+0000","last_fullsized":"2023-09-12T16:02:10.305835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:10:54.482078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.794317+0000","last_change":"2023-09-12T16:02:09.291493+0000","last_active":"2023-09-12T16:02:11.794317+0000","last_peered":"2023-09-12T16:02:11.794317+0000","last_clean":"2023-09-12T16:02:11.794317+0000","last_became_active":"2023-09-12T16:02:09.291044+0000","last_became_peered":"2023-09-12T16:02:09.291044+0000","last_unstale":"2023-09-12T16:02:11.794317+0000","last_undegraded":"2023-09-12T16:02:11.794317+0000","last_fullsized":"2023-09-12T16:02:11.794317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:34:30.269720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.650789+0000","last_change":"2023-09-12T16:02:04.029283+0000","last_active":"2023-09-12T16:02:24.650789+0000","last_peered":"2023-09-12T16:02:24.650789+0000","last_clean":"2023-09-12T16:02:24.650789+0000","last_became_active":"2023-09-12T16:02:04.028959+0000","last_became_peered":"2023-09-12T16:02:04.028959+0000","last_unstale":"2023-09-12T16:02:24.650789+0000","last_undegraded":"2023-09-12T16:02:24.650789+0000","last_fullsized":"2023-09-12T16:02:24.650789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:28:52.750278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266703+0000","last_change":"2023-09-12T16:02:05.034140+0000","last_active":"2023-09-12T16:02:10.266703+0000","last_peered":"2023-09-12T16:02:10.266703+0000","last_clean":"2023-09-12T16:02:10.266703+0000","last_became_active":"2023-09-12T16:02:05.033840+0000","last_became_peered":"2023-09-12T16:02:05.033840+0000","last_unstale":"2023-09-12T16:02:10.266703+0000","last_undegraded":"2023-09-12T16:02:10.266703+0000","last_fullsized":"2023-09-12T16:02:10.266703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:59:55.522608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793367+0000","last_change":"2023-09-12T16:02:09.289998+0000","last_active":"2023-09-12T16:02:11.793367+0000","last_peered":"2023-09-12T16:02:11.793367+0000","last_clean":"2023-09-12T16:02:11.793367+0000","last_became_active":"2023-09-12T16:02:09.289561+0000","last_became_peered":"2023-09-12T16:02:09.289561+0000","last_unstale":"2023-09-12T16:02:11.793367+0000","last_undegraded":"2023-09-12T16:02:11.793367+0000","last_fullsized":"2023-09-12T16:02:11.793367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:09:47.327213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265335+0000","last_change":"2023-09-12T16:02:04.029883+0000","last_active":"2023-09-12T16:02:10.265335+0000","last_peered":"2023-09-12T16:02:10.265335+0000","last_clean":"2023-09-12T16:02:10.265335+0000","last_became_active":"2023-09-12T16:02:04.029547+0000","last_became_peered":"2023-09-12T16:02:04.029547+0000","last_unstale":"2023-09-12T16:02:10.265335+0000","last_undegraded":"2023-09-12T16:02:10.265335+0000","last_fullsized":"2023-09-12T16:02:10.265335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:47:46.478325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265519+0000","last_change":"2023-09-12T16:02:05.036377+0000","last_active":"2023-09-12T16:02:10.265519+0000","last_peered":"2023-09-12T16:02:10.265519+0000","last_clean":"2023-09-12T16:02:10.265519+0000","last_became_active":"2023-09-12T16:02:05.036060+0000","last_became_peered":"2023-09-12T16:02:05.036060+0000","last_unstale":"2023-09-12T16:02:10.265519+0000","last_undegraded":"2023-09-12T16:02:10.265519+0000","last_fullsized":"2023-09-12T16:02:10.265519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:26:16.857431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792929+0000","last_change":"2023-09-12T16:02:09.295151+0000","last_active":"2023-09-12T16:02:11.792929+0000","last_peered":"2023-09-12T16:02:11.792929+0000","last_clean":"2023-09-12T16:02:11.792929+0000","last_became_active":"2023-09-12T16:02:09.294732+0000","last_became_peered":"2023-09-12T16:02:09.294732+0000","last_unstale":"2023-09-12T16:02:11.792929+0000","last_undegraded":"2023-09-12T16:02:11.792929+0000","last_fullsized":"2023-09-12T16:02:11.792929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:47:19.236310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760396+0000","last_change":"2023-09-12T16:02:04.025723+0000","last_active":"2023-09-12T16:02:10.760396+0000","last_peered":"2023-09-12T16:02:10.760396+0000","last_clean":"2023-09-12T16:02:10.760396+0000","last_became_active":"2023-09-12T16:02:04.025432+0000","last_became_peered":"2023-09-12T16:02:04.025432+0000","last_unstale":"2023-09-12T16:02:10.760396+0000","last_undegraded":"2023-09-12T16:02:10.760396+0000","last_fullsized":"2023-09-12T16:02:10.760396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:56.279910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265229+0000","last_change":"2023-09-12T16:02:05.031741+0000","last_active":"2023-09-12T16:02:10.265229+0000","last_peered":"2023-09-12T16:02:10.265229+0000","last_clean":"2023-09-12T16:02:10.265229+0000","last_became_active":"2023-09-12T16:02:05.031233+0000","last_became_peered":"2023-09-12T16:02:05.031233+0000","last_unstale":"2023-09-12T16:02:10.265229+0000","last_undegraded":"2023-09-12T16:02:10.265229+0000","last_fullsized":"2023-09-12T16:02:10.265229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:55:33.673706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793234+0000","last_change":"2023-09-12T16:02:09.286447+0000","last_active":"2023-09-12T16:02:11.793234+0000","last_peered":"2023-09-12T16:02:11.793234+0000","last_clean":"2023-09-12T16:02:11.793234+0000","last_became_active":"2023-09-12T16:02:09.285805+0000","last_became_peered":"2023-09-12T16:02:09.285805+0000","last_unstale":"2023-09-12T16:02:11.793234+0000","last_undegraded":"2023-09-12T16:02:11.793234+0000","last_fullsized":"2023-09-12T16:02:11.793234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:35:05.518872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.650732+0000","last_change":"2023-09-12T16:02:04.032527+0000","last_active":"2023-09-12T16:02:24.650732+0000","last_peered":"2023-09-12T16:02:24.650732+0000","last_clean":"2023-09-12T16:02:24.650732+0000","last_became_active":"2023-09-12T16:02:04.032264+0000","last_became_peered":"2023-09-12T16:02:04.032264+0000","last_unstale":"2023-09-12T16:02:24.650732+0000","last_undegraded":"2023-09-12T16:02:24.650732+0000","last_fullsized":"2023-09-12T16:02:24.650732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:53:52.433549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265316+0000","last_change":"2023-09-12T16:02:05.029281+0000","last_active":"2023-09-12T16:02:10.265316+0000","last_peered":"2023-09-12T16:02:10.265316+0000","last_clean":"2023-09-12T16:02:10.265316+0000","last_became_active":"2023-09-12T16:02:05.028919+0000","last_became_peered":"2023-09-12T16:02:05.028919+0000","last_unstale":"2023-09-12T16:02:10.265316+0000","last_undegraded":"2023-09-12T16:02:10.265316+0000","last_fullsized":"2023-09-12T16:02:10.265316+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:26.667060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792839+0000","last_change":"2023-09-12T16:02:09.295801+0000","last_active":"2023-09-12T16:02:11.792839+0000","last_peered":"2023-09-12T16:02:11.792839+0000","last_clean":"2023-09-12T16:02:11.792839+0000","last_became_active":"2023-09-12T16:02:09.295472+0000","last_became_peered":"2023-09-12T16:02:09.295472+0000","last_unstale":"2023-09-12T16:02:11.792839+0000","last_undegraded":"2023-09-12T16:02:11.792839+0000","last_fullsized":"2023-09-12T16:02:11.792839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:35:37.628783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305913+0000","last_change":"2023-09-12T16:02:04.026373+0000","last_active":"2023-09-12T16:02:10.305913+0000","last_peered":"2023-09-12T16:02:10.305913+0000","last_clean":"2023-09-12T16:02:10.305913+0000","last_became_active":"2023-09-12T16:02:04.025907+0000","last_became_peered":"2023-09-12T16:02:04.025907+0000","last_unstale":"2023-09-12T16:02:10.305913+0000","last_undegraded":"2023-09-12T16:02:10.305913+0000","last_fullsized":"2023-09-12T16:02:10.305913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:33:15.152128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892162+0000","last_change":"2023-09-12T16:02:05.040143+0000","last_active":"2023-09-12T16:02:10.892162+0000","last_peered":"2023-09-12T16:02:10.892162+0000","last_clean":"2023-09-12T16:02:10.892162+0000","last_became_active":"2023-09-12T16:02:05.039749+0000","last_became_peered":"2023-09-12T16:02:05.039749+0000","last_unstale":"2023-09-12T16:02:10.892162+0000","last_undegraded":"2023-09-12T16:02:10.892162+0000","last_fullsized":"2023-09-12T16:02:10.892162+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:49.561252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793807+0000","last_change":"2023-09-12T16:02:09.841670+0000","last_active":"2023-09-12T16:02:11.793807+0000","last_peered":"2023-09-12T16:02:11.793807+0000","last_clean":"2023-09-12T16:02:11.793807+0000","last_became_active":"2023-09-12T16:02:09.841203+0000","last_became_peered":"2023-09-12T16:02:09.841203+0000","last_unstale":"2023-09-12T16:02:11.793807+0000","last_undegraded":"2023-09-12T16:02:11.793807+0000","last_fullsized":"2023-09-12T16:02:11.793807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:55:46.128996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892443+0000","last_change":"2023-09-12T16:02:04.027953+0000","last_active":"2023-09-12T16:02:10.892443+0000","last_peered":"2023-09-12T16:02:10.892443+0000","last_clean":"2023-09-12T16:02:10.892443+0000","last_became_active":"2023-09-12T16:02:04.027624+0000","last_became_peered":"2023-09-12T16:02:04.027624+0000","last_unstale":"2023-09-12T16:02:10.892443+0000","last_undegraded":"2023-09-12T16:02:10.892443+0000","last_fullsized":"2023-09-12T16:02:10.892443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:33:58.356358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892073+0000","last_change":"2023-09-12T16:02:05.036593+0000","last_active":"2023-09-12T16:02:10.892073+0000","last_peered":"2023-09-12T16:02:10.892073+0000","last_clean":"2023-09-12T16:02:10.892073+0000","last_became_active":"2023-09-12T16:02:05.036204+0000","last_became_peered":"2023-09-12T16:02:05.036204+0000","last_unstale":"2023-09-12T16:02:10.892073+0000","last_undegraded":"2023-09-12T16:02:10.892073+0000","last_fullsized":"2023-09-12T16:02:10.892073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:12:31.068837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793070+0000","last_change":"2023-09-12T16:02:09.292405+0000","last_active":"2023-09-12T16:02:11.793070+0000","last_peered":"2023-09-12T16:02:11.793070+0000","last_clean":"2023-09-12T16:02:11.793070+0000","last_became_active":"2023-09-12T16:02:09.291877+0000","last_became_peered":"2023-09-12T16:02:09.291877+0000","last_unstale":"2023-09-12T16:02:11.793070+0000","last_undegraded":"2023-09-12T16:02:11.793070+0000","last_fullsized":"2023-09-12T16:02:11.793070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:53:04.847310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265427+0000","last_change":"2023-09-12T16:02:04.034738+0000","last_active":"2023-09-12T16:02:10.265427+0000","last_peered":"2023-09-12T16:02:10.265427+0000","last_clean":"2023-09-12T16:02:10.265427+0000","last_became_active":"2023-09-12T16:02:04.034408+0000","last_became_peered":"2023-09-12T16:02:04.034408+0000","last_unstale":"2023-09-12T16:02:10.265427+0000","last_undegraded":"2023-09-12T16:02:10.265427+0000","last_fullsized":"2023-09-12T16:02:10.265427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:27.845770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891985+0000","last_change":"2023-09-12T16:02:05.044936+0000","last_active":"2023-09-12T16:02:10.891985+0000","last_peered":"2023-09-12T16:02:10.891985+0000","last_clean":"2023-09-12T16:02:10.891985+0000","last_became_active":"2023-09-12T16:02:05.044617+0000","last_became_peered":"2023-09-12T16:02:05.044617+0000","last_unstale":"2023-09-12T16:02:10.891985+0000","last_undegraded":"2023-09-12T16:02:10.891985+0000","last_fullsized":"2023-09-12T16:02:10.891985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:41:32.639331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792871+0000","last_change":"2023-09-12T16:02:09.841919+0000","last_active":"2023-09-12T16:02:11.792871+0000","last_peered":"2023-09-12T16:02:11.792871+0000","last_clean":"2023-09-12T16:02:11.792871+0000","last_became_active":"2023-09-12T16:02:09.841410+0000","last_became_peered":"2023-09-12T16:02:09.841410+0000","last_unstale":"2023-09-12T16:02:11.792871+0000","last_undegraded":"2023-09-12T16:02:11.792871+0000","last_fullsized":"2023-09-12T16:02:11.792871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:29:57.511693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265034+0000","last_change":"2023-09-12T16:02:05.033181+0000","last_active":"2023-09-12T16:02:10.265034+0000","last_peered":"2023-09-12T16:02:10.265034+0000","last_clean":"2023-09-12T16:02:10.265034+0000","last_became_active":"2023-09-12T16:02:05.032888+0000","last_became_peered":"2023-09-12T16:02:05.032888+0000","last_unstale":"2023-09-12T16:02:10.265034+0000","last_undegraded":"2023-09-12T16:02:10.265034+0000","last_fullsized":"2023-09-12T16:02:10.265034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:11:06.693972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792084+0000","last_change":"2023-09-12T16:02:04.032892+0000","last_active":"2023-09-12T16:02:11.792084+0000","last_peered":"2023-09-12T16:02:11.792084+0000","last_clean":"2023-09-12T16:02:11.792084+0000","last_became_active":"2023-09-12T16:02:04.032637+0000","last_became_peered":"2023-09-12T16:02:04.032637+0000","last_unstale":"2023-09-12T16:02:11.792084+0000","last_undegraded":"2023-09-12T16:02:11.792084+0000","last_fullsized":"2023-09-12T16:02:11.792084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:22:55.805000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266685+0000","last_change":"2023-09-12T16:02:09.285127+0000","last_active":"2023-09-12T16:02:11.266685+0000","last_peered":"2023-09-12T16:02:11.266685+0000","last_clean":"2023-09-12T16:02:11.266685+0000","last_became_active":"2023-09-12T16:02:09.284790+0000","last_became_peered":"2023-09-12T16:02:09.284790+0000","last_unstale":"2023-09-12T16:02:11.266685+0000","last_undegraded":"2023-09-12T16:02:11.266685+0000","last_fullsized":"2023-09-12T16:02:11.266685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:50:50.578069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.627836+0000","last_change":"2023-09-12T16:02:05.027877+0000","last_active":"2023-09-12T16:02:10.627836+0000","last_peered":"2023-09-12T16:02:10.627836+0000","last_clean":"2023-09-12T16:02:10.627836+0000","last_became_active":"2023-09-12T16:02:05.027582+0000","last_became_peered":"2023-09-12T16:02:05.027582+0000","last_unstale":"2023-09-12T16:02:10.627836+0000","last_undegraded":"2023-09-12T16:02:10.627836+0000","last_fullsized":"2023-09-12T16:02:10.627836+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:47.835790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760444+0000","last_change":"2023-09-12T16:02:04.024800+0000","last_active":"2023-09-12T16:02:10.760444+0000","last_peered":"2023-09-12T16:02:10.760444+0000","last_clean":"2023-09-12T16:02:10.760444+0000","last_became_active":"2023-09-12T16:02:04.024436+0000","last_became_peered":"2023-09-12T16:02:04.024436+0000","last_unstale":"2023-09-12T16:02:10.760444+0000","last_undegraded":"2023-09-12T16:02:10.760444+0000","last_fullsized":"2023-09-12T16:02:10.760444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:57:35.093322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":311,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265106+0000","last_change":"2023-09-12T16:01:04.599706+0000","last_active":"2023-09-12T16:02:10.265106+0000","last_peered":"2023-09-12T16:02:10.265106+0000","last_clean":"2023-09-12T16:02:10.265106+0000","last_became_active":"2023-09-12T16:01:04.599328+0000","last_became_peered":"2023-09-12T16:01:04.599328+0000","last_unstale":"2023-09-12T16:02:10.265106+0000","last_undegraded":"2023-09-12T16:02:10.265106+0000","last_fullsized":"2023-09-12T16:02:10.265106+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_clean_scrub_stamp":"2023-09-12T15:58:15.250648+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:57:56.846488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.794449+0000","last_change":"2023-09-12T16:02:09.841960+0000","last_active":"2023-09-12T16:02:11.794449+0000","last_peered":"2023-09-12T16:02:11.794449+0000","last_clean":"2023-09-12T16:02:11.794449+0000","last_became_active":"2023-09-12T16:02:09.841279+0000","last_became_peered":"2023-09-12T16:02:09.841279+0000","last_unstale":"2023-09-12T16:02:11.794449+0000","last_undegraded":"2023-09-12T16:02:11.794449+0000","last_fullsized":"2023-09-12T16:02:11.794449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:17:19.419350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266459+0000","last_change":"2023-09-12T16:02:05.038666+0000","last_active":"2023-09-12T16:02:10.266459+0000","last_peered":"2023-09-12T16:02:10.266459+0000","last_clean":"2023-09-12T16:02:10.266459+0000","last_became_active":"2023-09-12T16:02:05.038336+0000","last_became_peered":"2023-09-12T16:02:05.038336+0000","last_unstale":"2023-09-12T16:02:10.266459+0000","last_undegraded":"2023-09-12T16:02:10.266459+0000","last_fullsized":"2023-09-12T16:02:10.266459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:20:35.844998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266212+0000","last_change":"2023-09-12T16:02:04.029790+0000","last_active":"2023-09-12T16:02:11.266212+0000","last_peered":"2023-09-12T16:02:11.266212+0000","last_clean":"2023-09-12T16:02:11.266212+0000","last_became_active":"2023-09-12T16:02:04.029440+0000","last_became_peered":"2023-09-12T16:02:04.029440+0000","last_unstale":"2023-09-12T16:02:11.266212+0000","last_undegraded":"2023-09-12T16:02:11.266212+0000","last_fullsized":"2023-09-12T16:02:11.266212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:50:33.690628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793848+0000","last_change":"2023-09-12T16:02:09.289757+0000","last_active":"2023-09-12T16:02:11.793848+0000","last_peered":"2023-09-12T16:02:11.793848+0000","last_clean":"2023-09-12T16:02:11.793848+0000","last_became_active":"2023-09-12T16:02:09.289348+0000","last_became_peered":"2023-09-12T16:02:09.289348+0000","last_unstale":"2023-09-12T16:02:11.793848+0000","last_undegraded":"2023-09-12T16:02:11.793848+0000","last_fullsized":"2023-09-12T16:02:11.793848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:15:14.695521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792913+0000","last_change":"2023-09-12T16:02:04.033213+0000","last_active":"2023-09-12T16:02:11.792913+0000","last_peered":"2023-09-12T16:02:11.792913+0000","last_clean":"2023-09-12T16:02:11.792913+0000","last_became_active":"2023-09-12T16:02:04.032978+0000","last_became_peered":"2023-09-12T16:02:04.032978+0000","last_unstale":"2023-09-12T16:02:11.792913+0000","last_undegraded":"2023-09-12T16:02:11.792913+0000","last_fullsized":"2023-09-12T16:02:11.792913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:25:33.718954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266899+0000","last_change":"2023-09-12T16:02:05.043252+0000","last_active":"2023-09-12T16:02:10.266899+0000","last_peered":"2023-09-12T16:02:10.266899+0000","last_clean":"2023-09-12T16:02:10.266899+0000","last_became_active":"2023-09-12T16:02:05.042975+0000","last_became_peered":"2023-09-12T16:02:05.042975+0000","last_unstale":"2023-09-12T16:02:10.266899+0000","last_undegraded":"2023-09-12T16:02:10.266899+0000","last_fullsized":"2023-09-12T16:02:10.266899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:46:31.549547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.265460+0000","last_change":"2023-09-12T16:02:09.841728+0000","last_active":"2023-09-12T16:02:11.265460+0000","last_peered":"2023-09-12T16:02:11.265460+0000","last_clean":"2023-09-12T16:02:11.265460+0000","last_became_active":"2023-09-12T16:02:09.841351+0000","last_became_peered":"2023-09-12T16:02:09.841351+0000","last_unstale":"2023-09-12T16:02:11.265460+0000","last_undegraded":"2023-09-12T16:02:11.265460+0000","last_fullsized":"2023-09-12T16:02:11.265460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:00:50.637272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892838+0000","last_change":"2023-09-12T16:02:04.025624+0000","last_active":"2023-09-12T16:02:10.892838+0000","last_peered":"2023-09-12T16:02:10.892838+0000","last_clean":"2023-09-12T16:02:10.892838+0000","last_became_active":"2023-09-12T16:02:04.025330+0000","last_became_peered":"2023-09-12T16:02:04.025330+0000","last_unstale":"2023-09-12T16:02:10.892838+0000","last_undegraded":"2023-09-12T16:02:10.892838+0000","last_fullsized":"2023-09-12T16:02:10.892838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:00:57.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.627988+0000","last_change":"2023-09-12T16:02:05.028414+0000","last_active":"2023-09-12T16:02:10.627988+0000","last_peered":"2023-09-12T16:02:10.627988+0000","last_clean":"2023-09-12T16:02:10.627988+0000","last_became_active":"2023-09-12T16:02:05.028081+0000","last_became_peered":"2023-09-12T16:02:05.028081+0000","last_unstale":"2023-09-12T16:02:10.627988+0000","last_undegraded":"2023-09-12T16:02:10.627988+0000","last_fullsized":"2023-09-12T16:02:10.627988+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:57:58.245105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266169+0000","last_change":"2023-09-12T16:02:09.283222+0000","last_active":"2023-09-12T16:02:11.266169+0000","last_peered":"2023-09-12T16:02:11.266169+0000","last_clean":"2023-09-12T16:02:11.266169+0000","last_became_active":"2023-09-12T16:02:09.282814+0000","last_became_peered":"2023-09-12T16:02:09.282814+0000","last_unstale":"2023-09-12T16:02:11.266169+0000","last_undegraded":"2023-09-12T16:02:11.266169+0000","last_fullsized":"2023-09-12T16:02:11.266169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:18.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"80'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760915+0000","last_change":"2023-09-12T16:02:04.026694+0000","last_active":"2023-09-12T16:02:10.760915+0000","last_peered":"2023-09-12T16:02:10.760915+0000","last_clean":"2023-09-12T16:02:10.760915+0000","last_became_active":"2023-09-12T16:02:04.026368+0000","last_became_peered":"2023-09-12T16:02:04.026368+0000","last_unstale":"2023-09-12T16:02:10.760915+0000","last_undegraded":"2023-09-12T16:02:10.760915+0000","last_fullsized":"2023-09-12T16:02:10.760915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:14:10.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891618+0000","last_change":"2023-09-12T16:02:05.039037+0000","last_active":"2023-09-12T16:02:10.891618+0000","last_peered":"2023-09-12T16:02:10.891618+0000","last_clean":"2023-09-12T16:02:10.891618+0000","last_became_active":"2023-09-12T16:02:05.038708+0000","last_became_peered":"2023-09-12T16:02:05.038708+0000","last_unstale":"2023-09-12T16:02:10.891618+0000","last_undegraded":"2023-09-12T16:02:10.891618+0000","last_fullsized":"2023-09-12T16:02:10.891618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:13:09.805646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793845+0000","last_change":"2023-09-12T16:02:09.841757+0000","last_active":"2023-09-12T16:02:11.793845+0000","last_peered":"2023-09-12T16:02:11.793845+0000","last_clean":"2023-09-12T16:02:11.793845+0000","last_became_active":"2023-09-12T16:02:09.841173+0000","last_became_peered":"2023-09-12T16:02:09.841173+0000","last_unstale":"2023-09-12T16:02:11.793845+0000","last_undegraded":"2023-09-12T16:02:11.793845+0000","last_fullsized":"2023-09-12T16:02:11.793845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:57:06.626828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573197+0000","last_change":"2023-09-12T16:02:04.031009+0000","last_active":"2023-09-12T16:02:10.573197+0000","last_peered":"2023-09-12T16:02:10.573197+0000","last_clean":"2023-09-12T16:02:10.573197+0000","last_became_active":"2023-09-12T16:02:04.030714+0000","last_became_peered":"2023-09-12T16:02:04.030714+0000","last_unstale":"2023-09-12T16:02:10.573197+0000","last_undegraded":"2023-09-12T16:02:10.573197+0000","last_fullsized":"2023-09-12T16:02:10.573197+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:12.578686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305382+0000","last_change":"2023-09-12T16:02:05.030778+0000","last_active":"2023-09-12T16:02:10.305382+0000","last_peered":"2023-09-12T16:02:10.305382+0000","last_clean":"2023-09-12T16:02:10.305382+0000","last_became_active":"2023-09-12T16:02:05.030454+0000","last_became_peered":"2023-09-12T16:02:05.030454+0000","last_unstale":"2023-09-12T16:02:10.305382+0000","last_undegraded":"2023-09-12T16:02:10.305382+0000","last_fullsized":"2023-09-12T16:02:10.305382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:52.951246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793015+0000","last_change":"2023-09-12T16:02:09.289668+0000","last_active":"2023-09-12T16:02:11.793015+0000","last_peered":"2023-09-12T16:02:11.793015+0000","last_clean":"2023-09-12T16:02:11.793015+0000","last_became_active":"2023-09-12T16:02:09.289301+0000","last_became_peered":"2023-09-12T16:02:09.289301+0000","last_unstale":"2023-09-12T16:02:11.793015+0000","last_undegraded":"2023-09-12T16:02:11.793015+0000","last_fullsized":"2023-09-12T16:02:11.793015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:20:04.528146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:23.643840+0000","last_change":"2023-09-12T16:02:04.036175+0000","last_active":"2023-09-12T16:02:23.643840+0000","last_peered":"2023-09-12T16:02:23.643840+0000","last_clean":"2023-09-12T16:02:23.643840+0000","last_became_active":"2023-09-12T16:02:04.035879+0000","last_became_peered":"2023-09-12T16:02:04.035879+0000","last_unstale":"2023-09-12T16:02:23.643840+0000","last_undegraded":"2023-09-12T16:02:23.643840+0000","last_fullsized":"2023-09-12T16:02:23.643840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:29:20.798716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891598+0000","last_change":"2023-09-12T16:02:05.034195+0000","last_active":"2023-09-12T16:02:10.891598+0000","last_peered":"2023-09-12T16:02:10.891598+0000","last_clean":"2023-09-12T16:02:10.891598+0000","last_became_active":"2023-09-12T16:02:05.033534+0000","last_became_peered":"2023-09-12T16:02:05.033534+0000","last_unstale":"2023-09-12T16:02:10.891598+0000","last_undegraded":"2023-09-12T16:02:10.891598+0000","last_fullsized":"2023-09-12T16:02:10.891598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:34:04.180218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793440+0000","last_change":"2023-09-12T16:02:09.287543+0000","last_active":"2023-09-12T16:02:11.793440+0000","last_peered":"2023-09-12T16:02:11.793440+0000","last_clean":"2023-09-12T16:02:11.793440+0000","last_became_active":"2023-09-12T16:02:09.287055+0000","last_became_peered":"2023-09-12T16:02:09.287055+0000","last_unstale":"2023-09-12T16:02:11.793440+0000","last_undegraded":"2023-09-12T16:02:11.793440+0000","last_fullsized":"2023-09-12T16:02:11.793440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:58.214381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892036+0000","last_change":"2023-09-12T16:02:04.030206+0000","last_active":"2023-09-12T16:02:10.892036+0000","last_peered":"2023-09-12T16:02:10.892036+0000","last_clean":"2023-09-12T16:02:10.892036+0000","last_became_active":"2023-09-12T16:02:04.029765+0000","last_became_peered":"2023-09-12T16:02:04.029765+0000","last_unstale":"2023-09-12T16:02:10.892036+0000","last_undegraded":"2023-09-12T16:02:10.892036+0000","last_fullsized":"2023-09-12T16:02:10.892036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:56:07.551681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.628253+0000","last_change":"2023-09-12T16:02:05.028300+0000","last_active":"2023-09-12T16:02:10.628253+0000","last_peered":"2023-09-12T16:02:10.628253+0000","last_clean":"2023-09-12T16:02:10.628253+0000","last_became_active":"2023-09-12T16:02:05.027970+0000","last_became_peered":"2023-09-12T16:02:05.027970+0000","last_unstale":"2023-09-12T16:02:10.628253+0000","last_undegraded":"2023-09-12T16:02:10.628253+0000","last_fullsized":"2023-09-12T16:02:10.628253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:39:44.571593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792625+0000","last_change":"2023-09-12T16:02:09.841661+0000","last_active":"2023-09-12T16:02:11.792625+0000","last_peered":"2023-09-12T16:02:11.792625+0000","last_clean":"2023-09-12T16:02:11.792625+0000","last_became_active":"2023-09-12T16:02:09.841039+0000","last_became_peered":"2023-09-12T16:02:09.841039+0000","last_unstale":"2023-09-12T16:02:11.792625+0000","last_undegraded":"2023-09-12T16:02:11.792625+0000","last_fullsized":"2023-09-12T16:02:11.792625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:34.529098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:23.642882+0000","last_change":"2023-09-12T16:02:04.031927+0000","last_active":"2023-09-12T16:02:23.642882+0000","last_peered":"2023-09-12T16:02:23.642882+0000","last_clean":"2023-09-12T16:02:23.642882+0000","last_became_active":"2023-09-12T16:02:04.031614+0000","last_became_peered":"2023-09-12T16:02:04.031614+0000","last_unstale":"2023-09-12T16:02:23.642882+0000","last_undegraded":"2023-09-12T16:02:23.642882+0000","last_fullsized":"2023-09-12T16:02:23.642882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:00:23.423933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265435+0000","last_change":"2023-09-12T16:02:05.032647+0000","last_active":"2023-09-12T16:02:10.265435+0000","last_peered":"2023-09-12T16:02:10.265435+0000","last_clean":"2023-09-12T16:02:10.265435+0000","last_became_active":"2023-09-12T16:02:05.031701+0000","last_became_peered":"2023-09-12T16:02:05.031701+0000","last_unstale":"2023-09-12T16:02:10.265435+0000","last_undegraded":"2023-09-12T16:02:10.265435+0000","last_fullsized":"2023-09-12T16:02:10.265435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:02:19.596913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792820+0000","last_change":"2023-09-12T16:02:09.291043+0000","last_active":"2023-09-12T16:02:11.792820+0000","last_peered":"2023-09-12T16:02:11.792820+0000","last_clean":"2023-09-12T16:02:11.792820+0000","last_became_active":"2023-09-12T16:02:09.290666+0000","last_became_peered":"2023-09-12T16:02:09.290666+0000","last_unstale":"2023-09-12T16:02:11.792820+0000","last_undegraded":"2023-09-12T16:02:11.792820+0000","last_fullsized":"2023-09-12T16:02:11.792820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:10:47.354479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.651169+0000","last_change":"2023-09-12T16:02:04.030501+0000","last_active":"2023-09-12T16:02:24.651169+0000","last_peered":"2023-09-12T16:02:24.651169+0000","last_clean":"2023-09-12T16:02:24.651169+0000","last_became_active":"2023-09-12T16:02:04.030196+0000","last_became_peered":"2023-09-12T16:02:04.030196+0000","last_unstale":"2023-09-12T16:02:24.651169+0000","last_undegraded":"2023-09-12T16:02:24.651169+0000","last_fullsized":"2023-09-12T16:02:24.651169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:58:34.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265985+0000","last_change":"2023-09-12T16:02:05.030466+0000","last_active":"2023-09-12T16:02:10.265985+0000","last_peered":"2023-09-12T16:02:10.265985+0000","last_clean":"2023-09-12T16:02:10.265985+0000","last_became_active":"2023-09-12T16:02:05.029988+0000","last_became_peered":"2023-09-12T16:02:05.029988+0000","last_unstale":"2023-09-12T16:02:10.265985+0000","last_undegraded":"2023-09-12T16:02:10.265985+0000","last_fullsized":"2023-09-12T16:02:10.265985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:08:56.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:12.221376+0000","last_change":"2023-09-12T16:02:09.294681+0000","last_active":"2023-09-12T16:02:12.221376+0000","last_peered":"2023-09-12T16:02:12.221376+0000","last_clean":"2023-09-12T16:02:12.221376+0000","last_became_active":"2023-09-12T16:02:09.294294+0000","last_became_peered":"2023-09-12T16:02:09.294294+0000","last_unstale":"2023-09-12T16:02:12.221376+0000","last_undegraded":"2023-09-12T16:02:12.221376+0000","last_fullsized":"2023-09-12T16:02:12.221376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:14:47.286285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265590+0000","last_change":"2023-09-12T16:02:04.027380+0000","last_active":"2023-09-12T16:02:10.265590+0000","last_peered":"2023-09-12T16:02:10.265590+0000","last_clean":"2023-09-12T16:02:10.265590+0000","last_became_active":"2023-09-12T16:02:04.027098+0000","last_became_peered":"2023-09-12T16:02:04.027098+0000","last_unstale":"2023-09-12T16:02:10.265590+0000","last_undegraded":"2023-09-12T16:02:10.265590+0000","last_fullsized":"2023-09-12T16:02:10.265590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:34:12.260747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265862+0000","last_change":"2023-09-12T16:02:05.029249+0000","last_active":"2023-09-12T16:02:10.265862+0000","last_peered":"2023-09-12T16:02:10.265862+0000","last_clean":"2023-09-12T16:02:10.265862+0000","last_became_active":"2023-09-12T16:02:05.028955+0000","last_became_peered":"2023-09-12T16:02:05.028955+0000","last_unstale":"2023-09-12T16:02:10.265862+0000","last_undegraded":"2023-09-12T16:02:10.265862+0000","last_fullsized":"2023-09-12T16:02:10.265862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:02:10.382712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792450+0000","last_change":"2023-09-12T16:02:09.288724+0000","last_active":"2023-09-12T16:02:11.792450+0000","last_peered":"2023-09-12T16:02:11.792450+0000","last_clean":"2023-09-12T16:02:11.792450+0000","last_became_active":"2023-09-12T16:02:09.288206+0000","last_became_peered":"2023-09-12T16:02:09.288206+0000","last_unstale":"2023-09-12T16:02:11.792450+0000","last_undegraded":"2023-09-12T16:02:11.792450+0000","last_fullsized":"2023-09-12T16:02:11.792450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:45:10.154195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.267028+0000","last_change":"2023-09-12T16:02:04.031219+0000","last_active":"2023-09-12T16:02:10.267028+0000","last_peered":"2023-09-12T16:02:10.267028+0000","last_clean":"2023-09-12T16:02:10.267028+0000","last_became_active":"2023-09-12T16:02:04.030867+0000","last_became_peered":"2023-09-12T16:02:04.030867+0000","last_unstale":"2023-09-12T16:02:10.267028+0000","last_undegraded":"2023-09-12T16:02:10.267028+0000","last_fullsized":"2023-09-12T16:02:10.267028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:16.885094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265856+0000","last_change":"2023-09-12T16:02:05.028004+0000","last_active":"2023-09-12T16:02:10.265856+0000","last_peered":"2023-09-12T16:02:10.265856+0000","last_clean":"2023-09-12T16:02:10.265856+0000","last_became_active":"2023-09-12T16:02:05.027668+0000","last_became_peered":"2023-09-12T16:02:05.027668+0000","last_unstale":"2023-09-12T16:02:10.265856+0000","last_undegraded":"2023-09-12T16:02:10.265856+0000","last_fullsized":"2023-09-12T16:02:10.265856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:41:23.019243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891680+0000","last_change":"2023-09-12T16:02:05.034380+0000","last_active":"2023-09-12T16:02:10.891680+0000","last_peered":"2023-09-12T16:02:10.891680+0000","last_clean":"2023-09-12T16:02:10.891680+0000","last_became_active":"2023-09-12T16:02:05.034022+0000","last_became_peered":"2023-09-12T16:02:05.034022+0000","last_unstale":"2023-09-12T16:02:10.891680+0000","last_undegraded":"2023-09-12T16:02:10.891680+0000","last_fullsized":"2023-09-12T16:02:10.891680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:55:56.404742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892801+0000","last_change":"2023-09-12T16:02:05.031723+0000","last_active":"2023-09-12T16:02:10.892801+0000","last_peered":"2023-09-12T16:02:10.892801+0000","last_clean":"2023-09-12T16:02:10.892801+0000","last_became_active":"2023-09-12T16:02:05.030954+0000","last_became_peered":"2023-09-12T16:02:05.030954+0000","last_unstale":"2023-09-12T16:02:10.892801+0000","last_undegraded":"2023-09-12T16:02:10.892801+0000","last_fullsized":"2023-09-12T16:02:10.892801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.319954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891934+0000","last_change":"2023-09-12T16:02:05.034851+0000","last_active":"2023-09-12T16:02:10.891934+0000","last_peered":"2023-09-12T16:02:10.891934+0000","last_clean":"2023-09-12T16:02:10.891934+0000","last_became_active":"2023-09-12T16:02:05.030733+0000","last_became_peered":"2023-09-12T16:02:05.030733+0000","last_unstale":"2023-09-12T16:02:10.891934+0000","last_undegraded":"2023-09-12T16:02:10.891934+0000","last_fullsized":"2023-09-12T16:02:10.891934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:33.605216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891582+0000","last_change":"2023-09-12T16:02:05.037331+0000","last_active":"2023-09-12T16:02:10.891582+0000","last_peered":"2023-09-12T16:02:10.891582+0000","last_clean":"2023-09-12T16:02:10.891582+0000","last_became_active":"2023-09-12T16:02:05.037015+0000","last_became_peered":"2023-09-12T16:02:05.037015+0000","last_unstale":"2023-09-12T16:02:10.891582+0000","last_undegraded":"2023-09-12T16:02:10.891582+0000","last_fullsized":"2023-09-12T16:02:10.891582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:29:14.035891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266985+0000","last_change":"2023-09-12T16:02:05.039803+0000","last_active":"2023-09-12T16:02:10.266985+0000","last_peered":"2023-09-12T16:02:10.266985+0000","last_clean":"2023-09-12T16:02:10.266985+0000","last_became_active":"2023-09-12T16:02:05.039390+0000","last_became_peered":"2023-09-12T16:02:05.039390+0000","last_unstale":"2023-09-12T16:02:10.266985+0000","last_undegraded":"2023-09-12T16:02:10.266985+0000","last_fullsized":"2023-09-12T16:02:10.266985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:22.422090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892400+0000","last_change":"2023-09-12T16:02:05.033153+0000","last_active":"2023-09-12T16:02:10.892400+0000","last_peered":"2023-09-12T16:02:10.892400+0000","last_clean":"2023-09-12T16:02:10.892400+0000","last_became_active":"2023-09-12T16:02:05.032800+0000","last_became_peered":"2023-09-12T16:02:05.032800+0000","last_unstale":"2023-09-12T16:02:10.892400+0000","last_undegraded":"2023-09-12T16:02:10.892400+0000","last_fullsized":"2023-09-12T16:02:10.892400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:31:43.090031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892295+0000","last_change":"2023-09-12T16:02:05.031519+0000","last_active":"2023-09-12T16:02:10.892295+0000","last_peered":"2023-09-12T16:02:10.892295+0000","last_clean":"2023-09-12T16:02:10.892295+0000","last_became_active":"2023-09-12T16:02:05.030227+0000","last_became_peered":"2023-09-12T16:02:05.030227+0000","last_unstale":"2023-09-12T16:02:10.892295+0000","last_undegraded":"2023-09-12T16:02:10.892295+0000","last_fullsized":"2023-09-12T16:02:10.892295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:00.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.628098+0000","last_change":"2023-09-12T16:02:05.031763+0000","last_active":"2023-09-12T16:02:10.628098+0000","last_peered":"2023-09-12T16:02:10.628098+0000","last_clean":"2023-09-12T16:02:10.628098+0000","last_became_active":"2023-09-12T16:02:05.031441+0000","last_became_peered":"2023-09-12T16:02:05.031441+0000","last_unstale":"2023-09-12T16:02:10.628098+0000","last_undegraded":"2023-09-12T16:02:10.628098+0000","last_fullsized":"2023-09-12T16:02:10.628098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:55.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.264912+0000","last_change":"2023-09-12T16:02:05.033786+0000","last_active":"2023-09-12T16:02:10.264912+0000","last_peered":"2023-09-12T16:02:10.264912+0000","last_clean":"2023-09-12T16:02:10.264912+0000","last_became_active":"2023-09-12T16:02:05.033497+0000","last_became_peered":"2023-09-12T16:02:05.033497+0000","last_unstale":"2023-09-12T16:02:10.264912+0000","last_undegraded":"2023-09-12T16:02:10.264912+0000","last_fullsized":"2023-09-12T16:02:10.264912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:01:58.912488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.627896+0000","last_change":"2023-09-12T16:02:05.031144+0000","last_active":"2023-09-12T16:02:10.627896+0000","last_peered":"2023-09-12T16:02:10.627896+0000","last_clean":"2023-09-12T16:02:10.627896+0000","last_became_active":"2023-09-12T16:02:05.030827+0000","last_became_peered":"2023-09-12T16:02:05.030827+0000","last_unstale":"2023-09-12T16:02:10.627896+0000","last_undegraded":"2023-09-12T16:02:10.627896+0000","last_fullsized":"2023-09-12T16:02:10.627896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:40:48.014344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573443+0000","last_change":"2023-09-12T16:02:05.021273+0000","last_active":"2023-09-12T16:02:10.573443+0000","last_peered":"2023-09-12T16:02:10.573443+0000","last_clean":"2023-09-12T16:02:10.573443+0000","last_became_active":"2023-09-12T16:02:05.020953+0000","last_became_peered":"2023-09-12T16:02:05.020953+0000","last_unstale":"2023-09-12T16:02:10.573443+0000","last_undegraded":"2023-09-12T16:02:10.573443+0000","last_fullsized":"2023-09-12T16:02:10.573443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:33:25.462412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266679+0000","last_change":"2023-09-12T16:02:05.039019+0000","last_active":"2023-09-12T16:02:10.266679+0000","last_peered":"2023-09-12T16:02:10.266679+0000","last_clean":"2023-09-12T16:02:10.266679+0000","last_became_active":"2023-09-12T16:02:05.038621+0000","last_became_peered":"2023-09-12T16:02:05.038621+0000","last_unstale":"2023-09-12T16:02:10.266679+0000","last_undegraded":"2023-09-12T16:02:10.266679+0000","last_fullsized":"2023-09-12T16:02:10.266679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:08:08.934375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892087+0000","last_change":"2023-09-12T16:02:05.039737+0000","last_active":"2023-09-12T16:02:10.892087+0000","last_peered":"2023-09-12T16:02:10.892087+0000","last_clean":"2023-09-12T16:02:10.892087+0000","last_became_active":"2023-09-12T16:02:05.039304+0000","last_became_peered":"2023-09-12T16:02:05.039304+0000","last_unstale":"2023-09-12T16:02:10.892087+0000","last_undegraded":"2023-09-12T16:02:10.892087+0000","last_fullsized":"2023-09-12T16:02:10.892087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:50:13.141915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244717,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5536,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5,"num_read_kb":7,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":17556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514511,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":465670,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776146,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1061458,"data_compressed":50331,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.88100000000000001}]},{"osd":3,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66000000000000003}]},{"osd":4,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84199999999999997}]},{"osd":7,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56899999999999995}]},{"osd":8,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":1.008},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.89500000000000002}]},{"osd":9,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.86699999999999999}]},{"osd":11,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.026}]}]},{"osd":9,"up_from":63,"seq":270582939671,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":466252,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.41999999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.438}]},{"osd":3,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.44900000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70699999999999996}]},{"osd":5,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61799999999999999}]},{"osd":8,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45500000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":8,"up_from":56,"seq":240518168603,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":465965,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.51700000000000002}]},{"osd":1,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.47599999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64100000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82299999999999995}]},{"osd":5,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.622}]},{"osd":6,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.755}]},{"osd":9,"last update":"Tue Sep 12 16:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.84299999999999997}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.88100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397536,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":1056911,"data_compressed":50331,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.66200000000000003}]},{"osd":2,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.64400000000000002}]},{"osd":3,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89100000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.77000000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.58499999999999996}]},{"osd":5,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.621,"15min":0.60399999999999998},"min":{"1min":0.46800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.84699999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.90400000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.82899999999999996}]},{"osd":10,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":6,"up_from":44,"seq":188978561060,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":465670,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.29999999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.376}]},{"osd":1,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92700000000000005,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55000000000000004}]},{"osd":2,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.45900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81899999999999995,"5min":1.129,"15min":1.129},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69999999999999996,"5min":1.083,"15min":1.083},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.504}]},{"osd":4,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91400000000000003,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.48799999999999999}]},{"osd":5,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.51100000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84099999999999997,"5min":1.022,"15min":1.022},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.48999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.623}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71199999999999997}]},{"osd":9,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.59099999999999997}]},{"osd":10,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.72399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":469239,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.50900000000000001,"15min":0.49199999999999999},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.51400000000000001},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56299999999999994,"15min":0.52600000000000002},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.93000000000000005}]},{"osd":3,"last update":"Tue Sep 12 16:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.51000000000000001},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.65600000000000003}]},{"osd":4,"last update":"Tue Sep 12 16:01:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.46700000000000003,"15min":0.44500000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.96399999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.876}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.38100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.91400000000000003}]},{"osd":8,"last update":"Tue Sep 12 16:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":1.071},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.94899999999999995}]},{"osd":9,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":1.0129999999999999}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.0580000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":465760,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60599999999999998},"min":{"1min":0.39900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79800000000000004,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60399999999999998,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79100000000000004,"5min":2.113,"15min":2.113},"last":0.74299999999999999}]},{"osd":2,"last update":"Tue Sep 12 16:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.52900000000000003,"15min":0.50900000000000001},"min":{"1min":0.41799999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.38200000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.72599999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.39000000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.443,"15min":0.433},"min":{"1min":0.309,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.627}]},{"osd":5,"last update":"Tue Sep 12 16:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.32900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.433},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.91100000000000003,"5min":1.004,"15min":1.004},"last":0.41499999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56799999999999995}]},{"osd":8,"last update":"Tue Sep 12 16:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68899999999999995}]},{"osd":9,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.495}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.71199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":469920,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.373,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63100000000000001,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66300000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.51100000000000001,"15min":0.48899999999999999},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.747,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.438},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.51800000000000002,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.313,"15min":0.313},"max":{"1min":0.85599999999999998,"5min":0.877,"15min":0.877},"last":0.54500000000000004}]},{"osd":3,"last update":"Tue Sep 12 16:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54100000000000004,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57099999999999995}]},{"osd":4,"last update":"Tue Sep 12 16:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76200000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76300000000000001}]},{"osd":6,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79500000000000004}]},{"osd":7,"last update":"Tue Sep 12 16:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83899999999999997}]},{"osd":8,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.82499999999999996}]},{"osd":9,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.89900000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":465864,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.33800000000000002},"min":{"1min":0.252,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.26900000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38800000000000001}]},{"osd":1,"last update":"Tue Sep 12 16:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.91800000000000004,"5min":3.552,"15min":3.552},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.45900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69999999999999996,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.64100000000000001}]},{"osd":2,"last update":"Tue Sep 12 16:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84899999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53900000000000003,"15min":0.51700000000000002},"min":{"1min":0.45600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84499999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.46600000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.90100000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.44},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41599999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.56100000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.5,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.749}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66100000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.46500000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953516,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":465419,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.46999999999999997},"min":{"1min":0.38500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.45800000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88800000000000001,"5min":1.458,"15min":1.458},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.30299999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.39300000000000002,"15min":0.36599999999999999},"min":{"1min":0.27900000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76700000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.54800000000000004,"15min":0.52500000000000002},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63300000000000001}]},{"osd":3,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.54300000000000004,"15min":0.496},"min":{"1min":0.45900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":1.383,"15min":1.383},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51000000000000001,"15min":0.48799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54900000000000004}]},{"osd":5,"last update":"Tue Sep 12 16:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86399999999999999,"5min":1.54,"15min":1.54},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.437,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93600000000000005,"5min":1.502,"15min":1.502},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69299999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65300000000000002,"15min":0.64000000000000001},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.639,"15min":1.639},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.749}]},{"osd":8,"last update":"Tue Sep 12 16:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.879}]},{"osd":9,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78800000000000003}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.92800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789991,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1055889,"data_compressed":50331,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.50800000000000001,"15min":0.51100000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.498}]},{"osd":1,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67500000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55200000000000005}]},{"osd":2,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.374,"15min":0.36699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.39400000000000002,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.245,"15min":0.245},"max":{"1min":0.58699999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.66000000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54400000000000004,"15min":0.55200000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71999999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.73599999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77100000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.83799999999999997}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.72599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.626,"15min":0.626},"min":{"1min":0.50600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83299999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73999999999999999}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.72599999999999998}]},{"osd":9,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78900000000000003}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T16:02:28.320 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T16:02:28.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 -- 172.21.15.130:0/3476441176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d01006e0 msgr2=0x7f88d0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 --2- 172.21.15.130:0/3476441176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d01006e0 0x7f88d0100b40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f88c00099c0 tx=0x7f88c0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 -- 172.21.15.130:0/3476441176 shutdown_connections 2023-09-12T16:02:28.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 --2- 172.21.15.130:0/3476441176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d0101080 0x7f88d0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 --2- 172.21.15.130:0/3476441176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d00ff4f0 0x7f88d00ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 --2- 172.21.15.130:0/3476441176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d01006e0 0x7f88d0100b40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 -- 172.21.15.130:0/3476441176 >> 172.21.15.130:0/3476441176 conn(0x7f88d00faa60 msgr2=0x7f88d00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:28.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 -- 172.21.15.130:0/3476441176 shutdown_connections 2023-09-12T16:02:28.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 -- 172.21.15.130:0/3476441176 wait complete. 2023-09-12T16:02:28.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 Processor -- start 2023-09-12T16:02:28.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.471+0000 7f88d64c1700 1 -- start start 2023-09-12T16:02:28.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d64c1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d00ff4f0 0x7f88d00694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:28.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d64c1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d01006e0 0x7f88d019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:28.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d64c1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0101080 0x7f88d019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:28.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d64c1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d019dda0 con 0x7f88d00ff4f0 2023-09-12T16:02:28.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d64c1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d019df10 con 0x7f88d0101080 2023-09-12T16:02:28.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d64c1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d019e0d0 con 0x7f88d01006e0 2023-09-12T16:02:28.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d00ff4f0 0x7f88d00694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:28.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d00ff4f0 0x7f88d00694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53842/0 (socket says 172.21.15.130:53842) 2023-09-12T16:02:28.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cffff700 1 -- 172.21.15.130:0/703315919 learned_addr learned my addr 172.21.15.130:0/703315919 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:28.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d4a5e700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0101080 0x7f88d019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:28.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cf7fe700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d01006e0 0x7f88d019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:28.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cffff700 1 -- 172.21.15.130:0/703315919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d01006e0 msgr2=0x7f88d019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cffff700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d01006e0 0x7f88d019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cffff700 1 -- 172.21.15.130:0/703315919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0101080 msgr2=0x7f88d019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cffff700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0101080 0x7f88d019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cffff700 1 -- 172.21.15.130:0/703315919 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c00095f0 con 0x7f88d00ff4f0 2023-09-12T16:02:28.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d4a5e700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0101080 0x7f88d019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:28.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cffff700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d00ff4f0 0x7f88d00694f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88c400e970 tx=0x7f88c400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:28.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cd7fa700 1 -- 172.21.15.130:0/703315919 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4009790 con 0x7f88d00ff4f0 2023-09-12T16:02:28.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d019e3b0 con 0x7f88d00ff4f0 2023-09-12T16:02:28.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cd7fa700 1 -- 172.21.15.130:0/703315919 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c40044e0 con 0x7f88d00ff4f0 2023-09-12T16:02:28.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88cd7fa700 1 -- 172.21.15.130:0/703315919 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4003e30 con 0x7f88d00ff4f0 2023-09-12T16:02:28.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.472+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d019e910 con 0x7f88d00ff4f0 2023-09-12T16:02:28.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.473+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d010ad10 con 0x7f88d00ff4f0 2023-09-12T16:02:28.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.474+0000 7f88cd7fa700 1 -- 172.21.15.130:0/703315919 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f88c4004000 con 0x7f88d00ff4f0 2023-09-12T16:02:28.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.475+0000 7f88cd7fa700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88b80a0710 0x7f88b80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:28.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.475+0000 7f88cd7fa700 1 -- 172.21.15.130:0/703315919 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f88c4014070 con 0x7f88d00ff4f0 2023-09-12T16:02:28.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.475+0000 7f88cf7fe700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88b80a0710 0x7f88b80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:28.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.475+0000 7f88cf7fe700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88b80a0710 0x7f88b80a2bd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88c000b580 tx=0x7f88c001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:28.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.478+0000 7f88cd7fa700 1 -- 172.21.15.130:0/703315919 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f88c4095b80 con 0x7f88d00ff4f0 2023-09-12T16:02:28.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.670+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88d0060a40 con 0x7f88b80a0710 2023-09-12T16:02:28.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.682+0000 7f88cd7fa700 1 -- 172.21.15.130:0/703315919 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418770 (secure 0 0 0) 0x7f88d0060a40 con 0x7f88b80a0710 2023-09-12T16:02:28.687 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:28.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88b80a0710 msgr2=0x7f88b80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88b80a0710 0x7f88b80a2bd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88c000b580 tx=0x7f88c001a040 comp rx=0 tx=0).stop 2023-09-12T16:02:28.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d00ff4f0 msgr2=0x7f88d00694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:28.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d00ff4f0 0x7f88d00694f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88c400e970 tx=0x7f88c400ed30 comp rx=0 tx=0).stop 2023-09-12T16:02:28.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 shutdown_connections 2023-09-12T16:02:28.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88b80a0710 0x7f88b80a2bd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0101080 0x7f88d019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88d00ff4f0 0x7f88d00694f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 --2- 172.21.15.130:0/703315919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d01006e0 0x7f88d019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:28.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 >> 172.21.15.130:0/703315919 conn(0x7f88d00faa60 msgr2=0x7f88d00fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:28.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 shutdown_connections 2023-09-12T16:02:28.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:28.686+0000 7f88d64c1700 1 -- 172.21.15.130:0/703315919 wait complete. 2023-09-12T16:02:28.693 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2023-09-12T16:02:28.699 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-09-12T16:02:26.760271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595904,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":131,"num_read_kb":116,"num_write":271,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335388,"kb_used_data":11164,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606436,"statfs":{"total":1151940427776,"available":1151596990464,"internally_reserved":0,"allocated":11431936,"data_stored":7374017,"data_compressed":283014,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3126,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":7,"num_write":45,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003104"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892578+0000","last_change":"2023-09-12T16:02:05.039617+0000","last_active":"2023-09-12T16:02:10.892578+0000","last_peered":"2023-09-12T16:02:10.892578+0000","last_clean":"2023-09-12T16:02:10.892578+0000","last_became_active":"2023-09-12T16:02:05.039264+0000","last_became_peered":"2023-09-12T16:02:05.039264+0000","last_unstale":"2023-09-12T16:02:10.892578+0000","last_undegraded":"2023-09-12T16:02:10.892578+0000","last_fullsized":"2023-09-12T16:02:10.892578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:37:01.381406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891691+0000","last_change":"2023-09-12T16:02:05.033151+0000","last_active":"2023-09-12T16:02:10.891691+0000","last_peered":"2023-09-12T16:02:10.891691+0000","last_clean":"2023-09-12T16:02:10.891691+0000","last_became_active":"2023-09-12T16:02:05.032866+0000","last_became_peered":"2023-09-12T16:02:05.032866+0000","last_unstale":"2023-09-12T16:02:10.891691+0000","last_undegraded":"2023-09-12T16:02:10.891691+0000","last_fullsized":"2023-09-12T16:02:10.891691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:35.921428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891781+0000","last_change":"2023-09-12T16:02:05.032478+0000","last_active":"2023-09-12T16:02:10.891781+0000","last_peered":"2023-09-12T16:02:10.891781+0000","last_clean":"2023-09-12T16:02:10.891781+0000","last_became_active":"2023-09-12T16:02:05.032124+0000","last_became_peered":"2023-09-12T16:02:05.032124+0000","last_unstale":"2023-09-12T16:02:10.891781+0000","last_undegraded":"2023-09-12T16:02:10.891781+0000","last_fullsized":"2023-09-12T16:02:10.891781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:58:54.112577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760684+0000","last_change":"2023-09-12T16:02:05.033088+0000","last_active":"2023-09-12T16:02:10.760684+0000","last_peered":"2023-09-12T16:02:10.760684+0000","last_clean":"2023-09-12T16:02:10.760684+0000","last_became_active":"2023-09-12T16:02:05.032332+0000","last_became_peered":"2023-09-12T16:02:05.032332+0000","last_unstale":"2023-09-12T16:02:10.760684+0000","last_undegraded":"2023-09-12T16:02:10.760684+0000","last_fullsized":"2023-09-12T16:02:10.760684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:25:38.751961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305466+0000","last_change":"2023-09-12T16:02:05.036612+0000","last_active":"2023-09-12T16:02:10.305466+0000","last_peered":"2023-09-12T16:02:10.305466+0000","last_clean":"2023-09-12T16:02:10.305466+0000","last_became_active":"2023-09-12T16:02:05.036215+0000","last_became_peered":"2023-09-12T16:02:05.036215+0000","last_unstale":"2023-09-12T16:02:10.305466+0000","last_undegraded":"2023-09-12T16:02:10.305466+0000","last_fullsized":"2023-09-12T16:02:10.305466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:15:04.561751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891896+0000","last_change":"2023-09-12T16:02:05.043816+0000","last_active":"2023-09-12T16:02:10.891896+0000","last_peered":"2023-09-12T16:02:10.891896+0000","last_clean":"2023-09-12T16:02:10.891896+0000","last_became_active":"2023-09-12T16:02:05.043599+0000","last_became_peered":"2023-09-12T16:02:05.043599+0000","last_unstale":"2023-09-12T16:02:10.891896+0000","last_undegraded":"2023-09-12T16:02:10.891896+0000","last_fullsized":"2023-09-12T16:02:10.891896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:32:00.286196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265184+0000","last_change":"2023-09-12T16:02:05.031476+0000","last_active":"2023-09-12T16:02:10.265184+0000","last_peered":"2023-09-12T16:02:10.265184+0000","last_clean":"2023-09-12T16:02:10.265184+0000","last_became_active":"2023-09-12T16:02:05.031150+0000","last_became_peered":"2023-09-12T16:02:05.031150+0000","last_unstale":"2023-09-12T16:02:10.265184+0000","last_undegraded":"2023-09-12T16:02:10.265184+0000","last_fullsized":"2023-09-12T16:02:10.265184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:22:27.405505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.627779+0000","last_change":"2023-09-12T16:02:05.031981+0000","last_active":"2023-09-12T16:02:10.627779+0000","last_peered":"2023-09-12T16:02:10.627779+0000","last_clean":"2023-09-12T16:02:10.627779+0000","last_became_active":"2023-09-12T16:02:05.031692+0000","last_became_peered":"2023-09-12T16:02:05.031692+0000","last_unstale":"2023-09-12T16:02:10.627779+0000","last_undegraded":"2023-09-12T16:02:10.627779+0000","last_fullsized":"2023-09-12T16:02:10.627779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:51:00.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573730+0000","last_change":"2023-09-12T16:02:05.029898+0000","last_active":"2023-09-12T16:02:10.573730+0000","last_peered":"2023-09-12T16:02:10.573730+0000","last_clean":"2023-09-12T16:02:10.573730+0000","last_became_active":"2023-09-12T16:02:05.029731+0000","last_became_peered":"2023-09-12T16:02:05.029731+0000","last_unstale":"2023-09-12T16:02:10.573730+0000","last_undegraded":"2023-09-12T16:02:10.573730+0000","last_fullsized":"2023-09-12T16:02:10.573730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:16:31.194561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892310+0000","last_change":"2023-09-12T16:02:05.044846+0000","last_active":"2023-09-12T16:02:10.892310+0000","last_peered":"2023-09-12T16:02:10.892310+0000","last_clean":"2023-09-12T16:02:10.892310+0000","last_became_active":"2023-09-12T16:02:05.044430+0000","last_became_peered":"2023-09-12T16:02:05.044430+0000","last_unstale":"2023-09-12T16:02:10.892310+0000","last_undegraded":"2023-09-12T16:02:10.892310+0000","last_fullsized":"2023-09-12T16:02:10.892310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:15.999080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265884+0000","last_change":"2023-09-12T16:02:05.030285+0000","last_active":"2023-09-12T16:02:10.265884+0000","last_peered":"2023-09-12T16:02:10.265884+0000","last_clean":"2023-09-12T16:02:10.265884+0000","last_became_active":"2023-09-12T16:02:05.029585+0000","last_became_peered":"2023-09-12T16:02:05.029585+0000","last_unstale":"2023-09-12T16:02:10.265884+0000","last_undegraded":"2023-09-12T16:02:10.265884+0000","last_fullsized":"2023-09-12T16:02:10.265884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:52:27.471412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760652+0000","last_change":"2023-09-12T16:02:05.037258+0000","last_active":"2023-09-12T16:02:10.760652+0000","last_peered":"2023-09-12T16:02:10.760652+0000","last_clean":"2023-09-12T16:02:10.760652+0000","last_became_active":"2023-09-12T16:02:05.036941+0000","last_became_peered":"2023-09-12T16:02:05.036941+0000","last_unstale":"2023-09-12T16:02:10.760652+0000","last_undegraded":"2023-09-12T16:02:10.760652+0000","last_fullsized":"2023-09-12T16:02:10.760652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:04:17.998927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266860+0000","last_change":"2023-09-12T16:02:05.041986+0000","last_active":"2023-09-12T16:02:10.266860+0000","last_peered":"2023-09-12T16:02:10.266860+0000","last_clean":"2023-09-12T16:02:10.266860+0000","last_became_active":"2023-09-12T16:02:05.041584+0000","last_became_peered":"2023-09-12T16:02:05.041584+0000","last_unstale":"2023-09-12T16:02:10.266860+0000","last_undegraded":"2023-09-12T16:02:10.266860+0000","last_fullsized":"2023-09-12T16:02:10.266860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:39.065525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891874+0000","last_change":"2023-09-12T16:02:05.030405+0000","last_active":"2023-09-12T16:02:10.891874+0000","last_peered":"2023-09-12T16:02:10.891874+0000","last_clean":"2023-09-12T16:02:10.891874+0000","last_became_active":"2023-09-12T16:02:05.029488+0000","last_became_peered":"2023-09-12T16:02:05.029488+0000","last_unstale":"2023-09-12T16:02:10.891874+0000","last_undegraded":"2023-09-12T16:02:10.891874+0000","last_fullsized":"2023-09-12T16:02:10.891874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.646069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891352+0000","last_change":"2023-09-12T16:02:05.032576+0000","last_active":"2023-09-12T16:02:10.891352+0000","last_peered":"2023-09-12T16:02:10.891352+0000","last_clean":"2023-09-12T16:02:10.891352+0000","last_became_active":"2023-09-12T16:02:05.032277+0000","last_became_peered":"2023-09-12T16:02:05.032277+0000","last_unstale":"2023-09-12T16:02:10.891352+0000","last_undegraded":"2023-09-12T16:02:10.891352+0000","last_fullsized":"2023-09-12T16:02:10.891352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:47:32.734830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573074+0000","last_change":"2023-09-12T16:02:05.023468+0000","last_active":"2023-09-12T16:02:10.573074+0000","last_peered":"2023-09-12T16:02:10.573074+0000","last_clean":"2023-09-12T16:02:10.573074+0000","last_became_active":"2023-09-12T16:02:05.023132+0000","last_became_peered":"2023-09-12T16:02:05.023132+0000","last_unstale":"2023-09-12T16:02:10.573074+0000","last_undegraded":"2023-09-12T16:02:10.573074+0000","last_fullsized":"2023-09-12T16:02:10.573074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:21:49.378779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892504+0000","last_change":"2023-09-12T16:02:05.039910+0000","last_active":"2023-09-12T16:02:10.892504+0000","last_peered":"2023-09-12T16:02:10.892504+0000","last_clean":"2023-09-12T16:02:10.892504+0000","last_became_active":"2023-09-12T16:02:05.039605+0000","last_became_peered":"2023-09-12T16:02:05.039605+0000","last_unstale":"2023-09-12T16:02:10.892504+0000","last_undegraded":"2023-09-12T16:02:10.892504+0000","last_fullsized":"2023-09-12T16:02:10.892504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:19:58.823471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892624+0000","last_change":"2023-09-12T16:02:05.031667+0000","last_active":"2023-09-12T16:02:10.892624+0000","last_peered":"2023-09-12T16:02:10.892624+0000","last_clean":"2023-09-12T16:02:10.892624+0000","last_became_active":"2023-09-12T16:02:05.030886+0000","last_became_peered":"2023-09-12T16:02:05.030886+0000","last_unstale":"2023-09-12T16:02:10.892624+0000","last_undegraded":"2023-09-12T16:02:10.892624+0000","last_fullsized":"2023-09-12T16:02:10.892624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:24:02.312902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892704+0000","last_change":"2023-09-12T16:02:05.037607+0000","last_active":"2023-09-12T16:02:10.892704+0000","last_peered":"2023-09-12T16:02:10.892704+0000","last_clean":"2023-09-12T16:02:10.892704+0000","last_became_active":"2023-09-12T16:02:05.037095+0000","last_became_peered":"2023-09-12T16:02:05.037095+0000","last_unstale":"2023-09-12T16:02:10.892704+0000","last_undegraded":"2023-09-12T16:02:10.892704+0000","last_fullsized":"2023-09-12T16:02:10.892704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:17:04.886737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266183+0000","last_change":"2023-09-12T16:02:09.292093+0000","last_active":"2023-09-12T16:02:11.266183+0000","last_peered":"2023-09-12T16:02:11.266183+0000","last_clean":"2023-09-12T16:02:11.266183+0000","last_became_active":"2023-09-12T16:02:09.291602+0000","last_became_peered":"2023-09-12T16:02:09.291602+0000","last_unstale":"2023-09-12T16:02:11.266183+0000","last_undegraded":"2023-09-12T16:02:11.266183+0000","last_fullsized":"2023-09-12T16:02:11.266183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:36:23.089498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892761+0000","last_change":"2023-09-12T16:02:04.029311+0000","last_active":"2023-09-12T16:02:10.892761+0000","last_peered":"2023-09-12T16:02:10.892761+0000","last_clean":"2023-09-12T16:02:10.892761+0000","last_became_active":"2023-09-12T16:02:04.028956+0000","last_became_peered":"2023-09-12T16:02:04.028956+0000","last_unstale":"2023-09-12T16:02:10.892761+0000","last_undegraded":"2023-09-12T16:02:10.892761+0000","last_fullsized":"2023-09-12T16:02:10.892761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:51:43.855047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.628092+0000","last_change":"2023-09-12T16:02:05.032191+0000","last_active":"2023-09-12T16:02:10.628092+0000","last_peered":"2023-09-12T16:02:10.628092+0000","last_clean":"2023-09-12T16:02:10.628092+0000","last_became_active":"2023-09-12T16:02:05.031888+0000","last_became_peered":"2023-09-12T16:02:05.031888+0000","last_unstale":"2023-09-12T16:02:10.628092+0000","last_undegraded":"2023-09-12T16:02:10.628092+0000","last_fullsized":"2023-09-12T16:02:10.628092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:50:15.775078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792837+0000","last_change":"2023-09-12T16:02:09.290650+0000","last_active":"2023-09-12T16:02:11.792837+0000","last_peered":"2023-09-12T16:02:11.792837+0000","last_clean":"2023-09-12T16:02:11.792837+0000","last_became_active":"2023-09-12T16:02:09.290205+0000","last_became_peered":"2023-09-12T16:02:09.290205+0000","last_unstale":"2023-09-12T16:02:11.792837+0000","last_undegraded":"2023-09-12T16:02:11.792837+0000","last_fullsized":"2023-09-12T16:02:11.792837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:44:04.930063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.265957+0000","last_change":"2023-09-12T16:02:04.032506+0000","last_active":"2023-09-12T16:02:11.265957+0000","last_peered":"2023-09-12T16:02:11.265957+0000","last_clean":"2023-09-12T16:02:11.265957+0000","last_became_active":"2023-09-12T16:02:04.032047+0000","last_became_peered":"2023-09-12T16:02:04.032047+0000","last_unstale":"2023-09-12T16:02:11.265957+0000","last_undegraded":"2023-09-12T16:02:11.265957+0000","last_fullsized":"2023-09-12T16:02:11.265957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:30:22.664465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891413+0000","last_change":"2023-09-12T16:02:05.025692+0000","last_active":"2023-09-12T16:02:10.891413+0000","last_peered":"2023-09-12T16:02:10.891413+0000","last_clean":"2023-09-12T16:02:10.891413+0000","last_became_active":"2023-09-12T16:02:05.025370+0000","last_became_peered":"2023-09-12T16:02:05.025370+0000","last_unstale":"2023-09-12T16:02:10.891413+0000","last_undegraded":"2023-09-12T16:02:10.891413+0000","last_fullsized":"2023-09-12T16:02:10.891413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.305375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793714+0000","last_change":"2023-09-12T16:02:09.290547+0000","last_active":"2023-09-12T16:02:11.793714+0000","last_peered":"2023-09-12T16:02:11.793714+0000","last_clean":"2023-09-12T16:02:11.793714+0000","last_became_active":"2023-09-12T16:02:09.290120+0000","last_became_peered":"2023-09-12T16:02:09.290120+0000","last_unstale":"2023-09-12T16:02:11.793714+0000","last_undegraded":"2023-09-12T16:02:11.793714+0000","last_fullsized":"2023-09-12T16:02:11.793714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:31:25.364487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.649918+0000","last_change":"2023-09-12T16:02:04.032079+0000","last_active":"2023-09-12T16:02:24.649918+0000","last_peered":"2023-09-12T16:02:24.649918+0000","last_clean":"2023-09-12T16:02:24.649918+0000","last_became_active":"2023-09-12T16:02:04.031835+0000","last_became_peered":"2023-09-12T16:02:04.031835+0000","last_unstale":"2023-09-12T16:02:24.649918+0000","last_undegraded":"2023-09-12T16:02:24.649918+0000","last_fullsized":"2023-09-12T16:02:24.649918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:48:33.935012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573472+0000","last_change":"2023-09-12T16:02:05.037671+0000","last_active":"2023-09-12T16:02:10.573472+0000","last_peered":"2023-09-12T16:02:10.573472+0000","last_clean":"2023-09-12T16:02:10.573472+0000","last_became_active":"2023-09-12T16:02:05.037307+0000","last_became_peered":"2023-09-12T16:02:05.037307+0000","last_unstale":"2023-09-12T16:02:10.573472+0000","last_undegraded":"2023-09-12T16:02:10.573472+0000","last_fullsized":"2023-09-12T16:02:10.573472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:47:23.836612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792372+0000","last_change":"2023-09-12T16:02:09.289165+0000","last_active":"2023-09-12T16:02:11.792372+0000","last_peered":"2023-09-12T16:02:11.792372+0000","last_clean":"2023-09-12T16:02:11.792372+0000","last_became_active":"2023-09-12T16:02:09.288903+0000","last_became_peered":"2023-09-12T16:02:09.288903+0000","last_unstale":"2023-09-12T16:02:11.792372+0000","last_undegraded":"2023-09-12T16:02:11.792372+0000","last_fullsized":"2023-09-12T16:02:11.792372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:27:09.160822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305747+0000","last_change":"2023-09-12T16:02:04.027279+0000","last_active":"2023-09-12T16:02:10.305747+0000","last_peered":"2023-09-12T16:02:10.305747+0000","last_clean":"2023-09-12T16:02:10.305747+0000","last_became_active":"2023-09-12T16:02:04.026975+0000","last_became_peered":"2023-09-12T16:02:04.026975+0000","last_unstale":"2023-09-12T16:02:10.305747+0000","last_undegraded":"2023-09-12T16:02:10.305747+0000","last_fullsized":"2023-09-12T16:02:10.305747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:45.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265479+0000","last_change":"2023-09-12T16:02:05.027582+0000","last_active":"2023-09-12T16:02:10.265479+0000","last_peered":"2023-09-12T16:02:10.265479+0000","last_clean":"2023-09-12T16:02:10.265479+0000","last_became_active":"2023-09-12T16:02:05.027236+0000","last_became_peered":"2023-09-12T16:02:05.027236+0000","last_unstale":"2023-09-12T16:02:10.265479+0000","last_undegraded":"2023-09-12T16:02:10.265479+0000","last_fullsized":"2023-09-12T16:02:10.265479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:06:24.812731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266605+0000","last_change":"2023-09-12T16:02:09.284381+0000","last_active":"2023-09-12T16:02:11.266605+0000","last_peered":"2023-09-12T16:02:11.266605+0000","last_clean":"2023-09-12T16:02:11.266605+0000","last_became_active":"2023-09-12T16:02:09.283947+0000","last_became_peered":"2023-09-12T16:02:09.283947+0000","last_unstale":"2023-09-12T16:02:11.266605+0000","last_undegraded":"2023-09-12T16:02:11.266605+0000","last_fullsized":"2023-09-12T16:02:11.266605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:05:24.803781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"80'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265192+0000","last_change":"2023-09-12T16:02:04.027693+0000","last_active":"2023-09-12T16:02:10.265192+0000","last_peered":"2023-09-12T16:02:10.265192+0000","last_clean":"2023-09-12T16:02:10.265192+0000","last_became_active":"2023-09-12T16:02:04.027315+0000","last_became_peered":"2023-09-12T16:02:04.027315+0000","last_unstale":"2023-09-12T16:02:10.265192+0000","last_undegraded":"2023-09-12T16:02:10.265192+0000","last_fullsized":"2023-09-12T16:02:10.265192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:32:21.756802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305697+0000","last_change":"2023-09-12T16:02:05.031673+0000","last_active":"2023-09-12T16:02:10.305697+0000","last_peered":"2023-09-12T16:02:10.305697+0000","last_clean":"2023-09-12T16:02:10.305697+0000","last_became_active":"2023-09-12T16:02:05.031364+0000","last_became_peered":"2023-09-12T16:02:05.031364+0000","last_unstale":"2023-09-12T16:02:10.305697+0000","last_undegraded":"2023-09-12T16:02:10.305697+0000","last_fullsized":"2023-09-12T16:02:10.305697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:48:05.222530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266486+0000","last_change":"2023-09-12T16:02:09.288536+0000","last_active":"2023-09-12T16:02:11.266486+0000","last_peered":"2023-09-12T16:02:11.266486+0000","last_clean":"2023-09-12T16:02:11.266486+0000","last_became_active":"2023-09-12T16:02:09.288062+0000","last_became_peered":"2023-09-12T16:02:09.288062+0000","last_unstale":"2023-09-12T16:02:11.266486+0000","last_undegraded":"2023-09-12T16:02:11.266486+0000","last_fullsized":"2023-09-12T16:02:11.266486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:39:28.935563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.649762+0000","last_change":"2023-09-12T16:02:04.026298+0000","last_active":"2023-09-12T16:02:24.649762+0000","last_peered":"2023-09-12T16:02:24.649762+0000","last_clean":"2023-09-12T16:02:24.649762+0000","last_became_active":"2023-09-12T16:02:04.025975+0000","last_became_peered":"2023-09-12T16:02:04.025975+0000","last_unstale":"2023-09-12T16:02:24.649762+0000","last_undegraded":"2023-09-12T16:02:24.649762+0000","last_fullsized":"2023-09-12T16:02:24.649762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:12:04.338497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265378+0000","last_change":"2023-09-12T16:02:05.028337+0000","last_active":"2023-09-12T16:02:10.265378+0000","last_peered":"2023-09-12T16:02:10.265378+0000","last_clean":"2023-09-12T16:02:10.265378+0000","last_became_active":"2023-09-12T16:02:05.028036+0000","last_became_peered":"2023-09-12T16:02:05.028036+0000","last_unstale":"2023-09-12T16:02:10.265378+0000","last_undegraded":"2023-09-12T16:02:10.265378+0000","last_fullsized":"2023-09-12T16:02:10.265378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:39:09.387191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793737+0000","last_change":"2023-09-12T16:02:09.290950+0000","last_active":"2023-09-12T16:02:11.793737+0000","last_peered":"2023-09-12T16:02:11.793737+0000","last_clean":"2023-09-12T16:02:11.793737+0000","last_became_active":"2023-09-12T16:02:09.290477+0000","last_became_peered":"2023-09-12T16:02:09.290477+0000","last_unstale":"2023-09-12T16:02:11.793737+0000","last_undegraded":"2023-09-12T16:02:11.793737+0000","last_fullsized":"2023-09-12T16:02:11.793737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:53:45.907575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266543+0000","last_change":"2023-09-12T16:02:04.026025+0000","last_active":"2023-09-12T16:02:10.266543+0000","last_peered":"2023-09-12T16:02:10.266543+0000","last_clean":"2023-09-12T16:02:10.266543+0000","last_became_active":"2023-09-12T16:02:04.025722+0000","last_became_peered":"2023-09-12T16:02:04.025722+0000","last_unstale":"2023-09-12T16:02:10.266543+0000","last_undegraded":"2023-09-12T16:02:10.266543+0000","last_fullsized":"2023-09-12T16:02:10.266543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:29:58.357960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.628351+0000","last_change":"2023-09-12T16:02:05.028461+0000","last_active":"2023-09-12T16:02:10.628351+0000","last_peered":"2023-09-12T16:02:10.628351+0000","last_clean":"2023-09-12T16:02:10.628351+0000","last_became_active":"2023-09-12T16:02:05.028176+0000","last_became_peered":"2023-09-12T16:02:05.028176+0000","last_unstale":"2023-09-12T16:02:10.628351+0000","last_undegraded":"2023-09-12T16:02:10.628351+0000","last_fullsized":"2023-09-12T16:02:10.628351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:16:48.652513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:12.221469+0000","last_change":"2023-09-12T16:02:09.295245+0000","last_active":"2023-09-12T16:02:12.221469+0000","last_peered":"2023-09-12T16:02:12.221469+0000","last_clean":"2023-09-12T16:02:12.221469+0000","last_became_active":"2023-09-12T16:02:09.294872+0000","last_became_peered":"2023-09-12T16:02:09.294872+0000","last_unstale":"2023-09-12T16:02:12.221469+0000","last_undegraded":"2023-09-12T16:02:12.221469+0000","last_fullsized":"2023-09-12T16:02:12.221469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:43:49.462317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.264897+0000","last_change":"2023-09-12T16:02:04.030815+0000","last_active":"2023-09-12T16:02:10.264897+0000","last_peered":"2023-09-12T16:02:10.264897+0000","last_clean":"2023-09-12T16:02:10.264897+0000","last_became_active":"2023-09-12T16:02:04.030435+0000","last_became_peered":"2023-09-12T16:02:04.030435+0000","last_unstale":"2023-09-12T16:02:10.264897+0000","last_undegraded":"2023-09-12T16:02:10.264897+0000","last_fullsized":"2023-09-12T16:02:10.264897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:39:59.661143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891762+0000","last_change":"2023-09-12T16:02:05.031155+0000","last_active":"2023-09-12T16:02:10.891762+0000","last_peered":"2023-09-12T16:02:10.891762+0000","last_clean":"2023-09-12T16:02:10.891762+0000","last_became_active":"2023-09-12T16:02:05.030803+0000","last_became_peered":"2023-09-12T16:02:05.030803+0000","last_unstale":"2023-09-12T16:02:10.891762+0000","last_undegraded":"2023-09-12T16:02:10.891762+0000","last_fullsized":"2023-09-12T16:02:10.891762+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:55:21.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.265570+0000","last_change":"2023-09-12T16:02:09.286445+0000","last_active":"2023-09-12T16:02:11.265570+0000","last_peered":"2023-09-12T16:02:11.265570+0000","last_clean":"2023-09-12T16:02:11.265570+0000","last_became_active":"2023-09-12T16:02:09.286049+0000","last_became_peered":"2023-09-12T16:02:09.286049+0000","last_unstale":"2023-09-12T16:02:11.265570+0000","last_undegraded":"2023-09-12T16:02:11.265570+0000","last_fullsized":"2023-09-12T16:02:11.265570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:43:13.943984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:23.645323+0000","last_change":"2023-09-12T16:02:04.035341+0000","last_active":"2023-09-12T16:02:23.645323+0000","last_peered":"2023-09-12T16:02:23.645323+0000","last_clean":"2023-09-12T16:02:23.645323+0000","last_became_active":"2023-09-12T16:02:04.035092+0000","last_became_peered":"2023-09-12T16:02:04.035092+0000","last_unstale":"2023-09-12T16:02:23.645323+0000","last_undegraded":"2023-09-12T16:02:23.645323+0000","last_fullsized":"2023-09-12T16:02:23.645323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:33:00.784401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573712+0000","last_change":"2023-09-12T16:02:05.025229+0000","last_active":"2023-09-12T16:02:10.573712+0000","last_peered":"2023-09-12T16:02:10.573712+0000","last_clean":"2023-09-12T16:02:10.573712+0000","last_became_active":"2023-09-12T16:02:05.024963+0000","last_became_peered":"2023-09-12T16:02:05.024963+0000","last_unstale":"2023-09-12T16:02:10.573712+0000","last_undegraded":"2023-09-12T16:02:10.573712+0000","last_fullsized":"2023-09-12T16:02:10.573712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:46:04.318082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792883+0000","last_change":"2023-09-12T16:02:09.295011+0000","last_active":"2023-09-12T16:02:11.792883+0000","last_peered":"2023-09-12T16:02:11.792883+0000","last_clean":"2023-09-12T16:02:11.792883+0000","last_became_active":"2023-09-12T16:02:09.294617+0000","last_became_peered":"2023-09-12T16:02:09.294617+0000","last_unstale":"2023-09-12T16:02:11.792883+0000","last_undegraded":"2023-09-12T16:02:11.792883+0000","last_fullsized":"2023-09-12T16:02:11.792883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:57:53.902946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:23.643545+0000","last_change":"2023-09-12T16:02:04.028086+0000","last_active":"2023-09-12T16:02:23.643545+0000","last_peered":"2023-09-12T16:02:23.643545+0000","last_clean":"2023-09-12T16:02:23.643545+0000","last_became_active":"2023-09-12T16:02:04.027798+0000","last_became_peered":"2023-09-12T16:02:04.027798+0000","last_unstale":"2023-09-12T16:02:23.643545+0000","last_undegraded":"2023-09-12T16:02:23.643545+0000","last_fullsized":"2023-09-12T16:02:23.643545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:11:16.921824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891728+0000","last_change":"2023-09-12T16:02:05.030691+0000","last_active":"2023-09-12T16:02:10.891728+0000","last_peered":"2023-09-12T16:02:10.891728+0000","last_clean":"2023-09-12T16:02:10.891728+0000","last_became_active":"2023-09-12T16:02:05.030379+0000","last_became_peered":"2023-09-12T16:02:05.030379+0000","last_unstale":"2023-09-12T16:02:10.891728+0000","last_undegraded":"2023-09-12T16:02:10.891728+0000","last_fullsized":"2023-09-12T16:02:10.891728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:15:05.221078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793996+0000","last_change":"2023-09-12T16:02:09.290703+0000","last_active":"2023-09-12T16:02:11.793996+0000","last_peered":"2023-09-12T16:02:11.793996+0000","last_clean":"2023-09-12T16:02:11.793996+0000","last_became_active":"2023-09-12T16:02:09.290227+0000","last_became_peered":"2023-09-12T16:02:09.290227+0000","last_unstale":"2023-09-12T16:02:11.793996+0000","last_undegraded":"2023-09-12T16:02:11.793996+0000","last_fullsized":"2023-09-12T16:02:11.793996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:07:06.182295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"80'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265450+0000","last_change":"2023-09-12T16:02:04.028630+0000","last_active":"2023-09-12T16:02:10.265450+0000","last_peered":"2023-09-12T16:02:10.265450+0000","last_clean":"2023-09-12T16:02:10.265450+0000","last_became_active":"2023-09-12T16:02:04.028319+0000","last_became_peered":"2023-09-12T16:02:04.028319+0000","last_unstale":"2023-09-12T16:02:10.265450+0000","last_undegraded":"2023-09-12T16:02:10.265450+0000","last_fullsized":"2023-09-12T16:02:10.265450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:13:43.349425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573598+0000","last_change":"2023-09-12T16:02:05.032392+0000","last_active":"2023-09-12T16:02:10.573598+0000","last_peered":"2023-09-12T16:02:10.573598+0000","last_clean":"2023-09-12T16:02:10.573598+0000","last_became_active":"2023-09-12T16:02:05.032099+0000","last_became_peered":"2023-09-12T16:02:05.032099+0000","last_unstale":"2023-09-12T16:02:10.573598+0000","last_undegraded":"2023-09-12T16:02:10.573598+0000","last_fullsized":"2023-09-12T16:02:10.573598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:46:03.108459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792388+0000","last_change":"2023-09-12T16:02:09.289573+0000","last_active":"2023-09-12T16:02:11.792388+0000","last_peered":"2023-09-12T16:02:11.792388+0000","last_clean":"2023-09-12T16:02:11.792388+0000","last_became_active":"2023-09-12T16:02:09.289179+0000","last_became_peered":"2023-09-12T16:02:09.289179+0000","last_unstale":"2023-09-12T16:02:11.792388+0000","last_undegraded":"2023-09-12T16:02:11.792388+0000","last_fullsized":"2023-09-12T16:02:11.792388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:52:35.905346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'12","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:25.657479+0000","last_change":"2023-09-12T16:02:04.025347+0000","last_active":"2023-09-12T16:02:25.657479+0000","last_peered":"2023-09-12T16:02:25.657479+0000","last_clean":"2023-09-12T16:02:25.657479+0000","last_became_active":"2023-09-12T16:02:04.025017+0000","last_became_peered":"2023-09-12T16:02:04.025017+0000","last_unstale":"2023-09-12T16:02:25.657479+0000","last_undegraded":"2023-09-12T16:02:25.657479+0000","last_fullsized":"2023-09-12T16:02:25.657479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:03:46.502387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305835+0000","last_change":"2023-09-12T16:02:05.031409+0000","last_active":"2023-09-12T16:02:10.305835+0000","last_peered":"2023-09-12T16:02:10.305835+0000","last_clean":"2023-09-12T16:02:10.305835+0000","last_became_active":"2023-09-12T16:02:05.031110+0000","last_became_peered":"2023-09-12T16:02:05.031110+0000","last_unstale":"2023-09-12T16:02:10.305835+0000","last_undegraded":"2023-09-12T16:02:10.305835+0000","last_fullsized":"2023-09-12T16:02:10.305835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:10:54.482078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.794317+0000","last_change":"2023-09-12T16:02:09.291493+0000","last_active":"2023-09-12T16:02:11.794317+0000","last_peered":"2023-09-12T16:02:11.794317+0000","last_clean":"2023-09-12T16:02:11.794317+0000","last_became_active":"2023-09-12T16:02:09.291044+0000","last_became_peered":"2023-09-12T16:02:09.291044+0000","last_unstale":"2023-09-12T16:02:11.794317+0000","last_undegraded":"2023-09-12T16:02:11.794317+0000","last_fullsized":"2023-09-12T16:02:11.794317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:34:30.269720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.650789+0000","last_change":"2023-09-12T16:02:04.029283+0000","last_active":"2023-09-12T16:02:24.650789+0000","last_peered":"2023-09-12T16:02:24.650789+0000","last_clean":"2023-09-12T16:02:24.650789+0000","last_became_active":"2023-09-12T16:02:04.028959+0000","last_became_peered":"2023-09-12T16:02:04.028959+0000","last_unstale":"2023-09-12T16:02:24.650789+0000","last_undegraded":"2023-09-12T16:02:24.650789+0000","last_fullsized":"2023-09-12T16:02:24.650789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:28:52.750278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266703+0000","last_change":"2023-09-12T16:02:05.034140+0000","last_active":"2023-09-12T16:02:10.266703+0000","last_peered":"2023-09-12T16:02:10.266703+0000","last_clean":"2023-09-12T16:02:10.266703+0000","last_became_active":"2023-09-12T16:02:05.033840+0000","last_became_peered":"2023-09-12T16:02:05.033840+0000","last_unstale":"2023-09-12T16:02:10.266703+0000","last_undegraded":"2023-09-12T16:02:10.266703+0000","last_fullsized":"2023-09-12T16:02:10.266703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:59:55.522608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793367+0000","last_change":"2023-09-12T16:02:09.289998+0000","last_active":"2023-09-12T16:02:11.793367+0000","last_peered":"2023-09-12T16:02:11.793367+0000","last_clean":"2023-09-12T16:02:11.793367+0000","last_became_active":"2023-09-12T16:02:09.289561+0000","last_became_peered":"2023-09-12T16:02:09.289561+0000","last_unstale":"2023-09-12T16:02:11.793367+0000","last_undegraded":"2023-09-12T16:02:11.793367+0000","last_fullsized":"2023-09-12T16:02:11.793367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:09:47.327213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265335+0000","last_change":"2023-09-12T16:02:04.029883+0000","last_active":"2023-09-12T16:02:10.265335+0000","last_peered":"2023-09-12T16:02:10.265335+0000","last_clean":"2023-09-12T16:02:10.265335+0000","last_became_active":"2023-09-12T16:02:04.029547+0000","last_became_peered":"2023-09-12T16:02:04.029547+0000","last_unstale":"2023-09-12T16:02:10.265335+0000","last_undegraded":"2023-09-12T16:02:10.265335+0000","last_fullsized":"2023-09-12T16:02:10.265335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:47:46.478325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265519+0000","last_change":"2023-09-12T16:02:05.036377+0000","last_active":"2023-09-12T16:02:10.265519+0000","last_peered":"2023-09-12T16:02:10.265519+0000","last_clean":"2023-09-12T16:02:10.265519+0000","last_became_active":"2023-09-12T16:02:05.036060+0000","last_became_peered":"2023-09-12T16:02:05.036060+0000","last_unstale":"2023-09-12T16:02:10.265519+0000","last_undegraded":"2023-09-12T16:02:10.265519+0000","last_fullsized":"2023-09-12T16:02:10.265519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:26:16.857431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792929+0000","last_change":"2023-09-12T16:02:09.295151+0000","last_active":"2023-09-12T16:02:11.792929+0000","last_peered":"2023-09-12T16:02:11.792929+0000","last_clean":"2023-09-12T16:02:11.792929+0000","last_became_active":"2023-09-12T16:02:09.294732+0000","last_became_peered":"2023-09-12T16:02:09.294732+0000","last_unstale":"2023-09-12T16:02:11.792929+0000","last_undegraded":"2023-09-12T16:02:11.792929+0000","last_fullsized":"2023-09-12T16:02:11.792929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:47:19.236310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760396+0000","last_change":"2023-09-12T16:02:04.025723+0000","last_active":"2023-09-12T16:02:10.760396+0000","last_peered":"2023-09-12T16:02:10.760396+0000","last_clean":"2023-09-12T16:02:10.760396+0000","last_became_active":"2023-09-12T16:02:04.025432+0000","last_became_peered":"2023-09-12T16:02:04.025432+0000","last_unstale":"2023-09-12T16:02:10.760396+0000","last_undegraded":"2023-09-12T16:02:10.760396+0000","last_fullsized":"2023-09-12T16:02:10.760396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:56.279910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265229+0000","last_change":"2023-09-12T16:02:05.031741+0000","last_active":"2023-09-12T16:02:10.265229+0000","last_peered":"2023-09-12T16:02:10.265229+0000","last_clean":"2023-09-12T16:02:10.265229+0000","last_became_active":"2023-09-12T16:02:05.031233+0000","last_became_peered":"2023-09-12T16:02:05.031233+0000","last_unstale":"2023-09-12T16:02:10.265229+0000","last_undegraded":"2023-09-12T16:02:10.265229+0000","last_fullsized":"2023-09-12T16:02:10.265229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:55:33.673706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793234+0000","last_change":"2023-09-12T16:02:09.286447+0000","last_active":"2023-09-12T16:02:11.793234+0000","last_peered":"2023-09-12T16:02:11.793234+0000","last_clean":"2023-09-12T16:02:11.793234+0000","last_became_active":"2023-09-12T16:02:09.285805+0000","last_became_peered":"2023-09-12T16:02:09.285805+0000","last_unstale":"2023-09-12T16:02:11.793234+0000","last_undegraded":"2023-09-12T16:02:11.793234+0000","last_fullsized":"2023-09-12T16:02:11.793234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:35:05.518872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.650732+0000","last_change":"2023-09-12T16:02:04.032527+0000","last_active":"2023-09-12T16:02:24.650732+0000","last_peered":"2023-09-12T16:02:24.650732+0000","last_clean":"2023-09-12T16:02:24.650732+0000","last_became_active":"2023-09-12T16:02:04.032264+0000","last_became_peered":"2023-09-12T16:02:04.032264+0000","last_unstale":"2023-09-12T16:02:24.650732+0000","last_undegraded":"2023-09-12T16:02:24.650732+0000","last_fullsized":"2023-09-12T16:02:24.650732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:53:52.433549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265316+0000","last_change":"2023-09-12T16:02:05.029281+0000","last_active":"2023-09-12T16:02:10.265316+0000","last_peered":"2023-09-12T16:02:10.265316+0000","last_clean":"2023-09-12T16:02:10.265316+0000","last_became_active":"2023-09-12T16:02:05.028919+0000","last_became_peered":"2023-09-12T16:02:05.028919+0000","last_unstale":"2023-09-12T16:02:10.265316+0000","last_undegraded":"2023-09-12T16:02:10.265316+0000","last_fullsized":"2023-09-12T16:02:10.265316+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:26.667060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792839+0000","last_change":"2023-09-12T16:02:09.295801+0000","last_active":"2023-09-12T16:02:11.792839+0000","last_peered":"2023-09-12T16:02:11.792839+0000","last_clean":"2023-09-12T16:02:11.792839+0000","last_became_active":"2023-09-12T16:02:09.295472+0000","last_became_peered":"2023-09-12T16:02:09.295472+0000","last_unstale":"2023-09-12T16:02:11.792839+0000","last_undegraded":"2023-09-12T16:02:11.792839+0000","last_fullsized":"2023-09-12T16:02:11.792839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:35:37.628783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305913+0000","last_change":"2023-09-12T16:02:04.026373+0000","last_active":"2023-09-12T16:02:10.305913+0000","last_peered":"2023-09-12T16:02:10.305913+0000","last_clean":"2023-09-12T16:02:10.305913+0000","last_became_active":"2023-09-12T16:02:04.025907+0000","last_became_peered":"2023-09-12T16:02:04.025907+0000","last_unstale":"2023-09-12T16:02:10.305913+0000","last_undegraded":"2023-09-12T16:02:10.305913+0000","last_fullsized":"2023-09-12T16:02:10.305913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:33:15.152128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892162+0000","last_change":"2023-09-12T16:02:05.040143+0000","last_active":"2023-09-12T16:02:10.892162+0000","last_peered":"2023-09-12T16:02:10.892162+0000","last_clean":"2023-09-12T16:02:10.892162+0000","last_became_active":"2023-09-12T16:02:05.039749+0000","last_became_peered":"2023-09-12T16:02:05.039749+0000","last_unstale":"2023-09-12T16:02:10.892162+0000","last_undegraded":"2023-09-12T16:02:10.892162+0000","last_fullsized":"2023-09-12T16:02:10.892162+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:49.561252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793807+0000","last_change":"2023-09-12T16:02:09.841670+0000","last_active":"2023-09-12T16:02:11.793807+0000","last_peered":"2023-09-12T16:02:11.793807+0000","last_clean":"2023-09-12T16:02:11.793807+0000","last_became_active":"2023-09-12T16:02:09.841203+0000","last_became_peered":"2023-09-12T16:02:09.841203+0000","last_unstale":"2023-09-12T16:02:11.793807+0000","last_undegraded":"2023-09-12T16:02:11.793807+0000","last_fullsized":"2023-09-12T16:02:11.793807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:55:46.128996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892443+0000","last_change":"2023-09-12T16:02:04.027953+0000","last_active":"2023-09-12T16:02:10.892443+0000","last_peered":"2023-09-12T16:02:10.892443+0000","last_clean":"2023-09-12T16:02:10.892443+0000","last_became_active":"2023-09-12T16:02:04.027624+0000","last_became_peered":"2023-09-12T16:02:04.027624+0000","last_unstale":"2023-09-12T16:02:10.892443+0000","last_undegraded":"2023-09-12T16:02:10.892443+0000","last_fullsized":"2023-09-12T16:02:10.892443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:33:58.356358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892073+0000","last_change":"2023-09-12T16:02:05.036593+0000","last_active":"2023-09-12T16:02:10.892073+0000","last_peered":"2023-09-12T16:02:10.892073+0000","last_clean":"2023-09-12T16:02:10.892073+0000","last_became_active":"2023-09-12T16:02:05.036204+0000","last_became_peered":"2023-09-12T16:02:05.036204+0000","last_unstale":"2023-09-12T16:02:10.892073+0000","last_undegraded":"2023-09-12T16:02:10.892073+0000","last_fullsized":"2023-09-12T16:02:10.892073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:12:31.068837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793070+0000","last_change":"2023-09-12T16:02:09.292405+0000","last_active":"2023-09-12T16:02:11.793070+0000","last_peered":"2023-09-12T16:02:11.793070+0000","last_clean":"2023-09-12T16:02:11.793070+0000","last_became_active":"2023-09-12T16:02:09.291877+0000","last_became_peered":"2023-09-12T16:02:09.291877+0000","last_unstale":"2023-09-12T16:02:11.793070+0000","last_undegraded":"2023-09-12T16:02:11.793070+0000","last_fullsized":"2023-09-12T16:02:11.793070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:53:04.847310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265427+0000","last_change":"2023-09-12T16:02:04.034738+0000","last_active":"2023-09-12T16:02:10.265427+0000","last_peered":"2023-09-12T16:02:10.265427+0000","last_clean":"2023-09-12T16:02:10.265427+0000","last_became_active":"2023-09-12T16:02:04.034408+0000","last_became_peered":"2023-09-12T16:02:04.034408+0000","last_unstale":"2023-09-12T16:02:10.265427+0000","last_undegraded":"2023-09-12T16:02:10.265427+0000","last_fullsized":"2023-09-12T16:02:10.265427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:27.845770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891985+0000","last_change":"2023-09-12T16:02:05.044936+0000","last_active":"2023-09-12T16:02:10.891985+0000","last_peered":"2023-09-12T16:02:10.891985+0000","last_clean":"2023-09-12T16:02:10.891985+0000","last_became_active":"2023-09-12T16:02:05.044617+0000","last_became_peered":"2023-09-12T16:02:05.044617+0000","last_unstale":"2023-09-12T16:02:10.891985+0000","last_undegraded":"2023-09-12T16:02:10.891985+0000","last_fullsized":"2023-09-12T16:02:10.891985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:41:32.639331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792871+0000","last_change":"2023-09-12T16:02:09.841919+0000","last_active":"2023-09-12T16:02:11.792871+0000","last_peered":"2023-09-12T16:02:11.792871+0000","last_clean":"2023-09-12T16:02:11.792871+0000","last_became_active":"2023-09-12T16:02:09.841410+0000","last_became_peered":"2023-09-12T16:02:09.841410+0000","last_unstale":"2023-09-12T16:02:11.792871+0000","last_undegraded":"2023-09-12T16:02:11.792871+0000","last_fullsized":"2023-09-12T16:02:11.792871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:29:57.511693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265034+0000","last_change":"2023-09-12T16:02:05.033181+0000","last_active":"2023-09-12T16:02:10.265034+0000","last_peered":"2023-09-12T16:02:10.265034+0000","last_clean":"2023-09-12T16:02:10.265034+0000","last_became_active":"2023-09-12T16:02:05.032888+0000","last_became_peered":"2023-09-12T16:02:05.032888+0000","last_unstale":"2023-09-12T16:02:10.265034+0000","last_undegraded":"2023-09-12T16:02:10.265034+0000","last_fullsized":"2023-09-12T16:02:10.265034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:11:06.693972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792084+0000","last_change":"2023-09-12T16:02:04.032892+0000","last_active":"2023-09-12T16:02:11.792084+0000","last_peered":"2023-09-12T16:02:11.792084+0000","last_clean":"2023-09-12T16:02:11.792084+0000","last_became_active":"2023-09-12T16:02:04.032637+0000","last_became_peered":"2023-09-12T16:02:04.032637+0000","last_unstale":"2023-09-12T16:02:11.792084+0000","last_undegraded":"2023-09-12T16:02:11.792084+0000","last_fullsized":"2023-09-12T16:02:11.792084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:22:55.805000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266685+0000","last_change":"2023-09-12T16:02:09.285127+0000","last_active":"2023-09-12T16:02:11.266685+0000","last_peered":"2023-09-12T16:02:11.266685+0000","last_clean":"2023-09-12T16:02:11.266685+0000","last_became_active":"2023-09-12T16:02:09.284790+0000","last_became_peered":"2023-09-12T16:02:09.284790+0000","last_unstale":"2023-09-12T16:02:11.266685+0000","last_undegraded":"2023-09-12T16:02:11.266685+0000","last_fullsized":"2023-09-12T16:02:11.266685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:50:50.578069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.627836+0000","last_change":"2023-09-12T16:02:05.027877+0000","last_active":"2023-09-12T16:02:10.627836+0000","last_peered":"2023-09-12T16:02:10.627836+0000","last_clean":"2023-09-12T16:02:10.627836+0000","last_became_active":"2023-09-12T16:02:05.027582+0000","last_became_peered":"2023-09-12T16:02:05.027582+0000","last_unstale":"2023-09-12T16:02:10.627836+0000","last_undegraded":"2023-09-12T16:02:10.627836+0000","last_fullsized":"2023-09-12T16:02:10.627836+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:47.835790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760444+0000","last_change":"2023-09-12T16:02:04.024800+0000","last_active":"2023-09-12T16:02:10.760444+0000","last_peered":"2023-09-12T16:02:10.760444+0000","last_clean":"2023-09-12T16:02:10.760444+0000","last_became_active":"2023-09-12T16:02:04.024436+0000","last_became_peered":"2023-09-12T16:02:04.024436+0000","last_unstale":"2023-09-12T16:02:10.760444+0000","last_undegraded":"2023-09-12T16:02:10.760444+0000","last_fullsized":"2023-09-12T16:02:10.760444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:57:35.093322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":311,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265106+0000","last_change":"2023-09-12T16:01:04.599706+0000","last_active":"2023-09-12T16:02:10.265106+0000","last_peered":"2023-09-12T16:02:10.265106+0000","last_clean":"2023-09-12T16:02:10.265106+0000","last_became_active":"2023-09-12T16:01:04.599328+0000","last_became_peered":"2023-09-12T16:01:04.599328+0000","last_unstale":"2023-09-12T16:02:10.265106+0000","last_undegraded":"2023-09-12T16:02:10.265106+0000","last_fullsized":"2023-09-12T16:02:10.265106+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_clean_scrub_stamp":"2023-09-12T15:58:15.250648+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:57:56.846488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.794449+0000","last_change":"2023-09-12T16:02:09.841960+0000","last_active":"2023-09-12T16:02:11.794449+0000","last_peered":"2023-09-12T16:02:11.794449+0000","last_clean":"2023-09-12T16:02:11.794449+0000","last_became_active":"2023-09-12T16:02:09.841279+0000","last_became_peered":"2023-09-12T16:02:09.841279+0000","last_unstale":"2023-09-12T16:02:11.794449+0000","last_undegraded":"2023-09-12T16:02:11.794449+0000","last_fullsized":"2023-09-12T16:02:11.794449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:17:19.419350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266459+0000","last_change":"2023-09-12T16:02:05.038666+0000","last_active":"2023-09-12T16:02:10.266459+0000","last_peered":"2023-09-12T16:02:10.266459+0000","last_clean":"2023-09-12T16:02:10.266459+0000","last_became_active":"2023-09-12T16:02:05.038336+0000","last_became_peered":"2023-09-12T16:02:05.038336+0000","last_unstale":"2023-09-12T16:02:10.266459+0000","last_undegraded":"2023-09-12T16:02:10.266459+0000","last_fullsized":"2023-09-12T16:02:10.266459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:20:35.844998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266212+0000","last_change":"2023-09-12T16:02:04.029790+0000","last_active":"2023-09-12T16:02:11.266212+0000","last_peered":"2023-09-12T16:02:11.266212+0000","last_clean":"2023-09-12T16:02:11.266212+0000","last_became_active":"2023-09-12T16:02:04.029440+0000","last_became_peered":"2023-09-12T16:02:04.029440+0000","last_unstale":"2023-09-12T16:02:11.266212+0000","last_undegraded":"2023-09-12T16:02:11.266212+0000","last_fullsized":"2023-09-12T16:02:11.266212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:50:33.690628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793848+0000","last_change":"2023-09-12T16:02:09.289757+0000","last_active":"2023-09-12T16:02:11.793848+0000","last_peered":"2023-09-12T16:02:11.793848+0000","last_clean":"2023-09-12T16:02:11.793848+0000","last_became_active":"2023-09-12T16:02:09.289348+0000","last_became_peered":"2023-09-12T16:02:09.289348+0000","last_unstale":"2023-09-12T16:02:11.793848+0000","last_undegraded":"2023-09-12T16:02:11.793848+0000","last_fullsized":"2023-09-12T16:02:11.793848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:15:14.695521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792913+0000","last_change":"2023-09-12T16:02:04.033213+0000","last_active":"2023-09-12T16:02:11.792913+0000","last_peered":"2023-09-12T16:02:11.792913+0000","last_clean":"2023-09-12T16:02:11.792913+0000","last_became_active":"2023-09-12T16:02:04.032978+0000","last_became_peered":"2023-09-12T16:02:04.032978+0000","last_unstale":"2023-09-12T16:02:11.792913+0000","last_undegraded":"2023-09-12T16:02:11.792913+0000","last_fullsized":"2023-09-12T16:02:11.792913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:25:33.718954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266899+0000","last_change":"2023-09-12T16:02:05.043252+0000","last_active":"2023-09-12T16:02:10.266899+0000","last_peered":"2023-09-12T16:02:10.266899+0000","last_clean":"2023-09-12T16:02:10.266899+0000","last_became_active":"2023-09-12T16:02:05.042975+0000","last_became_peered":"2023-09-12T16:02:05.042975+0000","last_unstale":"2023-09-12T16:02:10.266899+0000","last_undegraded":"2023-09-12T16:02:10.266899+0000","last_fullsized":"2023-09-12T16:02:10.266899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:46:31.549547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.265460+0000","last_change":"2023-09-12T16:02:09.841728+0000","last_active":"2023-09-12T16:02:11.265460+0000","last_peered":"2023-09-12T16:02:11.265460+0000","last_clean":"2023-09-12T16:02:11.265460+0000","last_became_active":"2023-09-12T16:02:09.841351+0000","last_became_peered":"2023-09-12T16:02:09.841351+0000","last_unstale":"2023-09-12T16:02:11.265460+0000","last_undegraded":"2023-09-12T16:02:11.265460+0000","last_fullsized":"2023-09-12T16:02:11.265460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:00:50.637272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892838+0000","last_change":"2023-09-12T16:02:04.025624+0000","last_active":"2023-09-12T16:02:10.892838+0000","last_peered":"2023-09-12T16:02:10.892838+0000","last_clean":"2023-09-12T16:02:10.892838+0000","last_became_active":"2023-09-12T16:02:04.025330+0000","last_became_peered":"2023-09-12T16:02:04.025330+0000","last_unstale":"2023-09-12T16:02:10.892838+0000","last_undegraded":"2023-09-12T16:02:10.892838+0000","last_fullsized":"2023-09-12T16:02:10.892838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:00:57.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.627988+0000","last_change":"2023-09-12T16:02:05.028414+0000","last_active":"2023-09-12T16:02:10.627988+0000","last_peered":"2023-09-12T16:02:10.627988+0000","last_clean":"2023-09-12T16:02:10.627988+0000","last_became_active":"2023-09-12T16:02:05.028081+0000","last_became_peered":"2023-09-12T16:02:05.028081+0000","last_unstale":"2023-09-12T16:02:10.627988+0000","last_undegraded":"2023-09-12T16:02:10.627988+0000","last_fullsized":"2023-09-12T16:02:10.627988+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:57:58.245105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.266169+0000","last_change":"2023-09-12T16:02:09.283222+0000","last_active":"2023-09-12T16:02:11.266169+0000","last_peered":"2023-09-12T16:02:11.266169+0000","last_clean":"2023-09-12T16:02:11.266169+0000","last_became_active":"2023-09-12T16:02:09.282814+0000","last_became_peered":"2023-09-12T16:02:09.282814+0000","last_unstale":"2023-09-12T16:02:11.266169+0000","last_undegraded":"2023-09-12T16:02:11.266169+0000","last_fullsized":"2023-09-12T16:02:11.266169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:18.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"80'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.760915+0000","last_change":"2023-09-12T16:02:04.026694+0000","last_active":"2023-09-12T16:02:10.760915+0000","last_peered":"2023-09-12T16:02:10.760915+0000","last_clean":"2023-09-12T16:02:10.760915+0000","last_became_active":"2023-09-12T16:02:04.026368+0000","last_became_peered":"2023-09-12T16:02:04.026368+0000","last_unstale":"2023-09-12T16:02:10.760915+0000","last_undegraded":"2023-09-12T16:02:10.760915+0000","last_fullsized":"2023-09-12T16:02:10.760915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:14:10.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891618+0000","last_change":"2023-09-12T16:02:05.039037+0000","last_active":"2023-09-12T16:02:10.891618+0000","last_peered":"2023-09-12T16:02:10.891618+0000","last_clean":"2023-09-12T16:02:10.891618+0000","last_became_active":"2023-09-12T16:02:05.038708+0000","last_became_peered":"2023-09-12T16:02:05.038708+0000","last_unstale":"2023-09-12T16:02:10.891618+0000","last_undegraded":"2023-09-12T16:02:10.891618+0000","last_fullsized":"2023-09-12T16:02:10.891618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:13:09.805646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793845+0000","last_change":"2023-09-12T16:02:09.841757+0000","last_active":"2023-09-12T16:02:11.793845+0000","last_peered":"2023-09-12T16:02:11.793845+0000","last_clean":"2023-09-12T16:02:11.793845+0000","last_became_active":"2023-09-12T16:02:09.841173+0000","last_became_peered":"2023-09-12T16:02:09.841173+0000","last_unstale":"2023-09-12T16:02:11.793845+0000","last_undegraded":"2023-09-12T16:02:11.793845+0000","last_fullsized":"2023-09-12T16:02:11.793845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:57:06.626828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573197+0000","last_change":"2023-09-12T16:02:04.031009+0000","last_active":"2023-09-12T16:02:10.573197+0000","last_peered":"2023-09-12T16:02:10.573197+0000","last_clean":"2023-09-12T16:02:10.573197+0000","last_became_active":"2023-09-12T16:02:04.030714+0000","last_became_peered":"2023-09-12T16:02:04.030714+0000","last_unstale":"2023-09-12T16:02:10.573197+0000","last_undegraded":"2023-09-12T16:02:10.573197+0000","last_fullsized":"2023-09-12T16:02:10.573197+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:12.578686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.305382+0000","last_change":"2023-09-12T16:02:05.030778+0000","last_active":"2023-09-12T16:02:10.305382+0000","last_peered":"2023-09-12T16:02:10.305382+0000","last_clean":"2023-09-12T16:02:10.305382+0000","last_became_active":"2023-09-12T16:02:05.030454+0000","last_became_peered":"2023-09-12T16:02:05.030454+0000","last_unstale":"2023-09-12T16:02:10.305382+0000","last_undegraded":"2023-09-12T16:02:10.305382+0000","last_fullsized":"2023-09-12T16:02:10.305382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:52.951246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793015+0000","last_change":"2023-09-12T16:02:09.289668+0000","last_active":"2023-09-12T16:02:11.793015+0000","last_peered":"2023-09-12T16:02:11.793015+0000","last_clean":"2023-09-12T16:02:11.793015+0000","last_became_active":"2023-09-12T16:02:09.289301+0000","last_became_peered":"2023-09-12T16:02:09.289301+0000","last_unstale":"2023-09-12T16:02:11.793015+0000","last_undegraded":"2023-09-12T16:02:11.793015+0000","last_fullsized":"2023-09-12T16:02:11.793015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:20:04.528146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:23.643840+0000","last_change":"2023-09-12T16:02:04.036175+0000","last_active":"2023-09-12T16:02:23.643840+0000","last_peered":"2023-09-12T16:02:23.643840+0000","last_clean":"2023-09-12T16:02:23.643840+0000","last_became_active":"2023-09-12T16:02:04.035879+0000","last_became_peered":"2023-09-12T16:02:04.035879+0000","last_unstale":"2023-09-12T16:02:23.643840+0000","last_undegraded":"2023-09-12T16:02:23.643840+0000","last_fullsized":"2023-09-12T16:02:23.643840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:29:20.798716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891598+0000","last_change":"2023-09-12T16:02:05.034195+0000","last_active":"2023-09-12T16:02:10.891598+0000","last_peered":"2023-09-12T16:02:10.891598+0000","last_clean":"2023-09-12T16:02:10.891598+0000","last_became_active":"2023-09-12T16:02:05.033534+0000","last_became_peered":"2023-09-12T16:02:05.033534+0000","last_unstale":"2023-09-12T16:02:10.891598+0000","last_undegraded":"2023-09-12T16:02:10.891598+0000","last_fullsized":"2023-09-12T16:02:10.891598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:34:04.180218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.793440+0000","last_change":"2023-09-12T16:02:09.287543+0000","last_active":"2023-09-12T16:02:11.793440+0000","last_peered":"2023-09-12T16:02:11.793440+0000","last_clean":"2023-09-12T16:02:11.793440+0000","last_became_active":"2023-09-12T16:02:09.287055+0000","last_became_peered":"2023-09-12T16:02:09.287055+0000","last_unstale":"2023-09-12T16:02:11.793440+0000","last_undegraded":"2023-09-12T16:02:11.793440+0000","last_fullsized":"2023-09-12T16:02:11.793440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:58.214381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892036+0000","last_change":"2023-09-12T16:02:04.030206+0000","last_active":"2023-09-12T16:02:10.892036+0000","last_peered":"2023-09-12T16:02:10.892036+0000","last_clean":"2023-09-12T16:02:10.892036+0000","last_became_active":"2023-09-12T16:02:04.029765+0000","last_became_peered":"2023-09-12T16:02:04.029765+0000","last_unstale":"2023-09-12T16:02:10.892036+0000","last_undegraded":"2023-09-12T16:02:10.892036+0000","last_fullsized":"2023-09-12T16:02:10.892036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:56:07.551681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.628253+0000","last_change":"2023-09-12T16:02:05.028300+0000","last_active":"2023-09-12T16:02:10.628253+0000","last_peered":"2023-09-12T16:02:10.628253+0000","last_clean":"2023-09-12T16:02:10.628253+0000","last_became_active":"2023-09-12T16:02:05.027970+0000","last_became_peered":"2023-09-12T16:02:05.027970+0000","last_unstale":"2023-09-12T16:02:10.628253+0000","last_undegraded":"2023-09-12T16:02:10.628253+0000","last_fullsized":"2023-09-12T16:02:10.628253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:39:44.571593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792625+0000","last_change":"2023-09-12T16:02:09.841661+0000","last_active":"2023-09-12T16:02:11.792625+0000","last_peered":"2023-09-12T16:02:11.792625+0000","last_clean":"2023-09-12T16:02:11.792625+0000","last_became_active":"2023-09-12T16:02:09.841039+0000","last_became_peered":"2023-09-12T16:02:09.841039+0000","last_unstale":"2023-09-12T16:02:11.792625+0000","last_undegraded":"2023-09-12T16:02:11.792625+0000","last_fullsized":"2023-09-12T16:02:11.792625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:34.529098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:23.642882+0000","last_change":"2023-09-12T16:02:04.031927+0000","last_active":"2023-09-12T16:02:23.642882+0000","last_peered":"2023-09-12T16:02:23.642882+0000","last_clean":"2023-09-12T16:02:23.642882+0000","last_became_active":"2023-09-12T16:02:04.031614+0000","last_became_peered":"2023-09-12T16:02:04.031614+0000","last_unstale":"2023-09-12T16:02:23.642882+0000","last_undegraded":"2023-09-12T16:02:23.642882+0000","last_fullsized":"2023-09-12T16:02:23.642882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:00:23.423933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265435+0000","last_change":"2023-09-12T16:02:05.032647+0000","last_active":"2023-09-12T16:02:10.265435+0000","last_peered":"2023-09-12T16:02:10.265435+0000","last_clean":"2023-09-12T16:02:10.265435+0000","last_became_active":"2023-09-12T16:02:05.031701+0000","last_became_peered":"2023-09-12T16:02:05.031701+0000","last_unstale":"2023-09-12T16:02:10.265435+0000","last_undegraded":"2023-09-12T16:02:10.265435+0000","last_fullsized":"2023-09-12T16:02:10.265435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:02:19.596913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792820+0000","last_change":"2023-09-12T16:02:09.291043+0000","last_active":"2023-09-12T16:02:11.792820+0000","last_peered":"2023-09-12T16:02:11.792820+0000","last_clean":"2023-09-12T16:02:11.792820+0000","last_became_active":"2023-09-12T16:02:09.290666+0000","last_became_peered":"2023-09-12T16:02:09.290666+0000","last_unstale":"2023-09-12T16:02:11.792820+0000","last_undegraded":"2023-09-12T16:02:11.792820+0000","last_fullsized":"2023-09-12T16:02:11.792820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:10:47.354479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:24.651169+0000","last_change":"2023-09-12T16:02:04.030501+0000","last_active":"2023-09-12T16:02:24.651169+0000","last_peered":"2023-09-12T16:02:24.651169+0000","last_clean":"2023-09-12T16:02:24.651169+0000","last_became_active":"2023-09-12T16:02:04.030196+0000","last_became_peered":"2023-09-12T16:02:04.030196+0000","last_unstale":"2023-09-12T16:02:24.651169+0000","last_undegraded":"2023-09-12T16:02:24.651169+0000","last_fullsized":"2023-09-12T16:02:24.651169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:58:34.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265985+0000","last_change":"2023-09-12T16:02:05.030466+0000","last_active":"2023-09-12T16:02:10.265985+0000","last_peered":"2023-09-12T16:02:10.265985+0000","last_clean":"2023-09-12T16:02:10.265985+0000","last_became_active":"2023-09-12T16:02:05.029988+0000","last_became_peered":"2023-09-12T16:02:05.029988+0000","last_unstale":"2023-09-12T16:02:10.265985+0000","last_undegraded":"2023-09-12T16:02:10.265985+0000","last_fullsized":"2023-09-12T16:02:10.265985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:08:56.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:12.221376+0000","last_change":"2023-09-12T16:02:09.294681+0000","last_active":"2023-09-12T16:02:12.221376+0000","last_peered":"2023-09-12T16:02:12.221376+0000","last_clean":"2023-09-12T16:02:12.221376+0000","last_became_active":"2023-09-12T16:02:09.294294+0000","last_became_peered":"2023-09-12T16:02:09.294294+0000","last_unstale":"2023-09-12T16:02:12.221376+0000","last_undegraded":"2023-09-12T16:02:12.221376+0000","last_fullsized":"2023-09-12T16:02:12.221376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:14:47.286285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265590+0000","last_change":"2023-09-12T16:02:04.027380+0000","last_active":"2023-09-12T16:02:10.265590+0000","last_peered":"2023-09-12T16:02:10.265590+0000","last_clean":"2023-09-12T16:02:10.265590+0000","last_became_active":"2023-09-12T16:02:04.027098+0000","last_became_peered":"2023-09-12T16:02:04.027098+0000","last_unstale":"2023-09-12T16:02:10.265590+0000","last_undegraded":"2023-09-12T16:02:10.265590+0000","last_fullsized":"2023-09-12T16:02:10.265590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:34:12.260747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265862+0000","last_change":"2023-09-12T16:02:05.029249+0000","last_active":"2023-09-12T16:02:10.265862+0000","last_peered":"2023-09-12T16:02:10.265862+0000","last_clean":"2023-09-12T16:02:10.265862+0000","last_became_active":"2023-09-12T16:02:05.028955+0000","last_became_peered":"2023-09-12T16:02:05.028955+0000","last_unstale":"2023-09-12T16:02:10.265862+0000","last_undegraded":"2023-09-12T16:02:10.265862+0000","last_fullsized":"2023-09-12T16:02:10.265862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:02:10.382712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-12T16:02:11.792450+0000","last_change":"2023-09-12T16:02:09.288724+0000","last_active":"2023-09-12T16:02:11.792450+0000","last_peered":"2023-09-12T16:02:11.792450+0000","last_clean":"2023-09-12T16:02:11.792450+0000","last_became_active":"2023-09-12T16:02:09.288206+0000","last_became_peered":"2023-09-12T16:02:09.288206+0000","last_unstale":"2023-09-12T16:02:11.792450+0000","last_undegraded":"2023-09-12T16:02:11.792450+0000","last_fullsized":"2023-09-12T16:02:11.792450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:45:10.154195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.267028+0000","last_change":"2023-09-12T16:02:04.031219+0000","last_active":"2023-09-12T16:02:10.267028+0000","last_peered":"2023-09-12T16:02:10.267028+0000","last_clean":"2023-09-12T16:02:10.267028+0000","last_became_active":"2023-09-12T16:02:04.030867+0000","last_became_peered":"2023-09-12T16:02:04.030867+0000","last_unstale":"2023-09-12T16:02:10.267028+0000","last_undegraded":"2023-09-12T16:02:10.267028+0000","last_fullsized":"2023-09-12T16:02:10.267028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:16.885094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.265856+0000","last_change":"2023-09-12T16:02:05.028004+0000","last_active":"2023-09-12T16:02:10.265856+0000","last_peered":"2023-09-12T16:02:10.265856+0000","last_clean":"2023-09-12T16:02:10.265856+0000","last_became_active":"2023-09-12T16:02:05.027668+0000","last_became_peered":"2023-09-12T16:02:05.027668+0000","last_unstale":"2023-09-12T16:02:10.265856+0000","last_undegraded":"2023-09-12T16:02:10.265856+0000","last_fullsized":"2023-09-12T16:02:10.265856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:41:23.019243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891680+0000","last_change":"2023-09-12T16:02:05.034380+0000","last_active":"2023-09-12T16:02:10.891680+0000","last_peered":"2023-09-12T16:02:10.891680+0000","last_clean":"2023-09-12T16:02:10.891680+0000","last_became_active":"2023-09-12T16:02:05.034022+0000","last_became_peered":"2023-09-12T16:02:05.034022+0000","last_unstale":"2023-09-12T16:02:10.891680+0000","last_undegraded":"2023-09-12T16:02:10.891680+0000","last_fullsized":"2023-09-12T16:02:10.891680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:55:56.404742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892801+0000","last_change":"2023-09-12T16:02:05.031723+0000","last_active":"2023-09-12T16:02:10.892801+0000","last_peered":"2023-09-12T16:02:10.892801+0000","last_clean":"2023-09-12T16:02:10.892801+0000","last_became_active":"2023-09-12T16:02:05.030954+0000","last_became_peered":"2023-09-12T16:02:05.030954+0000","last_unstale":"2023-09-12T16:02:10.892801+0000","last_undegraded":"2023-09-12T16:02:10.892801+0000","last_fullsized":"2023-09-12T16:02:10.892801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.319954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891934+0000","last_change":"2023-09-12T16:02:05.034851+0000","last_active":"2023-09-12T16:02:10.891934+0000","last_peered":"2023-09-12T16:02:10.891934+0000","last_clean":"2023-09-12T16:02:10.891934+0000","last_became_active":"2023-09-12T16:02:05.030733+0000","last_became_peered":"2023-09-12T16:02:05.030733+0000","last_unstale":"2023-09-12T16:02:10.891934+0000","last_undegraded":"2023-09-12T16:02:10.891934+0000","last_fullsized":"2023-09-12T16:02:10.891934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:33.605216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.891582+0000","last_change":"2023-09-12T16:02:05.037331+0000","last_active":"2023-09-12T16:02:10.891582+0000","last_peered":"2023-09-12T16:02:10.891582+0000","last_clean":"2023-09-12T16:02:10.891582+0000","last_became_active":"2023-09-12T16:02:05.037015+0000","last_became_peered":"2023-09-12T16:02:05.037015+0000","last_unstale":"2023-09-12T16:02:10.891582+0000","last_undegraded":"2023-09-12T16:02:10.891582+0000","last_fullsized":"2023-09-12T16:02:10.891582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:29:14.035891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266985+0000","last_change":"2023-09-12T16:02:05.039803+0000","last_active":"2023-09-12T16:02:10.266985+0000","last_peered":"2023-09-12T16:02:10.266985+0000","last_clean":"2023-09-12T16:02:10.266985+0000","last_became_active":"2023-09-12T16:02:05.039390+0000","last_became_peered":"2023-09-12T16:02:05.039390+0000","last_unstale":"2023-09-12T16:02:10.266985+0000","last_undegraded":"2023-09-12T16:02:10.266985+0000","last_fullsized":"2023-09-12T16:02:10.266985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:22.422090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892400+0000","last_change":"2023-09-12T16:02:05.033153+0000","last_active":"2023-09-12T16:02:10.892400+0000","last_peered":"2023-09-12T16:02:10.892400+0000","last_clean":"2023-09-12T16:02:10.892400+0000","last_became_active":"2023-09-12T16:02:05.032800+0000","last_became_peered":"2023-09-12T16:02:05.032800+0000","last_unstale":"2023-09-12T16:02:10.892400+0000","last_undegraded":"2023-09-12T16:02:10.892400+0000","last_fullsized":"2023-09-12T16:02:10.892400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:31:43.090031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892295+0000","last_change":"2023-09-12T16:02:05.031519+0000","last_active":"2023-09-12T16:02:10.892295+0000","last_peered":"2023-09-12T16:02:10.892295+0000","last_clean":"2023-09-12T16:02:10.892295+0000","last_became_active":"2023-09-12T16:02:05.030227+0000","last_became_peered":"2023-09-12T16:02:05.030227+0000","last_unstale":"2023-09-12T16:02:10.892295+0000","last_undegraded":"2023-09-12T16:02:10.892295+0000","last_fullsized":"2023-09-12T16:02:10.892295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:00.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.628098+0000","last_change":"2023-09-12T16:02:05.031763+0000","last_active":"2023-09-12T16:02:10.628098+0000","last_peered":"2023-09-12T16:02:10.628098+0000","last_clean":"2023-09-12T16:02:10.628098+0000","last_became_active":"2023-09-12T16:02:05.031441+0000","last_became_peered":"2023-09-12T16:02:05.031441+0000","last_unstale":"2023-09-12T16:02:10.628098+0000","last_undegraded":"2023-09-12T16:02:10.628098+0000","last_fullsized":"2023-09-12T16:02:10.628098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:55.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.264912+0000","last_change":"2023-09-12T16:02:05.033786+0000","last_active":"2023-09-12T16:02:10.264912+0000","last_peered":"2023-09-12T16:02:10.264912+0000","last_clean":"2023-09-12T16:02:10.264912+0000","last_became_active":"2023-09-12T16:02:05.033497+0000","last_became_peered":"2023-09-12T16:02:05.033497+0000","last_unstale":"2023-09-12T16:02:10.264912+0000","last_undegraded":"2023-09-12T16:02:10.264912+0000","last_fullsized":"2023-09-12T16:02:10.264912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:01:58.912488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.627896+0000","last_change":"2023-09-12T16:02:05.031144+0000","last_active":"2023-09-12T16:02:10.627896+0000","last_peered":"2023-09-12T16:02:10.627896+0000","last_clean":"2023-09-12T16:02:10.627896+0000","last_became_active":"2023-09-12T16:02:05.030827+0000","last_became_peered":"2023-09-12T16:02:05.030827+0000","last_unstale":"2023-09-12T16:02:10.627896+0000","last_undegraded":"2023-09-12T16:02:10.627896+0000","last_fullsized":"2023-09-12T16:02:10.627896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:40:48.014344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.573443+0000","last_change":"2023-09-12T16:02:05.021273+0000","last_active":"2023-09-12T16:02:10.573443+0000","last_peered":"2023-09-12T16:02:10.573443+0000","last_clean":"2023-09-12T16:02:10.573443+0000","last_became_active":"2023-09-12T16:02:05.020953+0000","last_became_peered":"2023-09-12T16:02:05.020953+0000","last_unstale":"2023-09-12T16:02:10.573443+0000","last_undegraded":"2023-09-12T16:02:10.573443+0000","last_fullsized":"2023-09-12T16:02:10.573443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:33:25.462412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.266679+0000","last_change":"2023-09-12T16:02:05.039019+0000","last_active":"2023-09-12T16:02:10.266679+0000","last_peered":"2023-09-12T16:02:10.266679+0000","last_clean":"2023-09-12T16:02:10.266679+0000","last_became_active":"2023-09-12T16:02:05.038621+0000","last_became_peered":"2023-09-12T16:02:05.038621+0000","last_unstale":"2023-09-12T16:02:10.266679+0000","last_undegraded":"2023-09-12T16:02:10.266679+0000","last_fullsized":"2023-09-12T16:02:10.266679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:08:08.934375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-12T16:02:10.892087+0000","last_change":"2023-09-12T16:02:05.039737+0000","last_active":"2023-09-12T16:02:10.892087+0000","last_peered":"2023-09-12T16:02:10.892087+0000","last_clean":"2023-09-12T16:02:10.892087+0000","last_became_active":"2023-09-12T16:02:05.039304+0000","last_became_peered":"2023-09-12T16:02:05.039304+0000","last_unstale":"2023-09-12T16:02:10.892087+0000","last_undegraded":"2023-09-12T16:02:10.892087+0000","last_fullsized":"2023-09-12T16:02:10.892087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:50:13.141915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244717,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5536,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5,"num_read_kb":7,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":17556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514511,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":465670,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776146,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1061458,"data_compressed":50331,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.88100000000000001}]},{"osd":3,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66000000000000003}]},{"osd":4,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84199999999999997}]},{"osd":7,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56899999999999995}]},{"osd":8,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":1.008},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.89500000000000002}]},{"osd":9,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.86699999999999999}]},{"osd":11,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.026}]}]},{"osd":9,"up_from":63,"seq":270582939671,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":466252,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.41999999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.438}]},{"osd":3,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.44900000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70699999999999996}]},{"osd":5,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61799999999999999}]},{"osd":8,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45500000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":8,"up_from":56,"seq":240518168603,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":465965,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.51700000000000002}]},{"osd":1,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.47599999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64100000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82299999999999995}]},{"osd":5,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.622}]},{"osd":6,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.755}]},{"osd":9,"last update":"Tue Sep 12 16:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.84299999999999997}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.88100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397536,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":1056911,"data_compressed":50331,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.66200000000000003}]},{"osd":2,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.64400000000000002}]},{"osd":3,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89100000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.77000000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.58499999999999996}]},{"osd":5,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.621,"15min":0.60399999999999998},"min":{"1min":0.46800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.84699999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.90400000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.82899999999999996}]},{"osd":10,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":6,"up_from":44,"seq":188978561060,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":465670,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.29999999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.376}]},{"osd":1,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92700000000000005,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55000000000000004}]},{"osd":2,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.45900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81899999999999995,"5min":1.129,"15min":1.129},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69999999999999996,"5min":1.083,"15min":1.083},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.504}]},{"osd":4,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91400000000000003,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.48799999999999999}]},{"osd":5,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.51100000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84099999999999997,"5min":1.022,"15min":1.022},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.48999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.623}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71199999999999997}]},{"osd":9,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.59099999999999997}]},{"osd":10,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.72399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":469239,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.50900000000000001,"15min":0.49199999999999999},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.51400000000000001},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56299999999999994,"15min":0.52600000000000002},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.93000000000000005}]},{"osd":3,"last update":"Tue Sep 12 16:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.51000000000000001},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.65600000000000003}]},{"osd":4,"last update":"Tue Sep 12 16:01:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.46700000000000003,"15min":0.44500000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.96399999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.876}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.38100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.91400000000000003}]},{"osd":8,"last update":"Tue Sep 12 16:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":1.071},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.94899999999999995}]},{"osd":9,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":1.0129999999999999}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.0580000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":465760,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60599999999999998},"min":{"1min":0.39900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79800000000000004,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60399999999999998,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79100000000000004,"5min":2.113,"15min":2.113},"last":0.74299999999999999}]},{"osd":2,"last update":"Tue Sep 12 16:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.52900000000000003,"15min":0.50900000000000001},"min":{"1min":0.41799999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.38200000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.72599999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.39000000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.443,"15min":0.433},"min":{"1min":0.309,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.627}]},{"osd":5,"last update":"Tue Sep 12 16:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.32900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.433},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.91100000000000003,"5min":1.004,"15min":1.004},"last":0.41499999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56799999999999995}]},{"osd":8,"last update":"Tue Sep 12 16:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68899999999999995}]},{"osd":9,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.495}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.71199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":469920,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.373,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63100000000000001,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66300000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.51100000000000001,"15min":0.48899999999999999},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.747,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.438},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.51800000000000002,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.313,"15min":0.313},"max":{"1min":0.85599999999999998,"5min":0.877,"15min":0.877},"last":0.54500000000000004}]},{"osd":3,"last update":"Tue Sep 12 16:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54100000000000004,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57099999999999995}]},{"osd":4,"last update":"Tue Sep 12 16:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76200000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76300000000000001}]},{"osd":6,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79500000000000004}]},{"osd":7,"last update":"Tue Sep 12 16:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83899999999999997}]},{"osd":8,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.82499999999999996}]},{"osd":9,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.89900000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":465864,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.33800000000000002},"min":{"1min":0.252,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.26900000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38800000000000001}]},{"osd":1,"last update":"Tue Sep 12 16:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.91800000000000004,"5min":3.552,"15min":3.552},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.45900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69999999999999996,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.64100000000000001}]},{"osd":2,"last update":"Tue Sep 12 16:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84899999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53900000000000003,"15min":0.51700000000000002},"min":{"1min":0.45600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84499999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Sep 12 16:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.46600000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.90100000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.44},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41599999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.56100000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.5,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.749}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66100000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.46500000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953516,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":465419,"data_compressed":14669,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.46999999999999997},"min":{"1min":0.38500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.45800000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88800000000000001,"5min":1.458,"15min":1.458},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.30299999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.39300000000000002,"15min":0.36599999999999999},"min":{"1min":0.27900000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76700000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.54800000000000004,"15min":0.52500000000000002},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63300000000000001}]},{"osd":3,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.54300000000000004,"15min":0.496},"min":{"1min":0.45900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":1.383,"15min":1.383},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51000000000000001,"15min":0.48799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54900000000000004}]},{"osd":5,"last update":"Tue Sep 12 16:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86399999999999999,"5min":1.54,"15min":1.54},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.437,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93600000000000005,"5min":1.502,"15min":1.502},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69299999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65300000000000002,"15min":0.64000000000000001},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.639,"15min":1.639},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.749}]},{"osd":8,"last update":"Tue Sep 12 16:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.879}]},{"osd":9,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78800000000000003}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.92800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789991,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1055889,"data_compressed":50331,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.50800000000000001,"15min":0.51100000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.498}]},{"osd":1,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67500000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55200000000000005}]},{"osd":2,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.374,"15min":0.36699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.39400000000000002,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.245,"15min":0.245},"max":{"1min":0.58699999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.66000000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54400000000000004,"15min":0.55200000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71999999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.73599999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77100000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.83799999999999997}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.72599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.626,"15min":0.626},"min":{"1min":0.50600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83299999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73999999999999999}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.72599999999999998}]},{"osd":9,"last update":"Tue Sep 12 16:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78900000000000003}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T16:02:28.708 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-12T16:02:28.708 INFO:tasks.cephadm:Setting up client nodes... 2023-09-12T16:02:28.709 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-09-12T16:02:28.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:28 smithi130 ceph-mon[98670]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:28.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:28 smithi130 ceph-mon[98670]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-09-12T16:02:28.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:28 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3978389283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:28.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:28 smithi159 ceph-mon[102522]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:28.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:28 smithi159 ceph-mon[102522]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-09-12T16:02:28.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:28 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3978389283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:28.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:28 smithi203 ceph-mon[103431]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:02:28.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:28 smithi203 ceph-mon[103431]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-09-12T16:02:28.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:28 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3978389283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:29.294 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:29.867 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:29 smithi130 ceph-mon[98670]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:02:29.868 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:29 smithi130 ceph-mon[98670]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:02:29.868 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:29 smithi130 ceph-mon[98670]: pgmap v275: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2023-09-12T16:02:29.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:29 smithi159 ceph-mon[102522]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:02:29.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:29 smithi159 ceph-mon[102522]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:02:29.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:29 smithi159 ceph-mon[102522]: pgmap v275: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2023-09-12T16:02:29.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:29 smithi203 ceph-mon[103431]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:02:29.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:29 smithi203 ceph-mon[103431]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:02:29.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:29 smithi203 ceph-mon[103431]: pgmap v275: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2023-09-12T16:02:30.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 -- 172.21.15.130:0/3848064763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c1018c0 msgr2=0x7f775c101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:30.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 --2- 172.21.15.130:0/3848064763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c1018c0 0x7f775c101d20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f77440099c0 tx=0x7f7744009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 -- 172.21.15.130:0/3848064763 shutdown_connections 2023-09-12T16:02:30.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 --2- 172.21.15.130:0/3848064763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f775c102260 0x7f775c108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 --2- 172.21.15.130:0/3848064763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c1006c0 0x7f775c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 --2- 172.21.15.130:0/3848064763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c1018c0 0x7f775c101d20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 -- 172.21.15.130:0/3848064763 >> 172.21.15.130:0/3848064763 conn(0x7f775c0fbc80 msgr2=0x7f775c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:30.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 -- 172.21.15.130:0/3848064763 shutdown_connections 2023-09-12T16:02:30.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 -- 172.21.15.130:0/3848064763 wait complete. 2023-09-12T16:02:30.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.605+0000 7f7761902700 1 Processor -- start 2023-09-12T16:02:30.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f7761902700 1 -- start start 2023-09-12T16:02:30.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f7761902700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c1006c0 0x7f775c196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:30.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f7761902700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f775c1018c0 0x7f775c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:30.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f7761902700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c102260 0x7f775c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:30.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f7761902700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c19dfd0 con 0x7f775c102260 2023-09-12T16:02:30.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f7761902700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c19e110 con 0x7f775c1018c0 2023-09-12T16:02:30.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f7761902700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c19e2a0 con 0x7f775c1006c0 2023-09-12T16:02:30.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c102260 0x7f775c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:30.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c102260 0x7f775c19da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53858/0 (socket says 172.21.15.130:53858) 2023-09-12T16:02:30.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775b7fe700 1 -- 172.21.15.130:0/1685198979 learned_addr learned my addr 172.21.15.130:0/1685198979 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:30.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775affd700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c1006c0 0x7f775c196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:30.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775a7fc700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f775c1018c0 0x7f775c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:30.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775b7fe700 1 -- 172.21.15.130:0/1685198979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c1006c0 msgr2=0x7f775c196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:30.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775b7fe700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c1006c0 0x7f775c196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775b7fe700 1 -- 172.21.15.130:0/1685198979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f775c1018c0 msgr2=0x7f775c196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:30.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775b7fe700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f775c1018c0 0x7f775c196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775b7fe700 1 -- 172.21.15.130:0/1685198979 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77440095f0 con 0x7f775c102260 2023-09-12T16:02:30.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775a7fc700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f775c1018c0 0x7f775c196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:30.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.606+0000 7f775b7fe700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c102260 0x7f775c19da00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f774c00b580 tx=0x7f774c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:30.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.607+0000 7f7760900700 1 -- 172.21.15.130:0/1685198979 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c011830 con 0x7f775c102260 2023-09-12T16:02:30.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.607+0000 7f7760900700 1 -- 172.21.15.130:0/1685198979 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774c011e50 con 0x7f775c102260 2023-09-12T16:02:30.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.607+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c19e580 con 0x7f775c102260 2023-09-12T16:02:30.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.607+0000 7f7760900700 1 -- 172.21.15.130:0/1685198979 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00f2e0 con 0x7f775c102260 2023-09-12T16:02:30.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.607+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c19eab0 con 0x7f775c102260 2023-09-12T16:02:30.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.608+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c107600 con 0x7f775c102260 2023-09-12T16:02:30.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.608+0000 7f7760900700 1 -- 172.21.15.130:0/1685198979 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f774c0105a0 con 0x7f775c102260 2023-09-12T16:02:30.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.609+0000 7f7760900700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77480a06d0 0x7f77480a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:30.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.609+0000 7f775affd700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77480a06d0 0x7f77480a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:30.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.609+0000 7f7760900700 1 -- 172.21.15.130:0/1685198979 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f774c0cb880 con 0x7f775c102260 2023-09-12T16:02:30.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.609+0000 7f775affd700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77480a06d0 0x7f77480a2b90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f775c101720 tx=0x7f775400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:30.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.613+0000 7f7760900700 1 -- 172.21.15.130:0/1685198979 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f774c095170 con 0x7f775c102260 2023-09-12T16:02:30.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.886+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f775c10c1c0 con 0x7f775c102260 2023-09-12T16:02:30.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.888+0000 7f7760900700 1 -- 172.21.15.130:0/1685198979 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f774c0150f0 con 0x7f775c102260 2023-09-12T16:02:30.888 INFO:teuthology.orchestra.run.smithi130.stdout:[client.0] 2023-09-12T16:02:30.889 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQCWiwBlYULpNBAA+rOZ/Lcf/CAgpeg3hjmL1A== 2023-09-12T16:02:30.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77480a06d0 msgr2=0x7f77480a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:30.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77480a06d0 0x7f77480a2b90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f775c101720 tx=0x7f775400d040 comp rx=0 tx=0).stop 2023-09-12T16:02:30.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c102260 msgr2=0x7f775c19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:30.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c102260 0x7f775c19da00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f774c00b580 tx=0x7f774c00b890 comp rx=0 tx=0).stop 2023-09-12T16:02:30.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 shutdown_connections 2023-09-12T16:02:30.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f77480a06d0 0x7f77480a2b90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f775c102260 0x7f775c19da00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c1006c0 0x7f775c196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 --2- 172.21.15.130:0/1685198979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f775c1018c0 0x7f775c196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:30.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 >> 172.21.15.130:0/1685198979 conn(0x7f775c0fbc80 msgr2=0x7f775c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:30.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 shutdown_connections 2023-09-12T16:02:30.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:30.890+0000 7f7761902700 1 -- 172.21.15.130:0/1685198979 wait complete. 2023-09-12T16:02:31.245 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T16:02:31.245 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-09-12T16:02:31.245 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-12T16:02:31.286 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph config log 1 --format=json 2023-09-12T16:02:31.876 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:32.105 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:31 smithi130 ceph-mon[98670]: pgmap v276: 129 pgs: 129 active+clean; 585 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-09-12T16:02:32.105 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:31 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1685198979' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-12T16:02:32.106 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:31 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1685198979' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-12T16:02:32.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:31 smithi159 ceph-mon[102522]: pgmap v276: 129 pgs: 129 active+clean; 585 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-09-12T16:02:32.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:31 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1685198979' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-12T16:02:32.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:31 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1685198979' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-12T16:02:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:31 smithi203 ceph-mon[103431]: pgmap v276: 129 pgs: 129 active+clean; 585 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-09-12T16:02:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:31 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1685198979' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-12T16:02:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:31 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1685198979' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-12T16:02:33.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 -- 172.21.15.130:0/642044510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000ff760 msgr2=0x7f2700105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:33.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 --2- 172.21.15.130:0/642044510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000ff760 0x7f2700105220 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009990 tx=0x7f26f4009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 -- 172.21.15.130:0/642044510 shutdown_connections 2023-09-12T16:02:33.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 --2- 172.21.15.130:0/642044510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2700105950 0x7f270010d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 --2- 172.21.15.130:0/642044510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27000fedb0 0x7f27000ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 --2- 172.21.15.130:0/642044510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000ff760 0x7f2700105220 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 -- 172.21.15.130:0/642044510 >> 172.21.15.130:0/642044510 conn(0x7f2700075a70 msgr2=0x7f27000fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:33.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 -- 172.21.15.130:0/642044510 shutdown_connections 2023-09-12T16:02:33.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 -- 172.21.15.130:0/642044510 wait complete. 2023-09-12T16:02:33.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.068+0000 7f27068a7700 1 Processor -- start 2023-09-12T16:02:33.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f27068a7700 1 -- start start 2023-09-12T16:02:33.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f27068a7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000fedb0 0x7f27000ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:33.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f27068a7700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27000ff760 0x7f2700100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:33.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f27068a7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700105950 0x7f27001007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:33.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f27068a7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27001a4a90 con 0x7f27000fedb0 2023-09-12T16:02:33.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f27068a7700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27001a4c00 con 0x7f27000ff760 2023-09-12T16:02:33.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f27068a7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27001a4dc0 con 0x7f2700105950 2023-09-12T16:02:33.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f26fffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000fedb0 0x7f27000ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:33.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f26fffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000fedb0 0x7f27000ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53866/0 (socket says 172.21.15.130:53866) 2023-09-12T16:02:33.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f26fffff700 1 -- 172.21.15.130:0/2333234354 learned_addr learned my addr 172.21.15.130:0/2333234354 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:33.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f26ff7fe700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27000ff760 0x7f2700100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:33.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f2704e44700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700105950 0x7f27001007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:33.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f26fffff700 1 -- 172.21.15.130:0/2333234354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700105950 msgr2=0x7f27001007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:33.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f26fffff700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700105950 0x7f27001007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f26fffff700 1 -- 172.21.15.130:0/2333234354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27000ff760 msgr2=0x7f2700100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:33.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f26fffff700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27000ff760 0x7f2700100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.069+0000 7f26fffff700 1 -- 172.21.15.130:0/2333234354 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40095f0 con 0x7f27000fedb0 2023-09-12T16:02:33.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.070+0000 7f26fffff700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000fedb0 0x7f27000ffd30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c920 tx=0x7f26f000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:33.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.070+0000 7f26fd7fa700 1 -- 172.21.15.130:0/2333234354 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0004a30 con 0x7f27000fedb0 2023-09-12T16:02:33.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.070+0000 7f26fd7fa700 1 -- 172.21.15.130:0/2333234354 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26f00054e0 con 0x7f27000fedb0 2023-09-12T16:02:33.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.070+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27001a50a0 con 0x7f27000fedb0 2023-09-12T16:02:33.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.070+0000 7f26fd7fa700 1 -- 172.21.15.130:0/2333234354 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0007bd0 con 0x7f27000fedb0 2023-09-12T16:02:33.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.070+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27001a55f0 con 0x7f27000fedb0 2023-09-12T16:02:33.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.071+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270004e300 con 0x7f27000fedb0 2023-09-12T16:02:33.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.072+0000 7f26fd7fa700 1 -- 172.21.15.130:0/2333234354 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f26f0007d30 con 0x7f27000fedb0 2023-09-12T16:02:33.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.073+0000 7f26fd7fa700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26e80a0750 0x7f26e80a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:33.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.073+0000 7f26ff7fe700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26e80a0750 0x7f26e80a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:33.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.073+0000 7f26fd7fa700 1 -- 172.21.15.130:0/2333234354 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f26f00cbea0 con 0x7f27000fedb0 2023-09-12T16:02:33.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.073+0000 7f26ff7fe700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26e80a0750 0x7f26e80a2c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2700101520 tx=0x7f26f4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:33.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.076+0000 7f26fd7fa700 1 -- 172.21.15.130:0/2333234354 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f26f0095700 con 0x7f27000fedb0 2023-09-12T16:02:33.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.265+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7f2700110640 con 0x7f27000fedb0 2023-09-12T16:02:33.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.266+0000 7f26fd7fa700 1 -- 172.21.15.130:0/2333234354 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v43) v1 ==== 86+0+133 (secure 0 0 0) 0x7f26f0098d20 con 0x7f27000fedb0 2023-09-12T16:02:33.267 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:33.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26e80a0750 msgr2=0x7f26e80a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:33.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26e80a0750 0x7f26e80a2c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2700101520 tx=0x7f26f4005960 comp rx=0 tx=0).stop 2023-09-12T16:02:33.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000fedb0 msgr2=0x7f27000ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:33.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000fedb0 0x7f27000ffd30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c920 tx=0x7f26f000cc30 comp rx=0 tx=0).stop 2023-09-12T16:02:33.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 shutdown_connections 2023-09-12T16:02:33.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f26e80a0750 0x7f26e80a2c10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700105950 0x7f27001007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27000fedb0 0x7f27000ffd30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 --2- 172.21.15.130:0/2333234354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27000ff760 0x7f2700100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:33.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 >> 172.21.15.130:0/2333234354 conn(0x7f2700075a70 msgr2=0x7f27001041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:33.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 shutdown_connections 2023-09-12T16:02:33.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:33.268+0000 7f27068a7700 1 -- 172.21.15.130:0/2333234354 wait complete. 2023-09-12T16:02:33.628 INFO:teuthology.orchestra.run.smithi130.stdout:[{"version":43,"timestamp":"2023-09-12T16:01:59.755460+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-09-12T16:02:33.629 INFO:tasks.ceph_manager:config epoch is 43 2023-09-12T16:02:33.629 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-12T16:02:33.630 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-09-12T16:02:33.630 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph mgr dump --format=json 2023-09-12T16:02:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:33 smithi130 ceph-mon[98670]: pgmap v277: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2023-09-12T16:02:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:33 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2333234354' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-12T16:02:34.181 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:34.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:33 smithi159 ceph-mon[102522]: pgmap v277: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2023-09-12T16:02:34.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:33 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2333234354' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-12T16:02:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:33 smithi203 ceph-mon[103431]: pgmap v277: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2023-09-12T16:02:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:33 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2333234354' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-12T16:02:35.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.358+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/4052819628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d540ff880 msgr2=0x7f6d540ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:35.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.358+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/4052819628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d540ff880 0x7f6d540ffc60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009990 tx=0x7f6d50009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.358+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/4052819628 shutdown_connections 2023-09-12T16:02:35.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.358+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/4052819628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d54101840 0x7f6d5410c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.358+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/4052819628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d540ff880 0x7f6d540ffc60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.358+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/4052819628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d541001a0 0x7f6d54101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.358+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/4052819628 >> 172.21.15.130:0/4052819628 conn(0x7f6d540765f0 msgr2=0x7f6d54076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:35.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.358+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/4052819628 shutdown_connections 2023-09-12T16:02:35.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.359+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/4052819628 wait complete. 2023-09-12T16:02:35.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.359+0000 7f6d5c3e5700 1 Processor -- start 2023-09-12T16:02:35.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.359+0000 7f6d5c3e5700 1 -- start start 2023-09-12T16:02:35.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5c3e5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d540ff880 0x7f6d5419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:35.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5c3e5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d541001a0 0x7f6d5419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:35.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5c3e5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d54101840 0x7f6d541988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:35.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5c3e5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d54198f50 con 0x7f6d541001a0 2023-09-12T16:02:35.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5c3e5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d541990c0 con 0x7f6d54101840 2023-09-12T16:02:35.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5c3e5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d54199280 con 0x7f6d540ff880 2023-09-12T16:02:35.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d59980700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d541001a0 0x7f6d5419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:35.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d59980700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d541001a0 0x7f6d5419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53882/0 (socket says 172.21.15.130:53882) 2023-09-12T16:02:35.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d59980700 1 -- 172.21.15.130:0/2879933698 learned_addr learned my addr 172.21.15.130:0/2879933698 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:35.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5a982700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d54101840 0x7f6d541988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:35.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5a181700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d540ff880 0x7f6d5419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:35.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d59980700 1 -- 172.21.15.130:0/2879933698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d540ff880 msgr2=0x7f6d5419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:35.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d59980700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d540ff880 0x7f6d5419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d59980700 1 -- 172.21.15.130:0/2879933698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d54101840 msgr2=0x7f6d541988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:35.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d59980700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d54101840 0x7f6d541988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d59980700 1 -- 172.21.15.130:0/2879933698 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500095f0 con 0x7f6d541001a0 2023-09-12T16:02:35.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5a181700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d540ff880 0x7f6d5419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:35.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d5a982700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d54101840 0x7f6d541988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:35.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d59980700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d541001a0 0x7f6d5419eda0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c8f0 tx=0x7f6d4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:35.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.360+0000 7f6d477fe700 1 -- 172.21.15.130:0/2879933698 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d48007b00 con 0x7f6d541001a0 2023-09-12T16:02:35.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.361+0000 7f6d477fe700 1 -- 172.21.15.130:0/2879933698 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d48007c60 con 0x7f6d541001a0 2023-09-12T16:02:35.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.361+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d54199560 con 0x7f6d541001a0 2023-09-12T16:02:35.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.361+0000 7f6d477fe700 1 -- 172.21.15.130:0/2879933698 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d48018660 con 0x7f6d541001a0 2023-09-12T16:02:35.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.361+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d541a5850 con 0x7f6d541001a0 2023-09-12T16:02:35.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.361+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5404e300 con 0x7f6d541001a0 2023-09-12T16:02:35.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.363+0000 7f6d477fe700 1 -- 172.21.15.130:0/2879933698 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f6d4800f450 con 0x7f6d541001a0 2023-09-12T16:02:35.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.363+0000 7f6d477fe700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6d400a06d0 0x7f6d400a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:35.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.363+0000 7f6d477fe700 1 -- 172.21.15.130:0/2879933698 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6d480cbf70 con 0x7f6d541001a0 2023-09-12T16:02:35.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.363+0000 7f6d5a181700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6d400a06d0 0x7f6d400a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:35.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.364+0000 7f6d5a181700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6d400a06d0 0x7f6d400a2b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d50006010 tx=0x7f6d5000b580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:35.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.367+0000 7f6d477fe700 1 -- 172.21.15.130:0/2879933698 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6d480957e0 con 0x7f6d541001a0 2023-09-12T16:02:35.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.610+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6d5410f5b0 con 0x7f6d541001a0 2023-09-12T16:02:35.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.614+0000 7f6d477fe700 1 -- 172.21.15.130:0/2879933698 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+259027 (secure 0 0 0) 0x7f6d48098e00 con 0x7f6d541001a0 2023-09-12T16:02:35.620 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:35.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6d400a06d0 msgr2=0x7f6d400a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:35.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6d400a06d0 0x7f6d400a2b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d50006010 tx=0x7f6d5000b580 comp rx=0 tx=0).stop 2023-09-12T16:02:35.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d541001a0 msgr2=0x7f6d5419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:35.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d541001a0 0x7f6d5419eda0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c8f0 tx=0x7f6d4800ccb0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 shutdown_connections 2023-09-12T16:02:35.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6d400a06d0 0x7f6d400a2b90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d54101840 0x7f6d541988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d540ff880 0x7f6d5419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 --2- 172.21.15.130:0/2879933698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d541001a0 0x7f6d5419eda0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:35.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.617+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 >> 172.21.15.130:0/2879933698 conn(0x7f6d540765f0 msgr2=0x7f6d54068950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:35.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.618+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 shutdown_connections 2023-09-12T16:02:35.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:35.618+0000 7f6d5c3e5700 1 -- 172.21.15.130:0/2879933698 wait complete. 2023-09-12T16:02:35.952 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:35 smithi130 ceph-mon[98670]: pgmap v278: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2023-09-12T16:02:35.953 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:35 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2879933698' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-12T16:02:36.030 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4273925772},{"type":"v1","addr":"172.21.15.130:6801","nonce":4273925772}]},"active_addr":"172.21.15.130:6801/4273925772","active_change":"2023-09-12T15:55:50.680354+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24194,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24205,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.130:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.130:0","nonce":609656151}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.130:0","nonce":2101559487}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.130:0","nonce":1633987371}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.130:0","nonce":3484494959}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.130:0","nonce":1305930682}]}]} 2023-09-12T16:02:36.037 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-09-12T16:02:36.037 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-09-12T16:02:36.038 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-12T16:02:36.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:35 smithi159 ceph-mon[102522]: pgmap v278: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2023-09-12T16:02:36.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:35 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2879933698' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-12T16:02:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:35 smithi203 ceph-mon[103431]: pgmap v278: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2023-09-12T16:02:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:35 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2879933698' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-12T16:02:36.591 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:37.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.756+0000 7f2699acc700 1 -- 172.21.15.130:0/1064505101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ff520 msgr2=0x7f26940ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:37.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.756+0000 7f2699acc700 1 --2- 172.21.15.130:0/1064505101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ff520 0x7f26940ff9a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2684009990 tx=0x7f2684009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.756+0000 7f2699acc700 1 -- 172.21.15.130:0/1064505101 shutdown_connections 2023-09-12T16:02:37.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.756+0000 7f2699acc700 1 --2- 172.21.15.130:0/1064505101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26940ffee0 0x7f269410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.756+0000 7f2699acc700 1 --2- 172.21.15.130:0/1064505101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2694104530 0x7f2694104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.756+0000 7f2699acc700 1 --2- 172.21.15.130:0/1064505101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ff520 0x7f26940ff9a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.756+0000 7f2699acc700 1 -- 172.21.15.130:0/1064505101 >> 172.21.15.130:0/1064505101 conn(0x7f2694076600 msgr2=0x7f2694076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:37.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.756+0000 7f2699acc700 1 -- 172.21.15.130:0/1064505101 shutdown_connections 2023-09-12T16:02:37.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.756+0000 7f2699acc700 1 -- 172.21.15.130:0/1064505101 wait complete. 2023-09-12T16:02:37.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2699acc700 1 Processor -- start 2023-09-12T16:02:37.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2699acc700 1 -- start start 2023-09-12T16:02:37.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2699acc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26940ff520 0x7f2694101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:37.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2699acc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ffee0 0x7f26941016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:37.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2699acc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2694104530 0x7f2694101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:37.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2699acc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26941a4be0 con 0x7f26940ffee0 2023-09-12T16:02:37.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2699acc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26941a4d50 con 0x7f2694104530 2023-09-12T16:02:37.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2699acc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26941a4f10 con 0x7f26940ff520 2023-09-12T16:02:37.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2692ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ffee0 0x7f26941016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:37.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2692ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ffee0 0x7f26941016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53896/0 (socket says 172.21.15.130:53896) 2023-09-12T16:02:37.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2692ffd700 1 -- 172.21.15.130:0/517300642 learned_addr learned my addr 172.21.15.130:0/517300642 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:37.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f26937fe700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26940ff520 0x7f2694101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:37.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2693fff700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2694104530 0x7f2694101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:37.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2692ffd700 1 -- 172.21.15.130:0/517300642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26940ff520 msgr2=0x7f2694101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:37.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2692ffd700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26940ff520 0x7f2694101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2692ffd700 1 -- 172.21.15.130:0/517300642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2694104530 msgr2=0x7f2694101c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:37.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2692ffd700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2694104530 0x7f2694101c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.757+0000 7f2692ffd700 1 -- 172.21.15.130:0/517300642 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840095f0 con 0x7f26940ffee0 2023-09-12T16:02:37.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.758+0000 7f2692ffd700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ffee0 0x7f26941016c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2684000c00 tx=0x7f268400f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:37.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.758+0000 7f2690ff9700 1 -- 172.21.15.130:0/517300642 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268401d070 con 0x7f26940ffee0 2023-09-12T16:02:37.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.758+0000 7f2690ff9700 1 -- 172.21.15.130:0/517300642 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f268400fc30 con 0x7f26940ffee0 2023-09-12T16:02:37.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.758+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26941a5190 con 0x7f26940ffee0 2023-09-12T16:02:37.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.758+0000 7f2690ff9700 1 -- 172.21.15.130:0/517300642 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684017650 con 0x7f26940ffee0 2023-09-12T16:02:37.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.758+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26941a5630 con 0x7f26940ffee0 2023-09-12T16:02:37.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.759+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269404e300 con 0x7f26940ffee0 2023-09-12T16:02:37.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.760+0000 7f2690ff9700 1 -- 172.21.15.130:0/517300642 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f2684022470 con 0x7f26940ffee0 2023-09-12T16:02:37.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.760+0000 7f2690ff9700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f267c0a06d0 0x7f267c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:37.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.760+0000 7f26937fe700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f267c0a06d0 0x7f267c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:37.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.760+0000 7f2690ff9700 1 -- 172.21.15.130:0/517300642 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f26840cd450 con 0x7f26940ffee0 2023-09-12T16:02:37.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.761+0000 7f26937fe700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f267c0a06d0 0x7f267c0a2b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2694069c10 tx=0x7f268800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:37.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.765+0000 7f2690ff9700 1 -- 172.21.15.130:0/517300642 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f268405ce10 con 0x7f26940ffee0 2023-09-12T16:02:37.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.955+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f269410f570 con 0x7f26940ffee0 2023-09-12T16:02:37.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.957+0000 7f2690ff9700 1 -- 172.21.15.130:0/517300642 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24004 (secure 0 0 0) 0x7f2684027090 con 0x7f26940ffee0 2023-09-12T16:02:37.958 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:37.958 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":86,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:11.262527+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:37.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f267c0a06d0 msgr2=0x7f267c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:37.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f267c0a06d0 0x7f267c0a2b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2694069c10 tx=0x7f268800a380 comp rx=0 tx=0).stop 2023-09-12T16:02:37.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ffee0 msgr2=0x7f26941016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:37.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ffee0 0x7f26941016c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2684000c00 tx=0x7f268400f780 comp rx=0 tx=0).stop 2023-09-12T16:02:37.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 shutdown_connections 2023-09-12T16:02:37.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f267c0a06d0 0x7f267c0a2b90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2694104530 0x7f2694101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26940ff520 0x7f2694101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 --2- 172.21.15.130:0/517300642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26940ffee0 0x7f26941016c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:37.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 >> 172.21.15.130:0/517300642 conn(0x7f2694076600 msgr2=0x7f2694068160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:37.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 shutdown_connections 2023-09-12T16:02:37.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:37.959+0000 7f2699acc700 1 -- 172.21.15.130:0/517300642 wait complete. 2023-09-12T16:02:37.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:37 smithi130 ceph-mon[98670]: pgmap v279: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-09-12T16:02:37.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:37 smithi159 ceph-mon[102522]: pgmap v279: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-09-12T16:02:37.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:37 smithi203 ceph-mon[103431]: pgmap v279: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-09-12T16:02:38.339 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-09-12T16:02:38.340 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-12T16:02:38.891 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:38.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:38 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/517300642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:38.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:38 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/517300642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:38.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:38 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/517300642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:39.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:39 smithi130 ceph-mon[98670]: pgmap v280: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2023-09-12T16:02:39.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:39 smithi159 ceph-mon[102522]: pgmap v280: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2023-09-12T16:02:39.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:39 smithi203 ceph-mon[103431]: pgmap v280: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2023-09-12T16:02:40.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 -- 172.21.15.130:0/3552374494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c100890 msgr2=0x7f6a5c100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:40.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 --2- 172.21.15.130:0/3552374494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c100890 0x7f6a5c100d10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009990 tx=0x7f6a48009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 -- 172.21.15.130:0/3552374494 shutdown_connections 2023-09-12T16:02:40.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 --2- 172.21.15.130:0/3552374494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a5c101250 0x7f6a5c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 --2- 172.21.15.130:0/3552374494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a5c106760 0x7f6a5c106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 --2- 172.21.15.130:0/3552374494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c100890 0x7f6a5c100d10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 -- 172.21.15.130:0/3552374494 >> 172.21.15.130:0/3552374494 conn(0x7f6a5c078620 msgr2=0x7f6a5c078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:40.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 -- 172.21.15.130:0/3552374494 shutdown_connections 2023-09-12T16:02:40.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 -- 172.21.15.130:0/3552374494 wait complete. 2023-09-12T16:02:40.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 Processor -- start 2023-09-12T16:02:40.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.082+0000 7f6a62ade700 1 -- start start 2023-09-12T16:02:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a62ade700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a5c100890 0x7f6a5c1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a62ade700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a5c101250 0x7f6a5c196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a62ade700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c106760 0x7f6a5c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a62ade700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c19a5c0 con 0x7f6a5c106760 2023-09-12T16:02:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a62ade700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c19a730 con 0x7f6a5c100890 2023-09-12T16:02:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a62ade700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c19a8f0 con 0x7f6a5c101250 2023-09-12T16:02:40.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6107b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c106760 0x7f6a5c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:40.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6107b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c106760 0x7f6a5c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53912/0 (socket says 172.21.15.130:53912) 2023-09-12T16:02:40.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6107b700 1 -- 172.21.15.130:0/110478487 learned_addr learned my addr 172.21.15.130:0/110478487 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:40.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a5bfff700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a5c101250 0x7f6a5c196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:40.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6087a700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a5c100890 0x7f6a5c1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:40.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6107b700 1 -- 172.21.15.130:0/110478487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a5c101250 msgr2=0x7f6a5c196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:40.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6107b700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a5c101250 0x7f6a5c196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6107b700 1 -- 172.21.15.130:0/110478487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a5c100890 msgr2=0x7f6a5c1966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:40.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6107b700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a5c100890 0x7f6a5c1966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6107b700 1 -- 172.21.15.130:0/110478487 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480095f0 con 0x7f6a5c106760 2023-09-12T16:02:40.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6087a700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a5c100890 0x7f6a5c1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:40.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a6107b700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c106760 0x7f6a5c199fc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400ada0 tx=0x7f6a5400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:40.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a59ffb700 1 -- 172.21.15.130:0/110478487 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400f070 con 0x7f6a5c106760 2023-09-12T16:02:40.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a59ffb700 1 -- 172.21.15.130:0/110478487 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a54007b30 con 0x7f6a5c106760 2023-09-12T16:02:40.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c19abd0 con 0x7f6a5c106760 2023-09-12T16:02:40.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.083+0000 7f6a59ffb700 1 -- 172.21.15.130:0/110478487 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54005440 con 0x7f6a5c106760 2023-09-12T16:02:40.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.084+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c1a3520 con 0x7f6a5c106760 2023-09-12T16:02:40.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.085+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c04e300 con 0x7f6a5c106760 2023-09-12T16:02:40.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.085+0000 7f6a59ffb700 1 -- 172.21.15.130:0/110478487 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f6a54004ad0 con 0x7f6a5c106760 2023-09-12T16:02:40.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.086+0000 7f6a59ffb700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6a4c0a06d0 0x7f6a4c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:40.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.086+0000 7f6a6087a700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6a4c0a06d0 0x7f6a4c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:40.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.086+0000 7f6a59ffb700 1 -- 172.21.15.130:0/110478487 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6a540cb5e0 con 0x7f6a5c106760 2023-09-12T16:02:40.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.087+0000 7f6a6087a700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6a4c0a06d0 0x7f6a4c0a2b90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6a50007900 tx=0x7f6a50008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:40.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.090+0000 7f6a59ffb700 1 -- 172.21.15.130:0/110478487 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6a54094e50 con 0x7f6a5c106760 2023-09-12T16:02:40.283 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.282+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a5c110690 con 0x7f6a5c106760 2023-09-12T16:02:40.284 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.283+0000 7f6a59ffb700 1 -- 172.21.15.130:0/110478487 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24004 (secure 0 0 0) 0x7f6a54098470 con 0x7f6a5c106760 2023-09-12T16:02:40.284 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:02:40.285 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":86,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:11.262527+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:02:40.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6a4c0a06d0 msgr2=0x7f6a4c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:40.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6a4c0a06d0 0x7f6a4c0a2b90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6a50007900 tx=0x7f6a50008040 comp rx=0 tx=0).stop 2023-09-12T16:02:40.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c106760 msgr2=0x7f6a5c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:40.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c106760 0x7f6a5c199fc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400ada0 tx=0x7f6a5400c330 comp rx=0 tx=0).stop 2023-09-12T16:02:40.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 shutdown_connections 2023-09-12T16:02:40.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6a4c0a06d0 0x7f6a4c0a2b90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6a5c106760 0x7f6a5c199fc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a5c100890 0x7f6a5c1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 --2- 172.21.15.130:0/110478487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a5c101250 0x7f6a5c196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:40.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 >> 172.21.15.130:0/110478487 conn(0x7f6a5c078620 msgr2=0x7f6a5c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:40.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 shutdown_connections 2023-09-12T16:02:40.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:40.286+0000 7f6a62ade700 1 -- 172.21.15.130:0/110478487 wait complete. 2023-09-12T16:02:40.652 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.0 flush_pg_stats 2023-09-12T16:02:40.653 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.1 flush_pg_stats 2023-09-12T16:02:40.654 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.2 flush_pg_stats 2023-09-12T16:02:40.655 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.3 flush_pg_stats 2023-09-12T16:02:40.655 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.4 flush_pg_stats 2023-09-12T16:02:40.656 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.5 flush_pg_stats 2023-09-12T16:02:40.657 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.6 flush_pg_stats 2023-09-12T16:02:40.657 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.7 flush_pg_stats 2023-09-12T16:02:40.658 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.8 flush_pg_stats 2023-09-12T16:02:40.659 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.9 flush_pg_stats 2023-09-12T16:02:40.660 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.10 flush_pg_stats 2023-09-12T16:02:40.661 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph tell osd.11 flush_pg_stats 2023-09-12T16:02:40.895 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:40 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/110478487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:40.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:40 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/110478487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:40.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:40 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/110478487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:02:41.640 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:41.655 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:41.884 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:41.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:41 smithi130 ceph-mon[98670]: pgmap v281: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-09-12T16:02:41.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:41 smithi159 ceph-mon[102522]: pgmap v281: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-09-12T16:02:41.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:41 smithi203 ceph-mon[103431]: pgmap v281: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-09-12T16:02:42.128 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:42.171 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:42.251 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:42.476 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:42.557 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:42.772 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:43.065 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:43.491 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:43.640 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:44.018 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:43 smithi130 ceph-mon[98670]: pgmap v282: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2023-09-12T16:02:44.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:43 smithi159 ceph-mon[102522]: pgmap v282: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2023-09-12T16:02:44.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:43 smithi203 ceph-mon[103431]: pgmap v282: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2023-09-12T16:02:46.174 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:45 smithi130 ceph-mon[98670]: pgmap v283: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-09-12T16:02:46.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:45 smithi159 ceph-mon[102522]: pgmap v283: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-09-12T16:02:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:45 smithi203 ceph-mon[103431]: pgmap v283: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-09-12T16:02:46.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.984+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/1367406518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c9c104530 msgr2=0x7f9c9c104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:46.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.984+0000 7f9ca3ff2700 1 --2- 172.21.15.130:0/1367406518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c9c104530 0x7f9c9c104910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009990 tx=0x7f9c98009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:46.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.984+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/1367406518 shutdown_connections 2023-09-12T16:02:46.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.984+0000 7f9ca3ff2700 1 --2- 172.21.15.130:0/1367406518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c9c0ffee0 0x7f9c9c10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:46.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.984+0000 7f9ca3ff2700 1 --2- 172.21.15.130:0/1367406518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c9c104530 0x7f9c9c104910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:46.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.984+0000 7f9ca3ff2700 1 --2- 172.21.15.130:0/1367406518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c9c0ff520 0x7f9c9c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:46.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.984+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/1367406518 >> 172.21.15.130:0/1367406518 conn(0x7f9c9c076600 msgr2=0x7f9c9c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:46.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.984+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/1367406518 shutdown_connections 2023-09-12T16:02:46.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.984+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/1367406518 wait complete. 2023-09-12T16:02:46.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.985+0000 7f9ca3ff2700 1 Processor -- start 2023-09-12T16:02:46.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.985+0000 7f9ca3ff2700 1 -- start start 2023-09-12T16:02:46.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca3ff2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c9c0ff520 0x7f9c9c1115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:46.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca3ff2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c9c0ffee0 0x7f9c9c10fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:46.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca3ff2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c9c104530 0x7f9c9c110340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:46.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca3ff2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c110880 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca3ff2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c1109f0 con 0x7f9c9c0ff520 2023-09-12T16:02:46.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca3ff2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c110b60 con 0x7f9c9c104530 2023-09-12T16:02:46.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca158d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c9c0ffee0 0x7f9c9c10fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:46.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca158d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c9c0ffee0 0x7f9c9c10fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53924/0 (socket says 172.21.15.130:53924) 2023-09-12T16:02:46.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca158d700 1 -- 172.21.15.130:0/3827050616 learned_addr learned my addr 172.21.15.130:0/3827050616 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:46.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca1d8e700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c9c0ff520 0x7f9c9c1115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:46.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca258f700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c9c104530 0x7f9c9c110340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:46.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca158d700 1 -- 172.21.15.130:0/3827050616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c9c104530 msgr2=0x7f9c9c110340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:46.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca158d700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c9c104530 0x7f9c9c110340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:46.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca158d700 1 -- 172.21.15.130:0/3827050616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c9c0ff520 msgr2=0x7f9c9c1115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:46.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca158d700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c9c0ff520 0x7f9c9c1115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:46.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca158d700 1 -- 172.21.15.130:0/3827050616 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c980095f0 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.986+0000 7f9ca1d8e700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c9c0ff520 0x7f9c9c1115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:02:46.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.987+0000 7f9ca158d700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c9c0ffee0 0x7f9c9c10fe00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00d980 tx=0x7f9c8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:46.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.987+0000 7f9c92ffd700 1 -- 172.21.15.130:0/3827050616 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c004a30 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.987+0000 7f9c92ffd700 1 -- 172.21.15.130:0/3827050616 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c8c0054e0 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.987+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c1a5200 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.987+0000 7f9c92ffd700 1 -- 172.21.15.130:0/3827050616 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00fbc0 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.987+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c1a56d0 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.988+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c9c04e300 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.990+0000 7f9c92ffd700 1 -- 172.21.15.130:0/3827050616 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f9c8c004c20 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.990+0000 7f9c92ffd700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c880a0750 0x7f9c880a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:46.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.990+0000 7f9ca1d8e700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c880a0750 0x7f9c880a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:46.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.990+0000 7f9c92ffd700 1 -- 172.21.15.130:0/3827050616 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9c8c0cbcf0 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.990+0000 7f9c92ffd700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] conn(0x7f9c880ab2d0 0x7f9c880ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:46.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.991+0000 7f9c92ffd700 1 -- 172.21.15.130:0/3827050616 --> [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c880adde0 con 0x7f9c880ab2d0 2023-09-12T16:02:46.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.991+0000 7f9c92ffd700 1 -- 172.21.15.130:0/3827050616 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c8c0ce370 con 0x7f9c9c0ffee0 2023-09-12T16:02:46.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.991+0000 7f9ca258f700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] conn(0x7f9c880ab2d0 0x7f9c880ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:46.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.991+0000 7f9ca1d8e700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c880a0750 0x7f9c880a2c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800b5c0 tx=0x7f9c98005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:46.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.991+0000 7f9ca258f700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] conn(0x7f9c880ab2d0 0x7f9c880ad710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:46.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:46.992+0000 7f9c92ffd700 1 -- 172.21.15.130:0/3827050616 <== osd.6 v2:172.21.15.130:6818/291320749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9c880adde0 con 0x7f9c880ab2d0 2023-09-12T16:02:47.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.008+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 --> [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c9c102a10 con 0x7f9c880ab2d0 2023-09-12T16:02:47.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.008+0000 7f9c92ffd700 1 -- 172.21.15.130:0/3827050616 <== osd.6 v2:172.21.15.130:6818/291320749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9c9c102a10 con 0x7f9c880ab2d0 2023-09-12T16:02:47.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] conn(0x7f9c880ab2d0 msgr2=0x7f9c880ad710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] conn(0x7f9c880ab2d0 0x7f9c880ad710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c880a0750 msgr2=0x7f9c880a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9c880a0750 0x7f9c880a2c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800b5c0 tx=0x7f9c98005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c9c0ffee0 msgr2=0x7f9c9c10fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 --2- 172.21.15.130:0/3827050616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c9c0ffee0 0x7f9c9c10fe00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00d980 tx=0x7f9c8c00dc90 comp rx=0 tx=0).stop 2023-09-12T16:02:47.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca258f700 1 -- 172.21.15.130:0/3827050616 reap_dead start 2023-09-12T16:02:47.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 shutdown_connections 2023-09-12T16:02:47.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 >> 172.21.15.130:0/3827050616 conn(0x7f9c9c076600 msgr2=0x7f9c9c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:47.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 shutdown_connections 2023-09-12T16:02:47.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.009+0000 7f9ca3ff2700 1 -- 172.21.15.130:0/3827050616 wait complete. 2023-09-12T16:02:47.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.092+0000 7f97f3b93700 1 -- 172.21.15.130:0/533205937 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97ec106770 msgr2=0x7f97ec106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.092+0000 7f97f3b93700 1 --2- 172.21.15.130:0/533205937 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97ec106770 0x7f97ec106b50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009990 tx=0x7f97e8009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.092+0000 7f97f3b93700 1 -- 172.21.15.130:0/533205937 shutdown_connections 2023-09-12T16:02:47.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.092+0000 7f97f3b93700 1 --2- 172.21.15.130:0/533205937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97ec068ae0 0x7f97ec10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.092+0000 7f97f3b93700 1 --2- 172.21.15.130:0/533205937 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97ec106770 0x7f97ec106b50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.092+0000 7f97f3b93700 1 --2- 172.21.15.130:0/533205937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97ec068120 0x7f97ec0685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.092+0000 7f97f3b93700 1 -- 172.21.15.130:0/533205937 >> 172.21.15.130:0/533205937 conn(0x7f97ec076600 msgr2=0x7f97ec076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:47.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.092+0000 7f97f3b93700 1 -- 172.21.15.130:0/533205937 shutdown_connections 2023-09-12T16:02:47.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.092+0000 7f97f3b93700 1 -- 172.21.15.130:0/533205937 wait complete. 2023-09-12T16:02:47.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f3b93700 1 Processor -- start 2023-09-12T16:02:47.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f3b93700 1 -- start start 2023-09-12T16:02:47.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f3b93700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97ec068120 0x7f97ec19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f3b93700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97ec068ae0 0x7f97ec19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f3b93700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97ec106770 0x7f97ec19d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f3b93700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec19d950 con 0x7f97ec106770 2023-09-12T16:02:47.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f3b93700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec19da90 con 0x7f97ec068ae0 2023-09-12T16:02:47.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f3b93700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec19dc50 con 0x7f97ec068120 2023-09-12T16:02:47.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f112e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97ec068ae0 0x7f97ec19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f2130700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97ec106770 0x7f97ec19d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f112e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97ec068ae0 0x7f97ec19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:37196/0 (socket says 172.21.15.130:37196) 2023-09-12T16:02:47.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.093+0000 7f97f112e700 1 -- 172.21.15.130:0/525521639 learned_addr learned my addr 172.21.15.130:0/525521639 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:47.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f192f700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97ec068120 0x7f97ec19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f112e700 1 -- 172.21.15.130:0/525521639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97ec068120 msgr2=0x7f97ec19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f112e700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97ec068120 0x7f97ec19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f112e700 1 -- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97ec106770 msgr2=0x7f97ec19d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f112e700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97ec106770 0x7f97ec19d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f112e700 1 -- 172.21.15.130:0/525521639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e80095f0 con 0x7f97ec068ae0 2023-09-12T16:02:47.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f2130700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97ec106770 0x7f97ec19d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:02:47.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f112e700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97ec068ae0 0x7f97ec19eaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00c9b0 tx=0x7f97dc00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:47.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97e2ffd700 1 -- 172.21.15.130:0/525521639 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc007b00 con 0x7f97ec068ae0 2023-09-12T16:02:47.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec19ded0 con 0x7f97ec068ae0 2023-09-12T16:02:47.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97e2ffd700 1 -- 172.21.15.130:0/525521639 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97dc007c60 con 0x7f97ec068ae0 2023-09-12T16:02:47.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97e2ffd700 1 -- 172.21.15.130:0/525521639 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc018660 con 0x7f97ec068ae0 2023-09-12T16:02:47.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.094+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec110050 con 0x7f97ec068ae0 2023-09-12T16:02:47.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.095+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97ec0fff20 con 0x7f97ec068ae0 2023-09-12T16:02:47.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.096+0000 7f97e2ffd700 1 -- 172.21.15.130:0/525521639 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f97dc00f450 con 0x7f97ec068ae0 2023-09-12T16:02:47.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.097+0000 7f97e2ffd700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f97d80a06d0 0x7f97d80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.097+0000 7f97f192f700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f97d80a06d0 0x7f97d80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.097+0000 7f97e2ffd700 1 -- 172.21.15.130:0/525521639 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f97dc0cc300 con 0x7f97ec068ae0 2023-09-12T16:02:47.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.098+0000 7f97e2ffd700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] conn(0x7f97d80ab250 0x7f97d80ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.098+0000 7f97e2ffd700 1 -- 172.21.15.130:0/525521639 --> [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97d80add60 con 0x7f97d80ab250 2023-09-12T16:02:47.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.098+0000 7f97e2ffd700 1 -- 172.21.15.130:0/525521639 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97dc0cc710 con 0x7f97ec068ae0 2023-09-12T16:02:47.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.098+0000 7f97f2130700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] conn(0x7f97d80ab250 0x7f97d80ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.098+0000 7f97f192f700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f97d80a06d0 0x7f97d80a2b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f97e800b5c0 tx=0x7f97e8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:47.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.098+0000 7f97f2130700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] conn(0x7f97d80ab250 0x7f97d80ad690 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:47.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.099+0000 7f97e2ffd700 1 -- 172.21.15.130:0/525521639 <== osd.3 v2:172.21.15.130:6810/3784588793 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f97d80add60 con 0x7f97d80ab250 2023-09-12T16:02:47.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.119+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 --> [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97ec04e300 con 0x7f97d80ab250 2023-09-12T16:02:47.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.119+0000 7f97e2ffd700 1 -- 172.21.15.130:0/525521639 <== osd.3 v2:172.21.15.130:6810/3784588793 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f97ec04e300 con 0x7f97d80ab250 2023-09-12T16:02:47.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] conn(0x7f97d80ab250 msgr2=0x7f97d80ad690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] conn(0x7f97d80ab250 0x7f97d80ad690 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f97d80a06d0 msgr2=0x7f97d80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f97d80a06d0 0x7f97d80a2b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f97e800b5c0 tx=0x7f97e8005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97ec068ae0 msgr2=0x7f97ec19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 --2- 172.21.15.130:0/525521639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97ec068ae0 0x7f97ec19eaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00c9b0 tx=0x7f97dc00ccc0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f2130700 1 -- 172.21.15.130:0/525521639 reap_dead start 2023-09-12T16:02:47.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 shutdown_connections 2023-09-12T16:02:47.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 >> 172.21.15.130:0/525521639 conn(0x7f97ec076600 msgr2=0x7f97ec1021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:47.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 shutdown_connections 2023-09-12T16:02:47.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.120+0000 7f97f3b93700 1 -- 172.21.15.130:0/525521639 wait complete. 2023-09-12T16:02:47.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/248285370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b74104530 msgr2=0x7f3b74104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 --2- 172.21.15.130:0/248285370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b74104530 0x7f3b74104910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b0a0 tx=0x7f3b6400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/248285370 shutdown_connections 2023-09-12T16:02:47.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 --2- 172.21.15.130:0/248285370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b740ffee0 0x7f3b7410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 --2- 172.21.15.130:0/248285370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b74104530 0x7f3b74104910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 --2- 172.21.15.130:0/248285370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b740ff520 0x7f3b740ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/248285370 >> 172.21.15.130:0/248285370 conn(0x7f3b74076600 msgr2=0x7f3b74076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:47.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/248285370 shutdown_connections 2023-09-12T16:02:47.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/248285370 wait complete. 2023-09-12T16:02:47.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.929+0000 7f3b7b2e0700 1 Processor -- start 2023-09-12T16:02:47.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7b2e0700 1 -- start start 2023-09-12T16:02:47.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7b2e0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b740ff520 0x7f3b7419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7b2e0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b740ffee0 0x7f3b7419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7b2e0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b74104530 0x7f3b74198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7b2e0700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74198ff0 con 0x7f3b740ff520 2023-09-12T16:02:47.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7b2e0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74199160 con 0x7f3b740ffee0 2023-09-12T16:02:47.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7b2e0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74199320 con 0x7f3b74104530 2023-09-12T16:02:47.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7907c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b740ff520 0x7f3b7419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7887b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b740ffee0 0x7f3b7419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7907c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b740ff520 0x7f3b7419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53954/0 (socket says 172.21.15.130:53954) 2023-09-12T16:02:47.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7887b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b740ffee0 0x7f3b7419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:37214/0 (socket says 172.21.15.130:37214) 2023-09-12T16:02:47.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7987d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b74104530 0x7f3b74198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7907c700 1 -- 172.21.15.130:0/1005354902 learned_addr learned my addr 172.21.15.130:0/1005354902 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:47.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7907c700 1 -- 172.21.15.130:0/1005354902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b74104530 msgr2=0x7f3b74198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7907c700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b74104530 0x7f3b74198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7907c700 1 -- 172.21.15.130:0/1005354902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b740ffee0 msgr2=0x7f3b7419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7907c700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b740ffee0 0x7f3b7419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7907c700 1 -- 172.21.15.130:0/1005354902 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b64009d00 con 0x7f3b740ff520 2023-09-12T16:02:47.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.930+0000 7f3b7907c700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b740ff520 0x7f3b7419e900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b070 tx=0x7f3b64004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:47.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.931+0000 7f3b727fc700 1 -- 172.21.15.130:0/1005354902 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400e040 con 0x7f3b740ff520 2023-09-12T16:02:47.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.931+0000 7f3b727fc700 1 -- 172.21.15.130:0/1005354902 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b64009460 con 0x7f3b740ff520 2023-09-12T16:02:47.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.931+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b741995a0 con 0x7f3b740ff520 2023-09-12T16:02:47.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.931+0000 7f3b727fc700 1 -- 172.21.15.130:0/1005354902 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64005730 con 0x7f3b740ff520 2023-09-12T16:02:47.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.931+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b741a58f0 con 0x7f3b740ff520 2023-09-12T16:02:47.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.931+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b7410f750 con 0x7f3b740ff520 2023-09-12T16:02:47.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.932+0000 7f3b727fc700 1 -- 172.21.15.130:0/1005354902 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f3b64005890 con 0x7f3b740ff520 2023-09-12T16:02:47.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.933+0000 7f3b727fc700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3b5c0a0750 0x7f3b5c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.933+0000 7f3b7887b700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3b5c0a0750 0x7f3b5c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.933+0000 7f3b727fc700 1 -- 172.21.15.130:0/1005354902 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3b640ce1f0 con 0x7f3b740ff520 2023-09-12T16:02:47.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.933+0000 7f3b727fc700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] conn(0x7f3b5c0ab1a0 0x7f3b5c0ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:47.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.933+0000 7f3b727fc700 1 -- 172.21.15.130:0/1005354902 --> [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b5c0adcb0 con 0x7f3b5c0ab1a0 2023-09-12T16:02:47.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.933+0000 7f3b727fc700 1 -- 172.21.15.130:0/1005354902 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b640ce600 con 0x7f3b740ff520 2023-09-12T16:02:47.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.933+0000 7f3b7887b700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3b5c0a0750 0x7f3b5c0a2c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b74076f10 tx=0x7f3b6800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:47.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.934+0000 7f3b7987d700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] conn(0x7f3b5c0ab1a0 0x7f3b5c0ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:47.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.934+0000 7f3b7987d700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] conn(0x7f3b5c0ab1a0 0x7f3b5c0ad5e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:47.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.935+0000 7f3b727fc700 1 -- 172.21.15.130:0/1005354902 <== osd.5 v2:172.21.15.203:6808/2191150970 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3b5c0adcb0 con 0x7f3b5c0ab1a0 2023-09-12T16:02:47.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.952+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 --> [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3b740fecb0 con 0x7f3b5c0ab1a0 2023-09-12T16:02:47.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.952+0000 7f3b727fc700 1 -- 172.21.15.130:0/1005354902 <== osd.5 v2:172.21.15.203:6808/2191150970 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3b740fecb0 con 0x7f3b5c0ab1a0 2023-09-12T16:02:47.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 >> [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] conn(0x7f3b5c0ab1a0 msgr2=0x7f3b5c0ad5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] conn(0x7f3b5c0ab1a0 0x7f3b5c0ad5e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:47.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3b5c0a0750 msgr2=0x7f3b5c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3b5c0a0750 0x7f3b5c0a2c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b74076f10 tx=0x7f3b6800a300 comp rx=0 tx=0).stop 2023-09-12T16:02:47.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b740ff520 msgr2=0x7f3b7419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:47.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 --2- 172.21.15.130:0/1005354902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b740ff520 0x7f3b7419e900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b070 tx=0x7f3b64004d00 comp rx=0 tx=0).stop 2023-09-12T16:02:47.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7987d700 1 -- 172.21.15.130:0/1005354902 reap_dead start 2023-09-12T16:02:47.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 shutdown_connections 2023-09-12T16:02:47.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 >> 172.21.15.130:0/1005354902 conn(0x7f3b74076600 msgr2=0x7f3b740fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:47.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 shutdown_connections 2023-09-12T16:02:47.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:47.953+0000 7f3b7b2e0700 1 -- 172.21.15.130:0/1005354902 wait complete. 2023-09-12T16:02:47.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:47 smithi130 ceph-mon[98670]: pgmap v284: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-09-12T16:02:47.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:47 smithi159 ceph-mon[102522]: pgmap v284: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-09-12T16:02:47.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:47 smithi203 ceph-mon[103431]: pgmap v284: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-09-12T16:02:48.261 INFO:teuthology.orchestra.run.smithi130.stdout:188978561065 2023-09-12T16:02:48.262 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.6 2023-09-12T16:02:49.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.195+0000 7f7c18130700 1 -- 172.21.15.130:0/1544527730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c10068120 msgr2=0x7f7c100685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.195+0000 7f7c18130700 1 --2- 172.21.15.130:0/1544527730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c10068120 0x7f7c100685a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009990 tx=0x7f7c04009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.195+0000 7f7c18130700 1 -- 172.21.15.130:0/1544527730 shutdown_connections 2023-09-12T16:02:49.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.195+0000 7f7c18130700 1 --2- 172.21.15.130:0/1544527730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c10068ae0 0x7f7c1010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.195+0000 7f7c18130700 1 --2- 172.21.15.130:0/1544527730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c10106770 0x7f7c10106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.195+0000 7f7c18130700 1 --2- 172.21.15.130:0/1544527730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c10068120 0x7f7c100685a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.195+0000 7f7c18130700 1 -- 172.21.15.130:0/1544527730 >> 172.21.15.130:0/1544527730 conn(0x7f7c10076600 msgr2=0x7f7c10076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:49.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.195+0000 7f7c18130700 1 -- 172.21.15.130:0/1544527730 shutdown_connections 2023-09-12T16:02:49.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.195+0000 7f7c18130700 1 -- 172.21.15.130:0/1544527730 wait complete. 2023-09-12T16:02:49.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c18130700 1 Processor -- start 2023-09-12T16:02:49.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c18130700 1 -- start start 2023-09-12T16:02:49.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c18130700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c10068120 0x7f7c100ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c18130700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c10068ae0 0x7f7c100ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c18130700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c10106770 0x7f7c10100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c18130700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101a4b40 con 0x7f7c10068ae0 2023-09-12T16:02:49.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c18130700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101a4cb0 con 0x7f7c10068120 2023-09-12T16:02:49.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c18130700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101a4e70 con 0x7f7c10106770 2023-09-12T16:02:49.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c156cb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c10068ae0 0x7f7c100ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c156cb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c10068ae0 0x7f7c100ffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53970/0 (socket says 172.21.15.130:53970) 2023-09-12T16:02:49.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c156cb700 1 -- 172.21.15.130:0/1667684340 learned_addr learned my addr 172.21.15.130:0/1667684340 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:49.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c15ecc700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c10068120 0x7f7c100ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c166cd700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c10106770 0x7f7c10100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c156cb700 1 -- 172.21.15.130:0/1667684340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c10106770 msgr2=0x7f7c10100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c156cb700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c10106770 0x7f7c10100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.196+0000 7f7c156cb700 1 -- 172.21.15.130:0/1667684340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c10068120 msgr2=0x7f7c100ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.197+0000 7f7c156cb700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c10068120 0x7f7c100ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.197+0000 7f7c156cb700 1 -- 172.21.15.130:0/1667684340 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040095f0 con 0x7f7c10068ae0 2023-09-12T16:02:49.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.197+0000 7f7c166cd700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c10106770 0x7f7c10100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:49.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.197+0000 7f7c156cb700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c10068ae0 0x7f7c100ffaf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c040095c0 tx=0x7f7c0400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.197+0000 7f7c0effd700 1 -- 172.21.15.130:0/1667684340 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0401d070 con 0x7f7c10068ae0 2023-09-12T16:02:49.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.197+0000 7f7c0effd700 1 -- 172.21.15.130:0/1667684340 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c0400fc00 con 0x7f7c10068ae0 2023-09-12T16:02:49.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.197+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c101a50f0 con 0x7f7c10068ae0 2023-09-12T16:02:49.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.197+0000 7f7c0effd700 1 -- 172.21.15.130:0/1667684340 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c040173f0 con 0x7f7c10068ae0 2023-09-12T16:02:49.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.197+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c101a5590 con 0x7f7c10068ae0 2023-09-12T16:02:49.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.198+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c10110930 con 0x7f7c10068ae0 2023-09-12T16:02:49.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.199+0000 7f7c0effd700 1 -- 172.21.15.130:0/1667684340 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f7c04022470 con 0x7f7c10068ae0 2023-09-12T16:02:49.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.199+0000 7f7c0effd700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7bf80a06d0 0x7f7bf80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.200+0000 7f7c15ecc700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7bf80a06d0 0x7f7bf80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.200+0000 7f7c0effd700 1 -- 172.21.15.130:0/1667684340 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7c040cd8d0 con 0x7f7c10068ae0 2023-09-12T16:02:49.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.200+0000 7f7c0effd700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] conn(0x7f7bf80ab250 0x7f7bf80ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.200+0000 7f7c0effd700 1 -- 172.21.15.130:0/1667684340 --> [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7bf80add60 con 0x7f7bf80ab250 2023-09-12T16:02:49.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.200+0000 7f7c0effd700 1 -- 172.21.15.130:0/1667684340 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c040cdce0 con 0x7f7c10068ae0 2023-09-12T16:02:49.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.200+0000 7f7c15ecc700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7bf80a06d0 0x7f7bf80a2b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c000094f0 tx=0x7f7c00009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.200+0000 7f7c166cd700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] conn(0x7f7bf80ab250 0x7f7bf80ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.200+0000 7f7c166cd700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] conn(0x7f7bf80ab250 0x7f7bf80ad690 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.201+0000 7f7c0effd700 1 -- 172.21.15.130:0/1667684340 <== osd.4 v2:172.21.15.159:6808/2237508615 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7bf80add60 con 0x7f7bf80ab250 2023-09-12T16:02:49.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.217+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 --> [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c1004e300 con 0x7f7bf80ab250 2023-09-12T16:02:49.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c0effd700 1 -- 172.21.15.130:0/1667684340 <== osd.4 v2:172.21.15.159:6808/2237508615 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7c1004e300 con 0x7f7bf80ab250 2023-09-12T16:02:49.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 >> [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] conn(0x7f7bf80ab250 msgr2=0x7f7bf80ad690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] conn(0x7f7bf80ab250 0x7f7bf80ad690 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7bf80a06d0 msgr2=0x7f7bf80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7bf80a06d0 0x7f7bf80a2b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c000094f0 tx=0x7f7c00009040 comp rx=0 tx=0).stop 2023-09-12T16:02:49.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c10068ae0 msgr2=0x7f7c100ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 --2- 172.21.15.130:0/1667684340 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c10068ae0 0x7f7c100ffaf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c040095c0 tx=0x7f7c0400f710 comp rx=0 tx=0).stop 2023-09-12T16:02:49.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c166cd700 1 -- 172.21.15.130:0/1667684340 reap_dead start 2023-09-12T16:02:49.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 shutdown_connections 2023-09-12T16:02:49.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 >> 172.21.15.130:0/1667684340 conn(0x7f7c10076600 msgr2=0x7f7c100fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:49.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 shutdown_connections 2023-09-12T16:02:49.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:48.218+0000 7f7c18130700 1 -- 172.21.15.130:0/1667684340 wait complete. 2023-09-12T16:02:49.189 INFO:teuthology.orchestra.run.smithi130.stdout:107374182453 2023-09-12T16:02:49.190 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.3 2023-09-12T16:02:49.234 INFO:teuthology.orchestra.run.smithi130.stdout:158913789997 2023-09-12T16:02:49.235 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.5 2023-09-12T16:02:49.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.325+0000 7efce2c2c700 1 -- 172.21.15.130:0/2543940261 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcdc105550 msgr2=0x7efcdc105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.325+0000 7efce2c2c700 1 --2- 172.21.15.130:0/2543940261 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcdc105550 0x7efcdc105930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efcd00099c0 tx=0x7efcd0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.325+0000 7efce2c2c700 1 -- 172.21.15.130:0/2543940261 shutdown_connections 2023-09-12T16:02:49.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.325+0000 7efce2c2c700 1 --2- 172.21.15.130:0/2543940261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcdc1006a0 0x7efcdc10c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.325+0000 7efce2c2c700 1 --2- 172.21.15.130:0/2543940261 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcdc105550 0x7efcdc105930 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.325+0000 7efce2c2c700 1 --2- 172.21.15.130:0/2543940261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcdc0ffce0 0x7efcdc100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.325+0000 7efce2c2c700 1 -- 172.21.15.130:0/2543940261 >> 172.21.15.130:0/2543940261 conn(0x7efcdc075a70 msgr2=0x7efcdc0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:49.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.325+0000 7efce2c2c700 1 -- 172.21.15.130:0/2543940261 shutdown_connections 2023-09-12T16:02:49.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.325+0000 7efce2c2c700 1 -- 172.21.15.130:0/2543940261 wait complete. 2023-09-12T16:02:49.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce2c2c700 1 Processor -- start 2023-09-12T16:02:49.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce2c2c700 1 -- start start 2023-09-12T16:02:49.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce2c2c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcdc0ffce0 0x7efcdc197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce2c2c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcdc1006a0 0x7efcdc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce2c2c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcdc105550 0x7efcdc1a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce2c2c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc1a2a10 con 0x7efcdc0ffce0 2023-09-12T16:02:49.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce2c2c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc1a2b50 con 0x7efcdc1006a0 2023-09-12T16:02:49.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce2c2c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc1a2c90 con 0x7efcdc105550 2023-09-12T16:02:49.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce09c8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcdc0ffce0 0x7efcdc197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce09c8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcdc0ffce0 0x7efcdc197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54000/0 (socket says 172.21.15.130:54000) 2023-09-12T16:02:49.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce09c8700 1 -- 172.21.15.130:0/624930543 learned_addr learned my addr 172.21.15.130:0/624930543 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:49.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efcdbfff700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcdc1006a0 0x7efcdc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce11c9700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcdc105550 0x7efcdc1a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce09c8700 1 -- 172.21.15.130:0/624930543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcdc105550 msgr2=0x7efcdc1a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce09c8700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcdc105550 0x7efcdc1a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce09c8700 1 -- 172.21.15.130:0/624930543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcdc1006a0 msgr2=0x7efcdc197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce09c8700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcdc1006a0 0x7efcdc197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce09c8700 1 -- 172.21.15.130:0/624930543 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd00095f0 con 0x7efcdc0ffce0 2023-09-12T16:02:49.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.326+0000 7efce11c9700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcdc105550 0x7efcdc1a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:49.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.327+0000 7efce09c8700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcdc0ffce0 0x7efcdc197450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efcd0005910 tx=0x7efcd00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.327+0000 7efcd9ffb700 1 -- 172.21.15.130:0/624930543 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd001d070 con 0x7efcdc0ffce0 2023-09-12T16:02:49.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.327+0000 7efcd9ffb700 1 -- 172.21.15.130:0/624930543 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd0004200 con 0x7efcdc0ffce0 2023-09-12T16:02:49.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.327+0000 7efcd9ffb700 1 -- 172.21.15.130:0/624930543 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd000f460 con 0x7efcdc0ffce0 2023-09-12T16:02:49.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.327+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc195300 con 0x7efcdc0ffce0 2023-09-12T16:02:49.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.327+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc195750 con 0x7efcdc0ffce0 2023-09-12T16:02:49.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.328+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efcdc10f750 con 0x7efcdc0ffce0 2023-09-12T16:02:49.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.330+0000 7efcd9ffb700 1 -- 172.21.15.130:0/624930543 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7efcd00226c0 con 0x7efcdc0ffce0 2023-09-12T16:02:49.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.331+0000 7efcd9ffb700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efccc0a06d0 0x7efccc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.331+0000 7efcdbfff700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efccc0a06d0 0x7efccc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.331+0000 7efcd9ffb700 1 -- 172.21.15.130:0/624930543 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efcd00cded0 con 0x7efcdc0ffce0 2023-09-12T16:02:49.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.331+0000 7efcd9ffb700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] conn(0x7efccc0ab250 0x7efccc0ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.331+0000 7efcd9ffb700 1 -- 172.21.15.130:0/624930543 --> [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efccc0add60 con 0x7efccc0ab250 2023-09-12T16:02:49.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.331+0000 7efcd9ffb700 1 -- 172.21.15.130:0/624930543 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcd00ce2e0 con 0x7efcdc0ffce0 2023-09-12T16:02:49.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.331+0000 7efcdbfff700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efccc0a06d0 0x7efccc0a2b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efcc8005fd0 tx=0x7efcc800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.331+0000 7efce11c9700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] conn(0x7efccc0ab250 0x7efccc0ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.332+0000 7efce11c9700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] conn(0x7efccc0ab250 0x7efccc0ad690 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.333+0000 7efcd9ffb700 1 -- 172.21.15.130:0/624930543 <== osd.7 v2:172.21.15.159:6816/315414716 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7efccc0add60 con 0x7efccc0ab250 2023-09-12T16:02:49.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.366+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 --> [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efcdc04e300 con 0x7efccc0ab250 2023-09-12T16:02:49.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efcd9ffb700 1 -- 172.21.15.130:0/624930543 <== osd.7 v2:172.21.15.159:6816/315414716 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efcdc04e300 con 0x7efccc0ab250 2023-09-12T16:02:49.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 >> [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] conn(0x7efccc0ab250 msgr2=0x7efccc0ad690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efce2c2c700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] conn(0x7efccc0ab250 0x7efccc0ad690 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efccc0a06d0 msgr2=0x7efccc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efce2c2c700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efccc0a06d0 0x7efccc0a2b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efcc8005fd0 tx=0x7efcc800a560 comp rx=0 tx=0).stop 2023-09-12T16:02:49.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcdc0ffce0 msgr2=0x7efcdc197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efce2c2c700 1 --2- 172.21.15.130:0/624930543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcdc0ffce0 0x7efcdc197450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efcd0005910 tx=0x7efcd00036f0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efce11c9700 1 -- 172.21.15.130:0/624930543 reap_dead start 2023-09-12T16:02:49.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 shutdown_connections 2023-09-12T16:02:49.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.367+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 >> 172.21.15.130:0/624930543 conn(0x7efcdc075a70 msgr2=0x7efcdc0fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:49.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.368+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 shutdown_connections 2023-09-12T16:02:49.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.368+0000 7efce2c2c700 1 -- 172.21.15.130:0/624930543 wait complete. 2023-09-12T16:02:49.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.162+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/791070530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdfd0104530 msgr2=0x7fdfd0104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.162+0000 7fdfd7c8a700 1 --2- 172.21.15.130:0/791070530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdfd0104530 0x7fdfd0104910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009990 tx=0x7fdfcc009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/791070530 shutdown_connections 2023-09-12T16:02:49.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 --2- 172.21.15.130:0/791070530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfd00ffee0 0x7fdfd010c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 --2- 172.21.15.130:0/791070530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdfd0104530 0x7fdfd0104910 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 --2- 172.21.15.130:0/791070530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd00ff520 0x7fdfd00ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/791070530 >> 172.21.15.130:0/791070530 conn(0x7fdfd0076600 msgr2=0x7fdfd0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:49.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/791070530 shutdown_connections 2023-09-12T16:02:49.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/791070530 wait complete. 2023-09-12T16:02:49.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 Processor -- start 2023-09-12T16:02:49.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 -- start start 2023-09-12T16:02:49.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdfd00ff520 0x7fdfd0195330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.163+0000 7fdfd7c8a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfd00ffee0 0x7fdfd0195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd7c8a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd0104530 0x7fdfd0195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd7c8a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd01a4b70 con 0x7fdfd00ff520 2023-09-12T16:02:49.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd7c8a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd01a4ce0 con 0x7fdfd0104530 2023-09-12T16:02:49.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd7c8a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd01a4ea0 con 0x7fdfd00ffee0 2023-09-12T16:02:49.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5a26700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdfd00ff520 0x7fdfd0195330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5a26700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdfd00ff520 0x7fdfd0195330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53984/0 (socket says 172.21.15.130:53984) 2023-09-12T16:02:49.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5a26700 1 -- 172.21.15.130:0/1979161921 learned_addr learned my addr 172.21.15.130:0/1979161921 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:49.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5225700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfd00ffee0 0x7fdfd0195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd6227700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd0104530 0x7fdfd0195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5a26700 1 -- 172.21.15.130:0/1979161921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfd00ffee0 msgr2=0x7fdfd0195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5a26700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfd00ffee0 0x7fdfd0195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5a26700 1 -- 172.21.15.130:0/1979161921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd0104530 msgr2=0x7fdfd0195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5a26700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd0104530 0x7fdfd0195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5a26700 1 -- 172.21.15.130:0/1979161921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc0095f0 con 0x7fdfd00ff520 2023-09-12T16:02:49.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd6227700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd0104530 0x7fdfd0195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:02:49.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd5a26700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdfd00ff520 0x7fdfd0195330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc004730 tx=0x7fdfcc004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfc6ffd700 1 -- 172.21.15.130:0/1979161921 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc01d070 con 0x7fdfd00ff520 2023-09-12T16:02:49.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd01a5120 con 0x7fdfd00ff520 2023-09-12T16:02:49.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfc6ffd700 1 -- 172.21.15.130:0/1979161921 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfcc004c90 con 0x7fdfd00ff520 2023-09-12T16:02:49.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfc6ffd700 1 -- 172.21.15.130:0/1979161921 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0054b0 con 0x7fdfd00ff520 2023-09-12T16:02:49.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.164+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd01a5530 con 0x7fdfd00ff520 2023-09-12T16:02:49.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.165+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdfd004e300 con 0x7fdfd00ff520 2023-09-12T16:02:49.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.166+0000 7fdfc6ffd700 1 -- 172.21.15.130:0/1979161921 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fdfcc00fbc0 con 0x7fdfd00ff520 2023-09-12T16:02:49.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.167+0000 7fdfc6ffd700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdfbc0a06d0 0x7fdfbc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.167+0000 7fdfd5225700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdfbc0a06d0 0x7fdfbc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.167+0000 7fdfc6ffd700 1 -- 172.21.15.130:0/1979161921 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdfcc0cd120 con 0x7fdfd00ff520 2023-09-12T16:02:49.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.167+0000 7fdfc6ffd700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] conn(0x7fdfbc0ab250 0x7fdfbc0ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.167+0000 7fdfc6ffd700 1 -- 172.21.15.130:0/1979161921 --> [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfbc0add60 con 0x7fdfbc0ab250 2023-09-12T16:02:49.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.167+0000 7fdfc6ffd700 1 -- 172.21.15.130:0/1979161921 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfcc0cf7a0 con 0x7fdfd00ff520 2023-09-12T16:02:49.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.168+0000 7fdfd5225700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdfbc0a06d0 0x7fdfbc0a2b90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0068a20 tx=0x7fdfc000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.168+0000 7fdfd6227700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] conn(0x7fdfbc0ab250 0x7fdfbc0ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.168+0000 7fdfd6227700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] conn(0x7fdfbc0ab250 0x7fdfbc0ad690 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.169+0000 7fdfc6ffd700 1 -- 172.21.15.130:0/1979161921 <== osd.11 v2:172.21.15.203:6824/768600250 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fdfbc0add60 con 0x7fdfbc0ab250 2023-09-12T16:02:49.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.190+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 --> [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdfd010f7a0 con 0x7fdfbc0ab250 2023-09-12T16:02:49.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.190+0000 7fdfc6ffd700 1 -- 172.21.15.130:0/1979161921 <== osd.11 v2:172.21.15.203:6824/768600250 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdfc8009410 con 0x7fdfbc0ab250 2023-09-12T16:02:49.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 >> [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] conn(0x7fdfbc0ab250 msgr2=0x7fdfbc0ad690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] conn(0x7fdfbc0ab250 0x7fdfbc0ad690 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdfbc0a06d0 msgr2=0x7fdfbc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdfbc0a06d0 0x7fdfbc0a2b90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0068a20 tx=0x7fdfc000a300 comp rx=0 tx=0).stop 2023-09-12T16:02:49.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdfd00ff520 msgr2=0x7fdfd0195330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 --2- 172.21.15.130:0/1979161921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdfd00ff520 0x7fdfd0195330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc004730 tx=0x7fdfcc004760 comp rx=0 tx=0).stop 2023-09-12T16:02:49.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd6227700 1 -- 172.21.15.130:0/1979161921 reap_dead start 2023-09-12T16:02:49.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 shutdown_connections 2023-09-12T16:02:49.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 >> 172.21.15.130:0/1979161921 conn(0x7fdfd0076600 msgr2=0x7fdfd00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:49.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 shutdown_connections 2023-09-12T16:02:49.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.191+0000 7fdfd7c8a700 1 -- 172.21.15.130:0/1979161921 wait complete. 2023-09-12T16:02:49.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.491+0000 7f914cbb5700 1 -- 172.21.15.130:0/706924550 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91481009f0 msgr2=0x7f914810c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.491+0000 7f914cbb5700 1 --2- 172.21.15.130:0/706924550 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91481009f0 0x7f914810c4c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f914000b0a0 tx=0x7f914000b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.491+0000 7f914cbb5700 1 -- 172.21.15.130:0/706924550 shutdown_connections 2023-09-12T16:02:49.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.491+0000 7f914cbb5700 1 --2- 172.21.15.130:0/706924550 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91481009f0 0x7f914810c4c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.491+0000 7f914cbb5700 1 --2- 172.21.15.130:0/706924550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9148105730 0x7f9148105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.491+0000 7f914cbb5700 1 --2- 172.21.15.130:0/706924550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9148100030 0x7f91481004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.491+0000 7f914cbb5700 1 -- 172.21.15.130:0/706924550 >> 172.21.15.130:0/706924550 conn(0x7f9148075a00 msgr2=0x7f9148075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:49.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.491+0000 7f914cbb5700 1 -- 172.21.15.130:0/706924550 shutdown_connections 2023-09-12T16:02:49.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.491+0000 7f914cbb5700 1 -- 172.21.15.130:0/706924550 wait complete. 2023-09-12T16:02:49.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.492+0000 7f914cbb5700 1 Processor -- start 2023-09-12T16:02:49.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.492+0000 7f914cbb5700 1 -- start start 2023-09-12T16:02:49.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.492+0000 7f914cbb5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9148100030 0x7f9148197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.492+0000 7f914cbb5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91481009f0 0x7f9148197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.492+0000 7f914cbb5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9148105730 0x7f91481a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914cbb5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9148198440 con 0x7f9148100030 2023-09-12T16:02:49.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914cbb5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481a2c00 con 0x7f91481009f0 2023-09-12T16:02:49.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914cbb5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481a2d40 con 0x7f9148105730 2023-09-12T16:02:49.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9148100030 0x7f9148197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9148100030 0x7f9148197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54016/0 (socket says 172.21.15.130:54016) 2023-09-12T16:02:49.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914659c700 1 -- 172.21.15.130:0/1700494310 learned_addr learned my addr 172.21.15.130:0/1700494310 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:49.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f9146d9d700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9148105730 0x7f91481a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f9145d9b700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91481009f0 0x7f9148197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914659c700 1 -- 172.21.15.130:0/1700494310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9148105730 msgr2=0x7f91481a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914659c700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9148105730 0x7f91481a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914659c700 1 -- 172.21.15.130:0/1700494310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91481009f0 msgr2=0x7f9148197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914659c700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91481009f0 0x7f9148197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914659c700 1 -- 172.21.15.130:0/1700494310 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9140009d00 con 0x7f9148100030 2023-09-12T16:02:49.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914659c700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9148100030 0x7f9148197640 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f913800d980 tx=0x7f913800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f91377fe700 1 -- 172.21.15.130:0/1700494310 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138004a30 con 0x7f9148100030 2023-09-12T16:02:49.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f91377fe700 1 -- 172.21.15.130:0/1700494310 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91380054e0 con 0x7f9148100030 2023-09-12T16:02:49.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91481a2fc0 con 0x7f9148100030 2023-09-12T16:02:49.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f91377fe700 1 -- 172.21.15.130:0/1700494310 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138009c90 con 0x7f9148100030 2023-09-12T16:02:49.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.493+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91481a3440 con 0x7f9148100030 2023-09-12T16:02:49.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.494+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f914810f7b0 con 0x7f9148100030 2023-09-12T16:02:49.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.496+0000 7f91377fe700 1 -- 172.21.15.130:0/1700494310 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f9138004c20 con 0x7f9148100030 2023-09-12T16:02:49.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.497+0000 7f91377fe700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91300a0750 0x7f91300a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.497+0000 7f91377fe700 1 -- 172.21.15.130:0/1700494310 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f91380cc330 con 0x7f9148100030 2023-09-12T16:02:49.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.497+0000 7f91377fe700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] conn(0x7f91300ab2d0 0x7f91300ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:49.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.497+0000 7f91377fe700 1 -- 172.21.15.130:0/1700494310 --> [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91300adde0 con 0x7f91300ab2d0 2023-09-12T16:02:49.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.497+0000 7f91377fe700 1 -- 172.21.15.130:0/1700494310 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91380cc6e0 con 0x7f9148100030 2023-09-12T16:02:49.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.497+0000 7f9145d9b700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91300a0750 0x7f91300a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.497+0000 7f9146d9d700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] conn(0x7f91300ab2d0 0x7f91300ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:49.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.498+0000 7f9145d9b700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91300a0750 0x7f91300a2c10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91480ff780 tx=0x7f913c00b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.498+0000 7f9146d9d700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] conn(0x7f91300ab2d0 0x7f91300ad710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:49.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.499+0000 7f91377fe700 1 -- 172.21.15.130:0/1700494310 <== osd.10 v2:172.21.15.159:6824/1888655979 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f91300adde0 con 0x7f91300ab2d0 2023-09-12T16:02:49.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.515+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 --> [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f914804e300 con 0x7f91300ab2d0 2023-09-12T16:02:49.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.515+0000 7f91377fe700 1 -- 172.21.15.130:0/1700494310 <== osd.10 v2:172.21.15.159:6824/1888655979 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f914804e300 con 0x7f91300ab2d0 2023-09-12T16:02:49.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 >> [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] conn(0x7f91300ab2d0 msgr2=0x7f91300ad710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] conn(0x7f91300ab2d0 0x7f91300ad710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91300a0750 msgr2=0x7f91300a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91300a0750 0x7f91300a2c10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91480ff780 tx=0x7f913c00b5b0 comp rx=0 tx=0).stop 2023-09-12T16:02:49.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9148100030 msgr2=0x7f9148197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:49.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 --2- 172.21.15.130:0/1700494310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9148100030 0x7f9148197640 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f913800d980 tx=0x7f913800dc90 comp rx=0 tx=0).stop 2023-09-12T16:02:49.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f9146d9d700 1 -- 172.21.15.130:0/1700494310 reap_dead start 2023-09-12T16:02:49.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 shutdown_connections 2023-09-12T16:02:49.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 >> 172.21.15.130:0/1700494310 conn(0x7f9148075a00 msgr2=0x7f91481064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:49.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 shutdown_connections 2023-09-12T16:02:49.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.516+0000 7f914cbb5700 1 -- 172.21.15.130:0/1700494310 wait complete. 2023-09-12T16:02:49.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:49 smithi130 ceph-mon[98670]: pgmap v285: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-09-12T16:02:50.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fab49800700 1 -- 172.21.15.130:0/2414055866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab441016a0 msgr2=0x7fab44101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fab49800700 1 --2- 172.21.15.130:0/2414055866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab441016a0 0x7fab44101a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fab340098f0 tx=0x7fab34009c00 comp rx=0 tx=0).stop 2023-09-12T16:02:50.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fab49800700 1 -- 172.21.15.130:0/2414055866 shutdown_connections 2023-09-12T16:02:50.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fab49800700 1 --2- 172.21.15.130:0/2414055866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab4410aa80 0x7fab4410cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fab49800700 1 --2- 172.21.15.130:0/2414055866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab441016a0 0x7fab44101a80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fab49800700 1 --2- 172.21.15.130:0/2414055866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab44101fc0 0x7fab4410a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fab49800700 1 -- 172.21.15.130:0/2414055866 >> 172.21.15.130:0/2414055866 conn(0x7fab440faeb0 msgr2=0x7fab440fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:50.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fab49800700 1 -- 172.21.15.130:0/2414055866 shutdown_connections 2023-09-12T16:02:50.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fab49800700 1 -- 172.21.15.130:0/2414055866 wait complete. 2023-09-12T16:02:50.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.968+0000 7fab49800700 1 Processor -- start 2023-09-12T16:02:50.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.968+0000 7fab49800700 1 -- start start 2023-09-12T16:02:50.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab49800700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab441016a0 0x7fab4419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab49800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab44101fc0 0x7fab4419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab49800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4410aa80 0x7fab44198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab49800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44198cf0 con 0x7fab44101fc0 2023-09-12T16:02:50.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab49800700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44198e60 con 0x7fab441016a0 2023-09-12T16:02:50.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab49800700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44199020 con 0x7fab4410aa80 2023-09-12T16:02:50.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab427fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab44101fc0 0x7fab4419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab427fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab44101fc0 0x7fab4419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54050/0 (socket says 172.21.15.130:54050) 2023-09-12T16:02:50.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab427fc700 1 -- 172.21.15.130:0/570914517 learned_addr learned my addr 172.21.15.130:0/570914517 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:50.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab437fe700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4410aa80 0x7fab44198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab42ffd700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab441016a0 0x7fab4419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab427fc700 1 -- 172.21.15.130:0/570914517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4410aa80 msgr2=0x7fab44198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab427fc700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab4410aa80 0x7fab44198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab427fc700 1 -- 172.21.15.130:0/570914517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab441016a0 msgr2=0x7fab4419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab427fc700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab441016a0 0x7fab4419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab427fc700 1 -- 172.21.15.130:0/570914517 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab34009520 con 0x7fab44101fc0 2023-09-12T16:02:50.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab427fc700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab44101fc0 0x7fab4419eb40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fab3800d980 tx=0x7fab3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:50.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fab2bfff700 1 -- 172.21.15.130:0/570914517 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800f890 con 0x7fab44101fc0 2023-09-12T16:02:50.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.970+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab44199300 con 0x7fab44101fc0 2023-09-12T16:02:50.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.970+0000 7fab2bfff700 1 -- 172.21.15.130:0/570914517 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab38010460 con 0x7fab44101fc0 2023-09-12T16:02:50.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.970+0000 7fab2bfff700 1 -- 172.21.15.130:0/570914517 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800e610 con 0x7fab44101fc0 2023-09-12T16:02:50.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.970+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab441a55f0 con 0x7fab44101fc0 2023-09-12T16:02:50.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.970+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab44110730 con 0x7fab44101fc0 2023-09-12T16:02:50.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.972+0000 7fab2bfff700 1 -- 172.21.15.130:0/570914517 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fab380105d0 con 0x7fab44101fc0 2023-09-12T16:02:50.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.972+0000 7fab2bfff700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fab2c0a06d0 0x7fab2c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.972+0000 7fab42ffd700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fab2c0a06d0 0x7fab2c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.973+0000 7fab2bfff700 1 -- 172.21.15.130:0/570914517 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fab380cbed0 con 0x7fab44101fc0 2023-09-12T16:02:50.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.973+0000 7fab2bfff700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] conn(0x7fab2c0ab250 0x7fab2c0ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.973+0000 7fab2bfff700 1 -- 172.21.15.130:0/570914517 --> [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab2c0add60 con 0x7fab2c0ab250 2023-09-12T16:02:50.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.973+0000 7fab2bfff700 1 -- 172.21.15.130:0/570914517 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab380cc2e0 con 0x7fab44101fc0 2023-09-12T16:02:50.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.973+0000 7fab437fe700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] conn(0x7fab2c0ab250 0x7fab2c0ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.973+0000 7fab42ffd700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fab2c0a06d0 0x7fab2c0a2b90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fab34005e90 tx=0x7fab34019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.973+0000 7fab437fe700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] conn(0x7fab2c0ab250 0x7fab2c0ad690 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.974+0000 7fab2bfff700 1 -- 172.21.15.130:0/570914517 <== osd.0 v2:172.21.15.130:6802/1767854979 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fab2c0add60 con 0x7fab2c0ab250 2023-09-12T16:02:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.992+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 --> [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab4404eb50 con 0x7fab2c0ab250 2023-09-12T16:02:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.992+0000 7fab2bfff700 1 -- 172.21.15.130:0/570914517 <== osd.0 v2:172.21.15.130:6802/1767854979 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab4404eb50 con 0x7fab2c0ab250 2023-09-12T16:02:50.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.992+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] conn(0x7fab2c0ab250 msgr2=0x7fab2c0ad690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.992+0000 7fab49800700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] conn(0x7fab2c0ab250 0x7fab2c0ad690 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.992+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fab2c0a06d0 msgr2=0x7fab2c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.992+0000 7fab49800700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fab2c0a06d0 0x7fab2c0a2b90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fab34005e90 tx=0x7fab34019040 comp rx=0 tx=0).stop 2023-09-12T16:02:50.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.992+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab44101fc0 msgr2=0x7fab4419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.992+0000 7fab49800700 1 --2- 172.21.15.130:0/570914517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab44101fc0 0x7fab4419eb40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fab3800d980 tx=0x7fab3800dc90 comp rx=0 tx=0).stop 2023-09-12T16:02:50.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.993+0000 7fab437fe700 1 -- 172.21.15.130:0/570914517 reap_dead start 2023-09-12T16:02:50.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.993+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 shutdown_connections 2023-09-12T16:02:50.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.993+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 >> 172.21.15.130:0/570914517 conn(0x7fab440faeb0 msgr2=0x7fab4410cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:50.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.993+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 shutdown_connections 2023-09-12T16:02:50.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.993+0000 7fab49800700 1 -- 172.21.15.130:0/570914517 wait complete. 2023-09-12T16:02:50.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.937+0000 7fe8baba4700 1 -- 172.21.15.130:0/2216353652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b4104530 msgr2=0x7fe8b4104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.937+0000 7fe8baba4700 1 --2- 172.21.15.130:0/2216353652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b4104530 0x7fe8b4104910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009990 tx=0x7fe8a8009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.937+0000 7fe8baba4700 1 -- 172.21.15.130:0/2216353652 shutdown_connections 2023-09-12T16:02:50.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.937+0000 7fe8baba4700 1 --2- 172.21.15.130:0/2216353652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b40ffee0 0x7fe8b410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.937+0000 7fe8baba4700 1 --2- 172.21.15.130:0/2216353652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b4104530 0x7fe8b4104910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.937+0000 7fe8baba4700 1 --2- 172.21.15.130:0/2216353652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8b40ff520 0x7fe8b40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.937+0000 7fe8baba4700 1 -- 172.21.15.130:0/2216353652 >> 172.21.15.130:0/2216353652 conn(0x7fe8b4076600 msgr2=0x7fe8b4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:50.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.937+0000 7fe8baba4700 1 -- 172.21.15.130:0/2216353652 shutdown_connections 2023-09-12T16:02:50.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.937+0000 7fe8baba4700 1 -- 172.21.15.130:0/2216353652 wait complete. 2023-09-12T16:02:50.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.938+0000 7fe8baba4700 1 Processor -- start 2023-09-12T16:02:50.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.938+0000 7fe8baba4700 1 -- start start 2023-09-12T16:02:50.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.938+0000 7fe8baba4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8b40ff520 0x7fe8b419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.938+0000 7fe8baba4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b40ffee0 0x7fe8b419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.938+0000 7fe8baba4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b4104530 0x7fe8b41965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.938+0000 7fe8baba4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4196c50 con 0x7fe8b4104530 2023-09-12T16:02:50.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.938+0000 7fe8baba4700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4196dc0 con 0x7fe8b40ffee0 2023-09-12T16:02:50.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.938+0000 7fe8baba4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4196f80 con 0x7fe8b40ff520 2023-09-12T16:02:50.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b9141700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b4104530 0x7fe8b41965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b8940700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8b40ff520 0x7fe8b419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b9141700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b4104530 0x7fe8b41965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54036/0 (socket says 172.21.15.130:54036) 2023-09-12T16:02:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b9141700 1 -- 172.21.15.130:0/3663844553 learned_addr learned my addr 172.21.15.130:0/3663844553 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b3fff700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b40ffee0 0x7fe8b419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b9141700 1 -- 172.21.15.130:0/3663844553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8b40ff520 msgr2=0x7fe8b419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b9141700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8b40ff520 0x7fe8b419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b9141700 1 -- 172.21.15.130:0/3663844553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b40ffee0 msgr2=0x7fe8b419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b9141700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b40ffee0 0x7fe8b419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b9141700 1 -- 172.21.15.130:0/3663844553 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a80095f0 con 0x7fe8b4104530 2023-09-12T16:02:50.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b9141700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b4104530 0x7fe8b41965e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00efc0 tx=0x7fe8ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:50.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b1ffb700 1 -- 172.21.15.130:0/3663844553 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac018070 con 0x7fe8b4104530 2023-09-12T16:02:50.125 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b4197200 con 0x7fe8b4104530 2023-09-12T16:02:50.125 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b1ffb700 1 -- 172.21.15.130:0/3663844553 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8ac00f040 con 0x7fe8b4104530 2023-09-12T16:02:50.125 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.939+0000 7fe8b1ffb700 1 -- 172.21.15.130:0/3663844553 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00cc10 con 0x7fe8b4104530 2023-09-12T16:02:50.125 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.940+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b41a3550 con 0x7fe8b4104530 2023-09-12T16:02:50.125 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.940+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8b410f760 con 0x7fe8b4104530 2023-09-12T16:02:50.125 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.942+0000 7fe8b1ffb700 1 -- 172.21.15.130:0/3663844553 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fe8ac00cd70 con 0x7fe8b4104530 2023-09-12T16:02:50.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.942+0000 7fe8b1ffb700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe8a40a06d0 0x7fe8a40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.942+0000 7fe8b1ffb700 1 -- 172.21.15.130:0/3663844553 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe8ac0cc1b0 con 0x7fe8b4104530 2023-09-12T16:02:50.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.942+0000 7fe8b8940700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe8a40a06d0 0x7fe8a40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.942+0000 7fe8b1ffb700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] conn(0x7fe8a40ab1d0 0x7fe8a40ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.942+0000 7fe8b1ffb700 1 -- 172.21.15.130:0/3663844553 --> [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8a40adce0 con 0x7fe8a40ab1d0 2023-09-12T16:02:50.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.942+0000 7fe8b1ffb700 1 -- 172.21.15.130:0/3663844553 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8ac0cc5c0 con 0x7fe8b4104530 2023-09-12T16:02:50.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.943+0000 7fe8b8940700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe8a40a06d0 0x7fe8a40a2b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c00 tx=0x7fe8a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:50.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.943+0000 7fe8b3fff700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] conn(0x7fe8a40ab1d0 0x7fe8a40ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.943+0000 7fe8b3fff700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] conn(0x7fe8a40ab1d0 0x7fe8a40ad610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:50.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.944+0000 7fe8b1ffb700 1 -- 172.21.15.130:0/3663844553 <== osd.2 v2:172.21.15.203:6800/3797136832 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe8a40adce0 con 0x7fe8a40ab1d0 2023-09-12T16:02:50.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.967+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 --> [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8b404e300 con 0x7fe8a40ab1d0 2023-09-12T16:02:50.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.968+0000 7fe8b1ffb700 1 -- 172.21.15.130:0/3663844553 <== osd.2 v2:172.21.15.203:6800/3797136832 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe8b404e300 con 0x7fe8a40ab1d0 2023-09-12T16:02:50.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.968+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 >> [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] conn(0x7fe8a40ab1d0 msgr2=0x7fe8a40ad610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.968+0000 7fe8baba4700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] conn(0x7fe8a40ab1d0 0x7fe8a40ad610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.968+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe8a40a06d0 msgr2=0x7fe8a40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.968+0000 7fe8baba4700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe8a40a06d0 0x7fe8a40a2b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c00 tx=0x7fe8a8005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.968+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b4104530 msgr2=0x7fe8b41965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.968+0000 7fe8baba4700 1 --2- 172.21.15.130:0/3663844553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b4104530 0x7fe8b41965e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00efc0 tx=0x7fe8ac00c3a0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fe8b9141700 1 -- 172.21.15.130:0/3663844553 reap_dead start 2023-09-12T16:02:50.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 shutdown_connections 2023-09-12T16:02:50.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 >> 172.21.15.130:0/3663844553 conn(0x7fe8b4076600 msgr2=0x7fe8b4068160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:50.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.969+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 shutdown_connections 2023-09-12T16:02:50.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:49.970+0000 7fe8baba4700 1 -- 172.21.15.130:0/3663844553 wait complete. 2023-09-12T16:02:50.223 INFO:teuthology.orchestra.run.smithi130.stdout:137438953522 2023-09-12T16:02:50.224 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.4 2023-09-12T16:02:50.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:49 smithi159 ceph-mon[102522]: pgmap v285: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-09-12T16:02:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:49 smithi203 ceph-mon[103431]: pgmap v285: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-09-12T16:02:50.611 INFO:teuthology.orchestra.run.smithi130.stdout:326417514517 2023-09-12T16:02:50.611 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.11 2023-09-12T16:02:50.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.541+0000 7f0a9a5be700 1 -- 172.21.15.130:0/2697975426 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a940ff520 msgr2=0x7f0a940ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.541+0000 7f0a9a5be700 1 --2- 172.21.15.130:0/2697975426 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a940ff520 0x7f0a940ff9a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009990 tx=0x7f0a7c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.541+0000 7f0a9a5be700 1 -- 172.21.15.130:0/2697975426 shutdown_connections 2023-09-12T16:02:50.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.541+0000 7f0a9a5be700 1 --2- 172.21.15.130:0/2697975426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a940ffee0 0x7f0a9410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.541+0000 7f0a9a5be700 1 --2- 172.21.15.130:0/2697975426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a94104530 0x7f0a94104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.541+0000 7f0a9a5be700 1 --2- 172.21.15.130:0/2697975426 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a940ff520 0x7f0a940ff9a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.541+0000 7f0a9a5be700 1 -- 172.21.15.130:0/2697975426 >> 172.21.15.130:0/2697975426 conn(0x7f0a94076600 msgr2=0x7f0a94076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:50.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.541+0000 7f0a9a5be700 1 -- 172.21.15.130:0/2697975426 shutdown_connections 2023-09-12T16:02:50.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.541+0000 7f0a9a5be700 1 -- 172.21.15.130:0/2697975426 wait complete. 2023-09-12T16:02:50.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a9a5be700 1 Processor -- start 2023-09-12T16:02:50.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a9a5be700 1 -- start start 2023-09-12T16:02:50.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a9a5be700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a940ff520 0x7f0a94195570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a9a5be700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a940ffee0 0x7f0a94195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a9a5be700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a94104530 0x7f0a94199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a9a5be700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9419a530 con 0x7f0a940ff520 2023-09-12T16:02:50.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a9a5be700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9419a6a0 con 0x7f0a94104530 2023-09-12T16:02:50.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a9a5be700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9419a860 con 0x7f0a940ffee0 2023-09-12T16:02:50.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a940ff520 0x7f0a94195570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a940ff520 0x7f0a94195570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54064/0 (socket says 172.21.15.130:54064) 2023-09-12T16:02:50.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.542+0000 7f0a93fff700 1 -- 172.21.15.130:0/3228484168 learned_addr learned my addr 172.21.15.130:0/3228484168 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:50.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a98b5b700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a94104530 0x7f0a94199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a937fe700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a940ffee0 0x7f0a94195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a93fff700 1 -- 172.21.15.130:0/3228484168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a940ffee0 msgr2=0x7f0a94195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a93fff700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a940ffee0 0x7f0a94195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a93fff700 1 -- 172.21.15.130:0/3228484168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a94104530 msgr2=0x7f0a94199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a93fff700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a94104530 0x7f0a94199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a93fff700 1 -- 172.21.15.130:0/3228484168 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c0095f0 con 0x7f0a940ff520 2023-09-12T16:02:50.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a937fe700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a940ffee0 0x7f0a94195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:50.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a93fff700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a940ff520 0x7f0a94195570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400ea80 tx=0x7f0a8400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:50.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a917fa700 1 -- 172.21.15.130:0/3228484168 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84013070 con 0x7f0a940ff520 2023-09-12T16:02:50.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9419ab40 con 0x7f0a940ff520 2023-09-12T16:02:50.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a917fa700 1 -- 172.21.15.130:0/3228484168 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a8400be80 con 0x7f0a940ff520 2023-09-12T16:02:50.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a917fa700 1 -- 172.21.15.130:0/3228484168 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84003ae0 con 0x7f0a940ff520 2023-09-12T16:02:50.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.543+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a94068290 con 0x7f0a940ff520 2023-09-12T16:02:50.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.544+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a9404e300 con 0x7f0a940ff520 2023-09-12T16:02:50.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.545+0000 7f0a917fa700 1 -- 172.21.15.130:0/3228484168 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f0a840041b0 con 0x7f0a940ff520 2023-09-12T16:02:50.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.546+0000 7f0a917fa700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a800a06d0 0x7f0a800a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.546+0000 7f0a937fe700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a800a06d0 0x7f0a800a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.546+0000 7f0a917fa700 1 -- 172.21.15.130:0/3228484168 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0a840ce470 con 0x7f0a940ff520 2023-09-12T16:02:50.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.546+0000 7f0a917fa700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] conn(0x7f0a800ab250 0x7f0a800ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:50.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.546+0000 7f0a917fa700 1 -- 172.21.15.130:0/3228484168 --> [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a800add60 con 0x7f0a800ab250 2023-09-12T16:02:50.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.546+0000 7f0a917fa700 1 -- 172.21.15.130:0/3228484168 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a840ce880 con 0x7f0a940ff520 2023-09-12T16:02:50.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.546+0000 7f0a937fe700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a800a06d0 0x7f0a800a2b90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00b2e0 tx=0x7f0a7c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:50.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.546+0000 7f0a98b5b700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] conn(0x7f0a800ab250 0x7f0a800ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:50.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.547+0000 7f0a98b5b700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] conn(0x7f0a800ab250 0x7f0a800ad690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:50.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.548+0000 7f0a917fa700 1 -- 172.21.15.130:0/3228484168 <== osd.8 v2:172.21.15.203:6816/2706023432 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0a800add60 con 0x7f0a800ab250 2023-09-12T16:02:50.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.563+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 --> [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a9410f750 con 0x7f0a800ab250 2023-09-12T16:02:50.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.564+0000 7f0a917fa700 1 -- 172.21.15.130:0/3228484168 <== osd.8 v2:172.21.15.203:6816/2706023432 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0a8c00f040 con 0x7f0a800ab250 2023-09-12T16:02:50.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.564+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 >> [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] conn(0x7f0a800ab250 msgr2=0x7f0a800ad690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.564+0000 7f0a9a5be700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] conn(0x7f0a800ab250 0x7f0a800ad690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.564+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a800a06d0 msgr2=0x7f0a800a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.564+0000 7f0a9a5be700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0a800a06d0 0x7f0a800a2b90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00b2e0 tx=0x7f0a7c005fb0 comp rx=0 tx=0).stop 2023-09-12T16:02:50.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.564+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a940ff520 msgr2=0x7f0a94195570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:50.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.564+0000 7f0a9a5be700 1 --2- 172.21.15.130:0/3228484168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a940ff520 0x7f0a94195570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400ea80 tx=0x7f0a8400ed90 comp rx=0 tx=0).stop 2023-09-12T16:02:50.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.565+0000 7f0a98b5b700 1 -- 172.21.15.130:0/3228484168 reap_dead start 2023-09-12T16:02:50.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.565+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 shutdown_connections 2023-09-12T16:02:50.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.565+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 >> 172.21.15.130:0/3228484168 conn(0x7f0a94076600 msgr2=0x7f0a940fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:50.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.565+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 shutdown_connections 2023-09-12T16:02:50.773 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.565+0000 7f0a9a5be700 1 -- 172.21.15.130:0/3228484168 wait complete. 2023-09-12T16:02:50.887 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-09-12T16:02:50.888 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-09-12T16:02:50.888 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-09-12T16:02:50.888 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-09-12T16:02:50.888 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-09-12T16:02:50.888 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-09-12T16:02:50.889 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2023-09-12T16:02:50.889 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-09-12T16:02:50.889 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2023-09-12T16:02:50.889 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-09-12T16:02:51.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-09-12T16:02:51.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-09-12T16:02:51.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-09-12T16:02:51.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-09-12T16:02:51.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-09-12T16:02:51.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-09-12T16:02:51.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2023-09-12T16:02:51.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-09-12T16:02:51.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2023-09-12T16:02:51.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-09-12T16:02:51.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-09-12T16:02:51.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-09-12T16:02:51.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-09-12T16:02:51.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-09-12T16:02:51.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-09-12T16:02:51.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-09-12T16:02:51.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2023-09-12T16:02:51.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-09-12T16:02:51.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2023-09-12T16:02:51.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-09-12T16:02:51.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.961+0000 7ff9b62b9700 1 -- 172.21.15.130:0/3443635078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9b0068ae0 msgr2=0x7ff9b010d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:51.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.961+0000 7ff9b62b9700 1 --2- 172.21.15.130:0/3443635078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9b0068ae0 0x7ff9b010d440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b0a0 tx=0x7ff9a800b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:51.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.961+0000 7ff9b62b9700 1 -- 172.21.15.130:0/3443635078 shutdown_connections 2023-09-12T16:02:51.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.961+0000 7ff9b62b9700 1 --2- 172.21.15.130:0/3443635078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9b0068ae0 0x7ff9b010d440 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:51.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.961+0000 7ff9b62b9700 1 --2- 172.21.15.130:0/3443635078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b0106770 0x7ff9b0106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:51.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.961+0000 7ff9b62b9700 1 --2- 172.21.15.130:0/3443635078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b0068120 0x7ff9b00685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:51.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.961+0000 7ff9b62b9700 1 -- 172.21.15.130:0/3443635078 >> 172.21.15.130:0/3443635078 conn(0x7ff9b0076600 msgr2=0x7ff9b0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:51.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.961+0000 7ff9b62b9700 1 -- 172.21.15.130:0/3443635078 shutdown_connections 2023-09-12T16:02:51.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.961+0000 7ff9b62b9700 1 -- 172.21.15.130:0/3443635078 wait complete. 2023-09-12T16:02:51.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.962+0000 7ff9b62b9700 1 Processor -- start 2023-09-12T16:02:51.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.962+0000 7ff9b62b9700 1 -- start start 2023-09-12T16:02:51.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.962+0000 7ff9b62b9700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b0068120 0x7ff9b019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:51.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.962+0000 7ff9b62b9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b0068ae0 0x7ff9b019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:51.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.962+0000 7ff9b62b9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9b0106770 0x7ff9b01966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:51.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.962+0000 7ff9b62b9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0196d30 con 0x7ff9b0106770 2023-09-12T16:02:51.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.962+0000 7ff9b62b9700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0196ea0 con 0x7ff9b0068120 2023-09-12T16:02:51.373 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.962+0000 7ff9b62b9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0197060 con 0x7ff9b0068ae0 2023-09-12T16:02:51.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b4856700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9b0106770 0x7ff9b01966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:51.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9affff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b0068120 0x7ff9b019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:51.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b4856700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9b0106770 0x7ff9b01966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54084/0 (socket says 172.21.15.130:54084) 2023-09-12T16:02:51.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9af7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b0068ae0 0x7ff9b019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:51.377 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b4856700 1 -- 172.21.15.130:0/1169569480 learned_addr learned my addr 172.21.15.130:0/1169569480 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:51.378 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b4856700 1 -- 172.21.15.130:0/1169569480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b0068ae0 msgr2=0x7ff9b019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:51.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b4856700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b0068ae0 0x7ff9b019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:51.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b4856700 1 -- 172.21.15.130:0/1169569480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b0068120 msgr2=0x7ff9b019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:51.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b4856700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b0068120 0x7ff9b019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:51.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b4856700 1 -- 172.21.15.130:0/1169569480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a8009d00 con 0x7ff9b0106770 2023-09-12T16:02:51.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b4856700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9b0106770 0x7ff9b01966c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b780 tx=0x7ff9a8007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:51.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9ad7fa700 1 -- 172.21.15.130:0/1169569480 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a800e040 con 0x7ff9b0106770 2023-09-12T16:02:51.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b01972e0 con 0x7ff9b0106770 2023-09-12T16:02:51.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9ad7fa700 1 -- 172.21.15.130:0/1169569480 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9a800be40 con 0x7ff9b0106770 2023-09-12T16:02:51.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9ad7fa700 1 -- 172.21.15.130:0/1169569480 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80052a0 con 0x7ff9b0106770 2023-09-12T16:02:51.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.963+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b01a3630 con 0x7ff9b0106770 2023-09-12T16:02:51.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.964+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9b0110930 con 0x7ff9b0106770 2023-09-12T16:02:51.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.965+0000 7ff9ad7fa700 1 -- 172.21.15.130:0/1169569480 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7ff9a801d470 con 0x7ff9b0106770 2023-09-12T16:02:51.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.966+0000 7ff9ad7fa700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9980a06d0 0x7ff9980a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:51.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.966+0000 7ff9affff700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9980a06d0 0x7ff9980a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:51.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.966+0000 7ff9ad7fa700 1 -- 172.21.15.130:0/1169569480 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff9a80cdd10 con 0x7ff9b0106770 2023-09-12T16:02:51.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.966+0000 7ff9ad7fa700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] conn(0x7ff9980ab6f0 0x7ff9980adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:51.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.966+0000 7ff9ad7fa700 1 -- 172.21.15.130:0/1169569480 --> [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9980ae260 con 0x7ff9980ab6f0 2023-09-12T16:02:51.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.966+0000 7ff9ad7fa700 1 -- 172.21.15.130:0/1169569480 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9a80ce120 con 0x7ff9b0106770 2023-09-12T16:02:51.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.966+0000 7ff9af7fe700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] conn(0x7ff9980ab6f0 0x7ff9980adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:51.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.966+0000 7ff9affff700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9980a06d0 0x7ff9980a2b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009be0 tx=0x7ff9a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:51.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.966+0000 7ff9af7fe700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] conn(0x7ff9980ab6f0 0x7ff9980adb90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:51.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.967+0000 7ff9ad7fa700 1 -- 172.21.15.130:0/1169569480 <== osd.9 v2:172.21.15.130:6826/715792473 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff9980ae260 con 0x7ff9980ab6f0 2023-09-12T16:02:51.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.983+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 --> [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9b004e300 con 0x7ff9980ab6f0 2023-09-12T16:02:51.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.983+0000 7ff9ad7fa700 1 -- 172.21.15.130:0/1169569480 <== osd.9 v2:172.21.15.130:6826/715792473 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff9b004e300 con 0x7ff9980ab6f0 2023-09-12T16:02:51.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] conn(0x7ff9980ab6f0 msgr2=0x7ff9980adb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:51.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] conn(0x7ff9980ab6f0 0x7ff9980adb90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:51.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9980a06d0 msgr2=0x7ff9980a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:51.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9980a06d0 0x7ff9980a2b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009be0 tx=0x7ff9a0009260 comp rx=0 tx=0).stop 2023-09-12T16:02:51.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9b0106770 msgr2=0x7ff9b01966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:51.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 --2- 172.21.15.130:0/1169569480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9b0106770 0x7ff9b01966c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b780 tx=0x7ff9a8007990 comp rx=0 tx=0).stop 2023-09-12T16:02:51.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b4856700 1 -- 172.21.15.130:0/1169569480 reap_dead start 2023-09-12T16:02:51.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 shutdown_connections 2023-09-12T16:02:51.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 >> 172.21.15.130:0/1169569480 conn(0x7ff9b0076600 msgr2=0x7ff9b00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:51.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 shutdown_connections 2023-09-12T16:02:51.386 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:50.984+0000 7ff9b62b9700 1 -- 172.21.15.130:0/1169569480 wait complete. 2023-09-12T16:02:51.576 INFO:teuthology.orchestra.run.smithi130.stdout:77309411385 2023-09-12T16:02:51.577 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.2 2023-09-12T16:02:51.820 INFO:teuthology.orchestra.run.smithi130.stdout:292057776151 2023-09-12T16:02:51.821 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.10 2023-09-12T16:02:51.842 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: pgmap v286: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-09-12T16:02:51.842 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-09-12T16:02:51.842 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-09-12T16:02:51.843 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-09-12T16:02:51.843 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-09-12T16:02:51.843 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-09-12T16:02:51.843 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-09-12T16:02:51.843 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2023-09-12T16:02:51.844 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-09-12T16:02:51.844 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2023-09-12T16:02:51.844 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-09-12T16:02:51.844 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:51 smithi130 ceph-mon[98670]: osdmap e87: 12 total, 12 up, 12 in 2023-09-12T16:02:51.864 INFO:teuthology.orchestra.run.smithi130.stdout:210453397541 2023-09-12T16:02:51.864 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.7 2023-09-12T16:02:52.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.006+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3698562482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7bc0ff520 msgr2=0x7fe7bc0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:52.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.006+0000 7fe7c3b78700 1 --2- 172.21.15.130:0/3698562482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7bc0ff520 0x7fe7bc0ff9a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009990 tx=0x7fe7ac009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:52.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.007+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3698562482 shutdown_connections 2023-09-12T16:02:52.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.007+0000 7fe7c3b78700 1 --2- 172.21.15.130:0/3698562482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7bc0ffee0 0x7fe7bc10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:52.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.007+0000 7fe7c3b78700 1 --2- 172.21.15.130:0/3698562482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7bc104530 0x7fe7bc104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:52.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.007+0000 7fe7c3b78700 1 --2- 172.21.15.130:0/3698562482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7bc0ff520 0x7fe7bc0ff9a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:52.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.007+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3698562482 >> 172.21.15.130:0/3698562482 conn(0x7fe7bc076600 msgr2=0x7fe7bc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:52.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.007+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3698562482 shutdown_connections 2023-09-12T16:02:52.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.007+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3698562482 wait complete. 2023-09-12T16:02:52.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c3b78700 1 Processor -- start 2023-09-12T16:02:52.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c3b78700 1 -- start start 2023-09-12T16:02:52.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c3b78700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7bc0ff520 0x7fe7bc195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:52.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c3b78700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7bc0ffee0 0x7fe7bc1957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:52.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c3b78700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7bc104530 0x7fe7bc195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:52.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c3b78700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc1a4ad0 con 0x7fe7bc104530 2023-09-12T16:02:52.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c3b78700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc1a4c40 con 0x7fe7bc0ff520 2023-09-12T16:02:52.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c3b78700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc1a4e00 con 0x7fe7bc0ffee0 2023-09-12T16:02:52.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c2115700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7bc104530 0x7fe7bc195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:52.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c2115700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7bc104530 0x7fe7bc195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54100/0 (socket says 172.21.15.130:54100) 2023-09-12T16:02:52.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c2115700 1 -- 172.21.15.130:0/3298517593 learned_addr learned my addr 172.21.15.130:0/3298517593 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:52.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c1914700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7bc0ff520 0x7fe7bc195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:52.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c1113700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7bc0ffee0 0x7fe7bc1957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:52.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c2115700 1 -- 172.21.15.130:0/3298517593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7bc0ffee0 msgr2=0x7fe7bc1957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:52.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c2115700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7bc0ffee0 0x7fe7bc1957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:52.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c2115700 1 -- 172.21.15.130:0/3298517593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7bc0ff520 msgr2=0x7fe7bc195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:52.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c2115700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7bc0ff520 0x7fe7bc195290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:52.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.008+0000 7fe7c2115700 1 -- 172.21.15.130:0/3298517593 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac0095f0 con 0x7fe7bc104530 2023-09-12T16:02:52.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.009+0000 7fe7c2115700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7bc104530 0x7fe7bc195d10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400efc0 tx=0x7fe7b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:52.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.009+0000 7fe7b2ffd700 1 -- 172.21.15.130:0/3298517593 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4018070 con 0x7fe7bc104530 2023-09-12T16:02:52.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.009+0000 7fe7b2ffd700 1 -- 172.21.15.130:0/3298517593 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7b400f040 con 0x7fe7bc104530 2023-09-12T16:02:52.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.009+0000 7fe7b2ffd700 1 -- 172.21.15.130:0/3298517593 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400cc10 con 0x7fe7bc104530 2023-09-12T16:02:52.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.009+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc1a5080 con 0x7fe7bc104530 2023-09-12T16:02:52.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.009+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc1a5530 con 0x7fe7bc104530 2023-09-12T16:02:52.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.010+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7bc04e300 con 0x7fe7bc104530 2023-09-12T16:02:52.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.012+0000 7fe7b2ffd700 1 -- 172.21.15.130:0/3298517593 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fe7b400cd90 con 0x7fe7bc104530 2023-09-12T16:02:52.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.012+0000 7fe7b2ffd700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe7a80a06d0 0x7fe7a80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:52.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.013+0000 7fe7b2ffd700 1 -- 172.21.15.130:0/3298517593 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe7b40cc150 con 0x7fe7bc104530 2023-09-12T16:02:52.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.013+0000 7fe7b2ffd700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] conn(0x7fe7a80ab6f0 0x7fe7a80adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:52.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.013+0000 7fe7b2ffd700 1 -- 172.21.15.130:0/3298517593 --> [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7a80ae260 con 0x7fe7a80ab6f0 2023-09-12T16:02:52.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.013+0000 7fe7b2ffd700 1 -- 172.21.15.130:0/3298517593 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7b40cc560 con 0x7fe7bc104530 2023-09-12T16:02:52.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.013+0000 7fe7c1914700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe7a80a06d0 0x7fe7a80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:52.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.013+0000 7fe7c1113700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] conn(0x7fe7a80ab6f0 0x7fe7a80adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:52.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.014+0000 7fe7c1113700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] conn(0x7fe7a80ab6f0 0x7fe7a80adb90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:52.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.014+0000 7fe7c1914700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe7a80a06d0 0x7fe7a80a2b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc069c10 tx=0x7fe7b800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:52.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.014+0000 7fe7b2ffd700 1 -- 172.21.15.130:0/3298517593 <== osd.1 v2:172.21.15.159:6800/1684620878 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe7a80ae260 con 0x7fe7a80ab6f0 2023-09-12T16:02:52.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.048+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 --> [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7bc10f750 con 0x7fe7a80ab6f0 2023-09-12T16:02:52.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.048+0000 7fe7b2ffd700 1 -- 172.21.15.130:0/3298517593 <== osd.1 v2:172.21.15.159:6800/1684620878 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe7ac01e040 con 0x7fe7a80ab6f0 2023-09-12T16:02:52.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 >> [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] conn(0x7fe7a80ab6f0 msgr2=0x7fe7a80adb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:52.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] conn(0x7fe7a80ab6f0 0x7fe7a80adb90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:52.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe7a80a06d0 msgr2=0x7fe7a80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:52.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe7a80a06d0 0x7fe7a80a2b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc069c10 tx=0x7fe7b800b410 comp rx=0 tx=0).stop 2023-09-12T16:02:52.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7bc104530 msgr2=0x7fe7bc195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:52.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 --2- 172.21.15.130:0/3298517593 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7bc104530 0x7fe7bc195d10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400efc0 tx=0x7fe7b400c3a0 comp rx=0 tx=0).stop 2023-09-12T16:02:52.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c2115700 1 -- 172.21.15.130:0/3298517593 reap_dead start 2023-09-12T16:02:52.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 shutdown_connections 2023-09-12T16:02:52.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 >> 172.21.15.130:0/3298517593 conn(0x7fe7bc076600 msgr2=0x7fe7bc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:52.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 shutdown_connections 2023-09-12T16:02:52.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:52.049+0000 7fe7c3b78700 1 -- 172.21.15.130:0/3298517593 wait complete. 2023-09-12T16:02:52.185 INFO:teuthology.orchestra.run.smithi130.stdout:240518168609 2023-09-12T16:02:52.186 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.8 2023-09-12T16:02:52.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: pgmap v286: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-09-12T16:02:52.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-09-12T16:02:52.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-09-12T16:02:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-09-12T16:02:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-09-12T16:02:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-09-12T16:02:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-09-12T16:02:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2023-09-12T16:02:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-09-12T16:02:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2023-09-12T16:02:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-09-12T16:02:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:51 smithi159 ceph-mon[102522]: osdmap e87: 12 total, 12 up, 12 in 2023-09-12T16:02:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: pgmap v286: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-09-12T16:02:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-09-12T16:02:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-09-12T16:02:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-09-12T16:02:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-09-12T16:02:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-09-12T16:02:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-09-12T16:02:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2023-09-12T16:02:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-09-12T16:02:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2023-09-12T16:02:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-09-12T16:02:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:51 smithi203 ceph-mon[103431]: osdmap e87: 12 total, 12 up, 12 in 2023-09-12T16:02:52.524 INFO:teuthology.orchestra.run.smithi130.stdout:34359738434 2023-09-12T16:02:52.525 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.0 2023-09-12T16:02:52.624 INFO:teuthology.orchestra.run.smithi130.stdout:270582939677 2023-09-12T16:02:52.625 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.9 2023-09-12T16:02:52.832 INFO:teuthology.orchestra.run.smithi130.stdout:55834574910 2023-09-12T16:02:52.833 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.1 2023-09-12T16:02:52.863 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:52.950 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:52.986 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:53.089 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:53.090 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:53.216 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:53.219 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:53.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:52 smithi130 ceph-mon[98670]: osdmap e88: 12 total, 12 up, 12 in 2023-09-12T16:02:53.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:52 smithi159 ceph-mon[102522]: osdmap e88: 12 total, 12 up, 12 in 2023-09-12T16:02:53.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:52 smithi203 ceph-mon[103431]: osdmap e88: 12 total, 12 up, 12 in 2023-09-12T16:02:53.513 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:53.697 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:54.100 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:53 smithi130 ceph-mon[98670]: pgmap v289: 129 pgs: 3 peering, 126 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-12T16:02:54.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:53 smithi159 ceph-mon[102522]: pgmap v289: 129 pgs: 3 peering, 126 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-12T16:02:54.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:53 smithi203 ceph-mon[103431]: pgmap v289: 129 pgs: 3 peering, 126 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-12T16:02:54.782 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:56.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.096+0000 7f60b5462700 1 -- 172.21.15.130:0/3453825087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b00ffeb0 msgr2=0x7f60b010c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:56.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.096+0000 7f60b5462700 1 --2- 172.21.15.130:0/3453825087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b00ffeb0 0x7f60b010c3c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009990 tx=0x7f60a0009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.096+0000 7f60b5462700 1 -- 172.21.15.130:0/3453825087 shutdown_connections 2023-09-12T16:02:56.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.096+0000 7f60b5462700 1 --2- 172.21.15.130:0/3453825087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b00ffeb0 0x7f60b010c3c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.096+0000 7f60b5462700 1 --2- 172.21.15.130:0/3453825087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0104500 0x7f60b01048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.096+0000 7f60b5462700 1 --2- 172.21.15.130:0/3453825087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b00ff4f0 0x7f60b00ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.096+0000 7f60b5462700 1 -- 172.21.15.130:0/3453825087 >> 172.21.15.130:0/3453825087 conn(0x7f60b0076650 msgr2=0x7f60b0076a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:56.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.097+0000 7f60b5462700 1 -- 172.21.15.130:0/3453825087 shutdown_connections 2023-09-12T16:02:56.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.097+0000 7f60b5462700 1 -- 172.21.15.130:0/3453825087 wait complete. 2023-09-12T16:02:56.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.097+0000 7f60b5462700 1 Processor -- start 2023-09-12T16:02:56.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60b5462700 1 -- start start 2023-09-12T16:02:56.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60b5462700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b00ff4f0 0x7f60b0197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:56.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60b5462700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b00ffeb0 0x7f60b0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:56.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60b5462700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b0104500 0x7f60b01a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:56.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60b5462700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b0198270 con 0x7f60b0104500 2023-09-12T16:02:56.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60b5462700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b01a29e0 con 0x7f60b00ff4f0 2023-09-12T16:02:56.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60b5462700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b01a2b50 con 0x7f60b00ffeb0 2023-09-12T16:02:56.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60af7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b0104500 0x7f60b01a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:56.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60af7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b0104500 0x7f60b01a24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54114/0 (socket says 172.21.15.130:54114) 2023-09-12T16:02:56.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60af7fe700 1 -- 172.21.15.130:0/192090613 learned_addr learned my addr 172.21.15.130:0/192090613 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:56.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60ae7fc700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b00ffeb0 0x7f60b0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:56.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60aeffd700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b00ff4f0 0x7f60b0197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:56.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60af7fe700 1 -- 172.21.15.130:0/192090613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b00ffeb0 msgr2=0x7f60b0197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:56.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60af7fe700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b00ffeb0 0x7f60b0197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60af7fe700 1 -- 172.21.15.130:0/192090613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b00ff4f0 msgr2=0x7f60b0197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:56.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60af7fe700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b00ff4f0 0x7f60b0197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60af7fe700 1 -- 172.21.15.130:0/192090613 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a00095f0 con 0x7f60b0104500 2023-09-12T16:02:56.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.098+0000 7f60aeffd700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b00ff4f0 0x7f60b0197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:56.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.099+0000 7f60af7fe700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b0104500 0x7f60b01a24a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f60a0005f50 tx=0x7f60a000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:56.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.099+0000 7f60a7fff700 1 -- 172.21.15.130:0/192090613 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a001d070 con 0x7f60b0104500 2023-09-12T16:02:56.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.099+0000 7f60a7fff700 1 -- 172.21.15.130:0/192090613 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a0005510 con 0x7f60b0104500 2023-09-12T16:02:56.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.099+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b01a2dd0 con 0x7f60b0104500 2023-09-12T16:02:56.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.099+0000 7f60a7fff700 1 -- 172.21.15.130:0/192090613 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0004c70 con 0x7f60b0104500 2023-09-12T16:02:56.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.099+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b01a31e0 con 0x7f60b0104500 2023-09-12T16:02:56.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.100+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b004e300 con 0x7f60b0104500 2023-09-12T16:02:56.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.101+0000 7f60a7fff700 1 -- 172.21.15.130:0/192090613 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f60a0005050 con 0x7f60b0104500 2023-09-12T16:02:56.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.101+0000 7f60a7fff700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f609c0a06d0 0x7f609c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:56.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.102+0000 7f60aeffd700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f609c0a06d0 0x7f609c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:56.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.102+0000 7f60a7fff700 1 -- 172.21.15.130:0/192090613 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f60a00cd300 con 0x7f60b0104500 2023-09-12T16:02:56.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.102+0000 7f60aeffd700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f609c0a06d0 0x7f609c0a2b90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f60a8007720 tx=0x7f60a80076b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:56.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.105+0000 7f60a7fff700 1 -- 172.21.15.130:0/192090613 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f60a0096a50 con 0x7f60b0104500 2023-09-12T16:02:56.208 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:55 smithi130 ceph-mon[98670]: pgmap v290: 129 pgs: 3 peering, 126 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:02:56.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:55 smithi159 ceph-mon[102522]: pgmap v290: 129 pgs: 3 peering, 126 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:02:56.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:55 smithi203 ceph-mon[103431]: pgmap v290: 129 pgs: 3 peering, 126 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:02:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.325+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f60b010f640 con 0x7f60b0104500 2023-09-12T16:02:56.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.326+0000 7f60a7fff700 1 -- 172.21.15.130:0/192090613 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f60a009a070 con 0x7f60b0104500 2023-09-12T16:02:56.326 INFO:teuthology.orchestra.run.smithi130.stdout:326417514518 2023-09-12T16:02:56.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f609c0a06d0 msgr2=0x7f609c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:56.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f609c0a06d0 0x7f609c0a2b90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f60a8007720 tx=0x7f60a80076b0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b0104500 msgr2=0x7f60b01a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:56.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b0104500 0x7f60b01a24a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f60a0005f50 tx=0x7f60a000f740 comp rx=0 tx=0).stop 2023-09-12T16:02:56.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 shutdown_connections 2023-09-12T16:02:56.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f609c0a06d0 0x7f609c0a2b90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f60b0104500 0x7f60b01a24a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b00ff4f0 0x7f60b0197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 --2- 172.21.15.130:0/192090613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b00ffeb0 0x7f60b0197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:56.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 >> 172.21.15.130:0/192090613 conn(0x7f60b0076650 msgr2=0x7f60b0101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:56.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 shutdown_connections 2023-09-12T16:02:56.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:56.328+0000 7f60b5462700 1 -- 172.21.15.130:0/192090613 wait complete. 2023-09-12T16:02:57.215 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:56 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/192090613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-12T16:02:57.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:56 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/192090613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-12T16:02:57.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:56 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/192090613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-12T16:02:57.470 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514517 got 326417514518 for osd.11 2023-09-12T16:02:57.471 DEBUG:teuthology.parallel:result is None 2023-09-12T16:02:57.586 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:02:58.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/3340521604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 msgr2=0x7f7a3810f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/3340521604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 0x7f7a3810f480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000b0a0 tx=0x7f7a3000b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/3340521604 shutdown_connections 2023-09-12T16:02:58.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/3340521604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 0x7f7a3810f480 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/3340521604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a38100050 0x7f7a38100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/3340521604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a38100970 0x7f7a3810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/3340521604 >> 172.21.15.130:0/3340521604 conn(0x7f7a380765f0 msgr2=0x7f7a38076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:58.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/3340521604 shutdown_connections 2023-09-12T16:02:58.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/3340521604 wait complete. 2023-09-12T16:02:58.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.060+0000 7f7a3dc4f700 1 Processor -- start 2023-09-12T16:02:58.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a3dc4f700 1 -- start start 2023-09-12T16:02:58.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a3dc4f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a38100050 0x7f7a3819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:58.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a3dc4f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a38100970 0x7f7a3819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:58.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a3dc4f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 0x7f7a38198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:58.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a3dc4f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38198fe0 con 0x7f7a3810d090 2023-09-12T16:02:58.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a3dc4f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38199150 con 0x7f7a38100970 2023-09-12T16:02:58.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a3dc4f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38199310 con 0x7f7a38100050 2023-09-12T16:02:58.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a37fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 0x7f7a38198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:58.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a37fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 0x7f7a38198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54126/0 (socket says 172.21.15.130:54126) 2023-09-12T16:02:58.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a37fff700 1 -- 172.21.15.130:0/1890744733 learned_addr learned my addr 172.21.15.130:0/1890744733 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:58.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a36ffd700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a38100970 0x7f7a3819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:58.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a377fe700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a38100050 0x7f7a3819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:58.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.061+0000 7f7a37fff700 1 -- 172.21.15.130:0/1890744733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a38100050 msgr2=0x7f7a3819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a37fff700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a38100050 0x7f7a3819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a37fff700 1 -- 172.21.15.130:0/1890744733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a38100970 msgr2=0x7f7a3819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a37fff700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a38100970 0x7f7a3819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a37fff700 1 -- 172.21.15.130:0/1890744733 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a30009d00 con 0x7f7a3810d090 2023-09-12T16:02:58.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a37fff700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 0x7f7a38198970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7a300062a0 tx=0x7f7a300079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:58.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a34ff9700 1 -- 172.21.15.130:0/1890744733 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000e040 con 0x7f7a3810d090 2023-09-12T16:02:58.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a34ff9700 1 -- 172.21.15.130:0/1890744733 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a30004d60 con 0x7f7a3810d090 2023-09-12T16:02:58.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a38199590 con 0x7f7a3810d090 2023-09-12T16:02:58.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a34ff9700 1 -- 172.21.15.130:0/1890744733 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3001d480 con 0x7f7a3810d090 2023-09-12T16:02:58.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.062+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a381a58e0 con 0x7f7a3810d090 2023-09-12T16:02:58.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.063+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3804e300 con 0x7f7a3810d090 2023-09-12T16:02:58.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.064+0000 7f7a34ff9700 1 -- 172.21.15.130:0/1890744733 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f7a300040b0 con 0x7f7a3810d090 2023-09-12T16:02:58.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.065+0000 7f7a34ff9700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a200a06d0 0x7f7a200a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:58.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.065+0000 7f7a34ff9700 1 -- 172.21.15.130:0/1890744733 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7a300cd760 con 0x7f7a3810d090 2023-09-12T16:02:58.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.065+0000 7f7a377fe700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a200a06d0 0x7f7a200a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:58.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.065+0000 7f7a377fe700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a200a06d0 0x7f7a200a2b90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a28005950 tx=0x7f7a280058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:58.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.070+0000 7f7a34ff9700 1 -- 172.21.15.130:0/1890744733 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7a30096eb0 con 0x7f7a3810d090 2023-09-12T16:02:58.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:57 smithi159 ceph-mon[102522]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s; 7.2 KiB/s, 0 objects/s recovering 2023-09-12T16:02:58.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:57 smithi130 ceph-mon[98670]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s; 7.2 KiB/s, 0 objects/s recovering 2023-09-12T16:02:58.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:57 smithi203 ceph-mon[103431]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s; 7.2 KiB/s, 0 objects/s recovering 2023-09-12T16:02:58.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.273+0000 7f22b09e0700 1 -- 172.21.15.130:0/1222146761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100fe0 msgr2=0x7f22ac10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.273+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1222146761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100fe0 0x7f22ac10d4f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f22a400b0a0 tx=0x7f22a400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.273+0000 7f22b09e0700 1 -- 172.21.15.130:0/1222146761 shutdown_connections 2023-09-12T16:02:58.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.273+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1222146761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100fe0 0x7f22ac10d4f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.273+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1222146761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22ac106620 0x7f22ac106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.273+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1222146761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac100620 0x7f22ac100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.273+0000 7f22b09e0700 1 -- 172.21.15.130:0/1222146761 >> 172.21.15.130:0/1222146761 conn(0x7f22ac0fc0f0 msgr2=0x7f22ac0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:58.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.273+0000 7f22b09e0700 1 -- 172.21.15.130:0/1222146761 shutdown_connections 2023-09-12T16:02:58.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.273+0000 7f22b09e0700 1 -- 172.21.15.130:0/1222146761 wait complete. 2023-09-12T16:02:58.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.274+0000 7f22b09e0700 1 Processor -- start 2023-09-12T16:02:58.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.274+0000 7f22b09e0700 1 -- start start 2023-09-12T16:02:58.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.274+0000 7f22b09e0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100620 0x7f22ac10c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:58.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.274+0000 7f22b09e0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22ac100fe0 0x7f22ac10c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:58.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.274+0000 7f22b09e0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac106620 0x7f22ac1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:58.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22b09e0700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1a1da0 con 0x7f22ac100620 2023-09-12T16:02:58.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22b09e0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1a1f10 con 0x7f22ac100fe0 2023-09-12T16:02:58.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22b09e0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1a20d0 con 0x7f22ac106620 2023-09-12T16:02:58.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aa59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100620 0x7f22ac10c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:58.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aa59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100620 0x7f22ac10c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54134/0 (socket says 172.21.15.130:54134) 2023-09-12T16:02:58.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aa59c700 1 -- 172.21.15.130:0/1152225400 learned_addr learned my addr 172.21.15.130:0/1152225400 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:58.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22a9d9b700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22ac100fe0 0x7f22ac10c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:58.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aad9d700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac106620 0x7f22ac1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:58.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aa59c700 1 -- 172.21.15.130:0/1152225400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac106620 msgr2=0x7f22ac1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aa59c700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac106620 0x7f22ac1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aa59c700 1 -- 172.21.15.130:0/1152225400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22ac100fe0 msgr2=0x7f22ac10c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aa59c700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22ac100fe0 0x7f22ac10c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aa59c700 1 -- 172.21.15.130:0/1152225400 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4009d00 con 0x7f22ac100620 2023-09-12T16:02:58.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aad9d700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac106620 0x7f22ac1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:58.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22a9d9b700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22ac100fe0 0x7f22ac10c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:02:58.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22aa59c700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100620 0x7f22ac10c460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f229c00d950 tx=0x7f229c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:58.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f229b7fe700 1 -- 172.21.15.130:0/1152225400 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c009940 con 0x7f22ac100620 2023-09-12T16:02:58.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f229b7fe700 1 -- 172.21.15.130:0/1152225400 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f229c010460 con 0x7f22ac100620 2023-09-12T16:02:58.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.275+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac1a23b0 con 0x7f22ac100620 2023-09-12T16:02:58.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.276+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac1a9c10 con 0x7f22ac100620 2023-09-12T16:02:58.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.276+0000 7f229b7fe700 1 -- 172.21.15.130:0/1152225400 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00f620 con 0x7f22ac100620 2023-09-12T16:02:58.283 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.277+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c005320 con 0x7f22ac100620 2023-09-12T16:02:58.284 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.279+0000 7f229b7fe700 1 -- 172.21.15.130:0/1152225400 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f229c009aa0 con 0x7f22ac100620 2023-09-12T16:02:58.284 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.279+0000 7f229b7fe700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f22940a06d0 0x7f22940a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:58.284 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.279+0000 7f229b7fe700 1 -- 172.21.15.130:0/1152225400 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f229c0cc0c0 con 0x7f22ac100620 2023-09-12T16:02:58.284 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.279+0000 7f22a9d9b700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f22940a06d0 0x7f22940a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:58.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.280+0000 7f22a9d9b700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f22940a06d0 0x7f22940a2b90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f22ac19e460 tx=0x7f22a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:58.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.282+0000 7f229b7fe700 1 -- 172.21.15.130:0/1152225400 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f229c095780 con 0x7f22ac100620 2023-09-12T16:02:58.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.283+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7a38112880 con 0x7f7a3810d090 2023-09-12T16:02:58.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.283+0000 7f7a34ff9700 1 -- 172.21.15.130:0/1890744733 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a3009a4d0 con 0x7f7a3810d090 2023-09-12T16:02:58.286 INFO:teuthology.orchestra.run.smithi130.stdout:158913789998 2023-09-12T16:02:58.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a200a06d0 msgr2=0x7f7a200a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a200a06d0 0x7f7a200a2b90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a28005950 tx=0x7f7a280058e0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 msgr2=0x7f7a38198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 0x7f7a38198970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7a300062a0 tx=0x7f7a300079b0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 shutdown_connections 2023-09-12T16:02:58.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a200a06d0 0x7f7a200a2b90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3810d090 0x7f7a38198970 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a38100050 0x7f7a3819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 --2- 172.21.15.130:0/1890744733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a38100970 0x7f7a3819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 >> 172.21.15.130:0/1890744733 conn(0x7f7a380765f0 msgr2=0x7f7a3810e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:58.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 shutdown_connections 2023-09-12T16:02:58.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.286+0000 7f7a3dc4f700 1 -- 172.21.15.130:0/1890744733 wait complete. 2023-09-12T16:02:58.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.491+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f228c0058a0 con 0x7f22ac100620 2023-09-12T16:02:58.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.491+0000 7f229b7fe700 1 -- 172.21.15.130:0/1152225400 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f229c020020 con 0x7f22ac100620 2023-09-12T16:02:58.491 INFO:teuthology.orchestra.run.smithi130.stdout:240518168610 2023-09-12T16:02:58.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f22940a06d0 msgr2=0x7f22940a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f22940a06d0 0x7f22940a2b90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f22ac19e460 tx=0x7f22a0009260 comp rx=0 tx=0).stop 2023-09-12T16:02:58.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100620 msgr2=0x7f22ac10c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:58.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100620 0x7f22ac10c460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f229c00d950 tx=0x7f229c00dc60 comp rx=0 tx=0).stop 2023-09-12T16:02:58.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 shutdown_connections 2023-09-12T16:02:58.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f22940a06d0 0x7f22940a2b90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac106620 0x7f22ac1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ac100620 0x7f22ac10c460 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 --2- 172.21.15.130:0/1152225400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22ac100fe0 0x7f22ac10c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:58.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 >> 172.21.15.130:0/1152225400 conn(0x7f22ac0fc0f0 msgr2=0x7f22ac0fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:58.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 shutdown_connections 2023-09-12T16:02:58.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:58.494+0000 7f22b09e0700 1 -- 172.21.15.130:0/1152225400 wait complete. 2023-09-12T16:02:59.042 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:58 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1890744733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-12T16:02:59.042 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:58 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1152225400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-12T16:02:59.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:58 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1890744733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-12T16:02:59.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:58 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1152225400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-12T16:02:59.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:58 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1890744733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-12T16:02:59.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:58 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1152225400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-12T16:02:59.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.414+0000 7f9362aa6700 1 -- 172.21.15.130:0/281584316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c0ff520 msgr2=0x7f935c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.414+0000 7f9362aa6700 1 --2- 172.21.15.130:0/281584316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c0ff520 0x7f935c0ff9a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9348009990 tx=0x7f9348009ca0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.414+0000 7f9362aa6700 1 -- 172.21.15.130:0/281584316 shutdown_connections 2023-09-12T16:02:59.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.414+0000 7f9362aa6700 1 --2- 172.21.15.130:0/281584316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f935c0ffee0 0x7f935c10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.414+0000 7f9362aa6700 1 --2- 172.21.15.130:0/281584316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f935c104530 0x7f935c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.414+0000 7f9362aa6700 1 --2- 172.21.15.130:0/281584316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c0ff520 0x7f935c0ff9a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.414+0000 7f9362aa6700 1 -- 172.21.15.130:0/281584316 >> 172.21.15.130:0/281584316 conn(0x7f935c076600 msgr2=0x7f935c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:59.416 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.414+0000 7f9362aa6700 1 -- 172.21.15.130:0/281584316 shutdown_connections 2023-09-12T16:02:59.416 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.414+0000 7f9362aa6700 1 -- 172.21.15.130:0/281584316 wait complete. 2023-09-12T16:02:59.417 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.415+0000 7f9362aa6700 1 Processor -- start 2023-09-12T16:02:59.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.415+0000 7f9362aa6700 1 -- start start 2023-09-12T16:02:59.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.415+0000 7f9362aa6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f935c0ff520 0x7f935c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:59.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.415+0000 7f9362aa6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f935c0ffee0 0x7f935c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:59.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.415+0000 7f9362aa6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c104530 0x7f935c1a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:59.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.415+0000 7f9362aa6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c198380 con 0x7f935c104530 2023-09-12T16:02:59.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.415+0000 7f9362aa6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1a2ac0 con 0x7f935c0ffee0 2023-09-12T16:02:59.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.415+0000 7f9362aa6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1a2c80 con 0x7f935c0ff520 2023-09-12T16:02:59.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.415+0000 7f9361043700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c104530 0x7f935c1a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:59.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9361043700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c104530 0x7f935c1a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54152/0 (socket says 172.21.15.130:54152) 2023-09-12T16:02:59.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9361043700 1 -- 172.21.15.130:0/2754397143 learned_addr learned my addr 172.21.15.130:0/2754397143 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:59.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9360842700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f935c0ff520 0x7f935c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:59.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f935bfff700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f935c0ffee0 0x7f935c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:59.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9361043700 1 -- 172.21.15.130:0/2754397143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f935c0ff520 msgr2=0x7f935c197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9361043700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f935c0ff520 0x7f935c197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9361043700 1 -- 172.21.15.130:0/2754397143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f935c0ffee0 msgr2=0x7f935c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9361043700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f935c0ffee0 0x7f935c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9361043700 1 -- 172.21.15.130:0/2754397143 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93480095f0 con 0x7f935c104530 2023-09-12T16:02:59.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9361043700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c104530 0x7f935c1a2580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f935400efc0 tx=0x7f935400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:59.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9359ffb700 1 -- 172.21.15.130:0/2754397143 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354018070 con 0x7f935c104530 2023-09-12T16:02:59.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9359ffb700 1 -- 172.21.15.130:0/2754397143 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f935400f040 con 0x7f935c104530 2023-09-12T16:02:59.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c1a2f00 con 0x7f935c104530 2023-09-12T16:02:59.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.416+0000 7f9359ffb700 1 -- 172.21.15.130:0/2754397143 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935400ccc0 con 0x7f935c104530 2023-09-12T16:02:59.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.417+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c1a33d0 con 0x7f935c104530 2023-09-12T16:02:59.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.418+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c04e300 con 0x7f935c104530 2023-09-12T16:02:59.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.419+0000 7f9359ffb700 1 -- 172.21.15.130:0/2754397143 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f9354003a00 con 0x7f935c104530 2023-09-12T16:02:59.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.419+0000 7f9359ffb700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f934c0a0820 0x7f934c0a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:59.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.420+0000 7f9360842700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f934c0a0820 0x7f934c0a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:59.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.420+0000 7f9359ffb700 1 -- 172.21.15.130:0/2754397143 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f93540cb980 con 0x7f935c104530 2023-09-12T16:02:59.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.420+0000 7f9360842700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f934c0a0820 0x7f934c0a2ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f935c069c10 tx=0x7f935000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:59.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.425+0000 7f9359ffb700 1 -- 172.21.15.130:0/2754397143 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f93540950d0 con 0x7f935c104530 2023-09-12T16:02:59.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.625+0000 7fc38c36c700 1 -- 172.21.15.130:0/1818497876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840ff4b0 msgr2=0x7fc38410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.625+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1818497876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840ff4b0 0x7fc38410d4e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00b0a0 tx=0x7fc37c00b3b0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.626+0000 7fc38c36c700 1 -- 172.21.15.130:0/1818497876 shutdown_connections 2023-09-12T16:02:59.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.626+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1818497876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840ff4b0 0x7fc38410d4e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.626+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1818497876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc384106610 0x7fc3841069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.626+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1818497876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3840feaf0 0x7fc3840fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.626+0000 7fc38c36c700 1 -- 172.21.15.130:0/1818497876 >> 172.21.15.130:0/1818497876 conn(0x7fc384075a70 msgr2=0x7fc3840fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:59.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.626+0000 7fc38c36c700 1 -- 172.21.15.130:0/1818497876 shutdown_connections 2023-09-12T16:02:59.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.626+0000 7fc38c36c700 1 -- 172.21.15.130:0/1818497876 wait complete. 2023-09-12T16:02:59.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.626+0000 7fc38c36c700 1 Processor -- start 2023-09-12T16:02:59.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.626+0000 7fc38c36c700 1 -- start start 2023-09-12T16:02:59.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38c36c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840feaf0 0x7fc3840ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:59.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38c36c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3840ff4b0 0x7fc384100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:59.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38c36c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc384106610 0x7fc384100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:59.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38c36c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3841a91e0 con 0x7fc3840feaf0 2023-09-12T16:02:59.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38c36c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3841a9320 con 0x7fc3840ff4b0 2023-09-12T16:02:59.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38c36c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38419a9a0 con 0x7fc384106610 2023-09-12T16:02:59.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a108700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840feaf0 0x7fc3840ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:59.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a108700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840feaf0 0x7fc3840ffcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54162/0 (socket says 172.21.15.130:54162) 2023-09-12T16:02:59.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a108700 1 -- 172.21.15.130:0/1466934041 learned_addr learned my addr 172.21.15.130:0/1466934041 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:02:59.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a909700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc384106610 0x7fc384100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:59.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc389907700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3840ff4b0 0x7fc384100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:59.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a909700 1 -- 172.21.15.130:0/1466934041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3840ff4b0 msgr2=0x7fc384100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a909700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3840ff4b0 0x7fc384100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a909700 1 -- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840feaf0 msgr2=0x7fc3840ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a909700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840feaf0 0x7fc3840ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a909700 1 -- 172.21.15.130:0/1466934041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c009d00 con 0x7fc384106610 2023-09-12T16:02:59.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a108700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840feaf0 0x7fc3840ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:02:59.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc389907700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3840ff4b0 0x7fc384100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:02:59.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38a909700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc384106610 0x7fc384100750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc37c015000 tx=0x7fc37c007990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:59.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc37b7fe700 1 -- 172.21.15.130:0/1466934041 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00e040 con 0x7fc384106610 2023-09-12T16:02:59.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.627+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38419ac20 con 0x7fc384106610 2023-09-12T16:02:59.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.628+0000 7fc37b7fe700 1 -- 172.21.15.130:0/1466934041 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc37c00be40 con 0x7fc384106610 2023-09-12T16:02:59.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.628+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38419b120 con 0x7fc384106610 2023-09-12T16:02:59.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.628+0000 7fc37b7fe700 1 -- 172.21.15.130:0/1466934041 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0052a0 con 0x7fc384106610 2023-09-12T16:02:59.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.628+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38404e300 con 0x7fc384106610 2023-09-12T16:02:59.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.630+0000 7fc37b7fe700 1 -- 172.21.15.130:0/1466934041 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fc37c01d470 con 0x7fc384106610 2023-09-12T16:02:59.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.630+0000 7fc37b7fe700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3700a06d0 0x7fc3700a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:02:59.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.630+0000 7fc38a108700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3700a06d0 0x7fc3700a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:02:59.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.630+0000 7fc37b7fe700 1 -- 172.21.15.130:0/1466934041 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc37c0cdb80 con 0x7fc384106610 2023-09-12T16:02:59.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.631+0000 7fc38a108700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3700a06d0 0x7fc3700a2b90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc3800103d0 tx=0x7fc380007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:02:59.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.633+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f935c10f710 con 0x7f935c104530 2023-09-12T16:02:59.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.634+0000 7f9359ffb700 1 -- 172.21.15.130:0/2754397143 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93540986f0 con 0x7f935c104530 2023-09-12T16:02:59.637 INFO:teuthology.orchestra.run.smithi130.stdout:137438953524 2023-09-12T16:02:59.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.634+0000 7fc37b7fe700 1 -- 172.21.15.130:0/1466934041 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc37c0973f0 con 0x7fc384106610 2023-09-12T16:02:59.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f934c0a0820 msgr2=0x7f934c0a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f934c0a0820 0x7f934c0a2ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f935c069c10 tx=0x7f935000b410 comp rx=0 tx=0).stop 2023-09-12T16:02:59.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c104530 msgr2=0x7f935c1a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c104530 0x7f935c1a2580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f935400efc0 tx=0x7f935400c3a0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 shutdown_connections 2023-09-12T16:02:59.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f934c0a0820 0x7f934c0a2ce0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f935c104530 0x7f935c1a2580 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f935c0ff520 0x7f935c197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 --2- 172.21.15.130:0/2754397143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f935c0ffee0 0x7f935c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 >> 172.21.15.130:0/2754397143 conn(0x7f935c076600 msgr2=0x7f935c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:59.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 shutdown_connections 2023-09-12T16:02:59.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.636+0000 7f9362aa6700 1 -- 172.21.15.130:0/2754397143 wait complete. 2023-09-12T16:02:59.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.835+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc3841105a0 con 0x7fc384106610 2023-09-12T16:02:59.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.835+0000 7fc37b7fe700 1 -- 172.21.15.130:0/1466934041 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc37c021090 con 0x7fc384106610 2023-09-12T16:02:59.835 INFO:teuthology.orchestra.run.smithi130.stdout:77309411387 2023-09-12T16:02:59.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3700a06d0 msgr2=0x7fc3700a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3700a06d0 0x7fc3700a2b90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc3800103d0 tx=0x7fc380007480 comp rx=0 tx=0).stop 2023-09-12T16:02:59.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc384106610 msgr2=0x7fc384100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:02:59.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc384106610 0x7fc384100750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc37c015000 tx=0x7fc37c007990 comp rx=0 tx=0).stop 2023-09-12T16:02:59.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 shutdown_connections 2023-09-12T16:02:59.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3700a06d0 0x7fc3700a2b90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc384106610 0x7fc384100750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3840feaf0 0x7fc3840ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 --2- 172.21.15.130:0/1466934041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3840ff4b0 0x7fc384100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:02:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 >> 172.21.15.130:0/1466934041 conn(0x7fc384075a70 msgr2=0x7fc3840fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:02:59.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 shutdown_connections 2023-09-12T16:02:59.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:02:59.838+0000 7fc38c36c700 1 -- 172.21.15.130:0/1466934041 wait complete. 2023-09-12T16:02:59.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:59 smithi130 ceph-mon[98670]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 3 op/s; 7.3 KiB/s, 1 objects/s recovering 2023-09-12T16:02:59.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:59 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2754397143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-12T16:02:59.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:02:59 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1466934041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-12T16:03:00.000 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168609 got 240518168610 for osd.8 2023-09-12T16:03:00.001 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:00.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:59 smithi159 ceph-mon[102522]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 3 op/s; 7.3 KiB/s, 1 objects/s recovering 2023-09-12T16:03:00.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:59 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2754397143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-12T16:03:00.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:02:59 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1466934041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-12T16:03:00.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:59 smithi203 ceph-mon[103431]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 3 op/s; 7.3 KiB/s, 1 objects/s recovering 2023-09-12T16:03:00.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:59 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2754397143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-12T16:03:00.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:02:59 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1466934041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-12T16:03:00.427 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953522 got 137438953524 for osd.4 2023-09-12T16:03:00.428 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:00.909 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:00 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/567570566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-12T16:03:01.025 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411385 got 77309411387 for osd.2 2023-09-12T16:03:01.026 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:01.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.966+0000 7f0810c78700 1 -- 172.21.15.130:0/869472221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c101020 msgr2=0x7f080c10d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.966+0000 7f0810c78700 1 --2- 172.21.15.130:0/869472221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c101020 0x7f080c10d530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f080400b0a0 tx=0x7f080400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.966+0000 7f0810c78700 1 -- 172.21.15.130:0/869472221 shutdown_connections 2023-09-12T16:03:01.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.966+0000 7f0810c78700 1 --2- 172.21.15.130:0/869472221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c101020 0x7f080c10d530 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.966+0000 7f0810c78700 1 --2- 172.21.15.130:0/869472221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f080c106660 0x7f080c106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.966+0000 7f0810c78700 1 --2- 172.21.15.130:0/869472221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f080c100660 0x7f080c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.966+0000 7f0810c78700 1 -- 172.21.15.130:0/869472221 >> 172.21.15.130:0/869472221 conn(0x7f080c0fc150 msgr2=0x7f080c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.966+0000 7f0810c78700 1 -- 172.21.15.130:0/869472221 shutdown_connections 2023-09-12T16:03:01.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.966+0000 7f0810c78700 1 -- 172.21.15.130:0/869472221 wait complete. 2023-09-12T16:03:01.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f0810c78700 1 Processor -- start 2023-09-12T16:03:01.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f0810c78700 1 -- start start 2023-09-12T16:03:01.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f0810c78700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f080c100660 0x7f080c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f0810c78700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f080c101020 0x7f080c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f0810c78700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c106660 0x7f080c0733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f0810c78700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c1a4cd0 con 0x7f080c106660 2023-09-12T16:03:01.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f0810c78700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c1a4e40 con 0x7f080c101020 2023-09-12T16:03:01.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f0810c78700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c1a5000 con 0x7f080c100660 2023-09-12T16:03:01.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f080ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c106660 0x7f080c0733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f080ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c106660 0x7f080c0733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54192/0 (socket says 172.21.15.130:54192) 2023-09-12T16:03:01.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f080ad9d700 1 -- 172.21.15.130:0/1954874819 learned_addr learned my addr 172.21.15.130:0/1954874819 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:01.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f080a59c700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f080c100660 0x7f080c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f0809d9b700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f080c101020 0x7f080c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f080ad9d700 1 -- 172.21.15.130:0/1954874819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f080c100660 msgr2=0x7f080c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f080ad9d700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f080c100660 0x7f080c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f080ad9d700 1 -- 172.21.15.130:0/1954874819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f080c101020 msgr2=0x7f080c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.967+0000 7f080ad9d700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f080c101020 0x7f080c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.968+0000 7f080ad9d700 1 -- 172.21.15.130:0/1954874819 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0804009d00 con 0x7f080c106660 2023-09-12T16:03:01.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.968+0000 7f080ad9d700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c106660 0x7f080c0733c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0804009fd0 tx=0x7f080400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.968+0000 7f07fb7fe700 1 -- 172.21.15.130:0/1954874819 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080400e040 con 0x7f080c106660 2023-09-12T16:03:01.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.968+0000 7f07fb7fe700 1 -- 172.21.15.130:0/1954874819 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0804004e60 con 0x7f080c106660 2023-09-12T16:03:01.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.968+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080c1a5280 con 0x7f080c106660 2023-09-12T16:03:01.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.968+0000 7f07fb7fe700 1 -- 172.21.15.130:0/1954874819 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08040053c0 con 0x7f080c106660 2023-09-12T16:03:01.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.968+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c1a5740 con 0x7f080c106660 2023-09-12T16:03:01.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.969+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080c04e300 con 0x7f080c106660 2023-09-12T16:03:01.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.970+0000 7f07fb7fe700 1 -- 172.21.15.130:0/1954874819 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f080401da30 con 0x7f080c106660 2023-09-12T16:03:01.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.971+0000 7f07fb7fe700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07f40a0750 0x7f07f40a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.971+0000 7f080a59c700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07f40a0750 0x7f07f40a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.971+0000 7f07fb7fe700 1 -- 172.21.15.130:0/1954874819 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f08040cd8c0 con 0x7f080c106660 2023-09-12T16:03:01.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.971+0000 7f080a59c700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07f40a0750 0x7f07f40a2c10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0800005950 tx=0x7f080000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.975+0000 7f07fb7fe700 1 -- 172.21.15.130:0/1954874819 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0804097010 con 0x7f080c106660 2023-09-12T16:03:01.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.540+0000 7f27605e6700 1 -- 172.21.15.130:0/267180725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ff520 msgr2=0x7f27580ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.540+0000 7f27605e6700 1 --2- 172.21.15.130:0/267180725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ff520 0x7f27580ff9a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2748009990 tx=0x7f2748009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.540+0000 7f27605e6700 1 -- 172.21.15.130:0/267180725 shutdown_connections 2023-09-12T16:03:01.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.540+0000 7f27605e6700 1 --2- 172.21.15.130:0/267180725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27580ffee0 0x7f275810c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.540+0000 7f27605e6700 1 --2- 172.21.15.130:0/267180725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2758104530 0x7f2758104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.540+0000 7f27605e6700 1 --2- 172.21.15.130:0/267180725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ff520 0x7f27580ff9a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.540+0000 7f27605e6700 1 -- 172.21.15.130:0/267180725 >> 172.21.15.130:0/267180725 conn(0x7f2758076600 msgr2=0x7f2758076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 -- 172.21.15.130:0/267180725 shutdown_connections 2023-09-12T16:03:01.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 -- 172.21.15.130:0/267180725 wait complete. 2023-09-12T16:03:01.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 Processor -- start 2023-09-12T16:03:01.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 -- start start 2023-09-12T16:03:01.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27580ff520 0x7f27581955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ffee0 0x7f2758195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2758104530 0x7f2758199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275819a560 con 0x7f27580ffee0 2023-09-12T16:03:01.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275819a6d0 con 0x7f27580ff520 2023-09-12T16:03:01.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f27605e6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275819a890 con 0x7f2758104530 2023-09-12T16:03:01.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.541+0000 7f275db81700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ffee0 0x7f2758195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275db81700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ffee0 0x7f2758195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54178/0 (socket says 172.21.15.130:54178) 2023-09-12T16:03:01.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275db81700 1 -- 172.21.15.130:0/567570566 learned_addr learned my addr 172.21.15.130:0/567570566 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:01.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275e382700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27580ff520 0x7f27581955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275eb83700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2758104530 0x7f2758199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275db81700 1 -- 172.21.15.130:0/567570566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2758104530 msgr2=0x7f2758199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275db81700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2758104530 0x7f2758199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275db81700 1 -- 172.21.15.130:0/567570566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27580ff520 msgr2=0x7f27581955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275db81700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27580ff520 0x7f27581955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275db81700 1 -- 172.21.15.130:0/567570566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27480095f0 con 0x7f27580ffee0 2023-09-12T16:03:01.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275eb83700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2758104530 0x7f2758199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:01.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f275db81700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ffee0 0x7f2758195ae0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27480098e0 tx=0x7f274800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f274f7fe700 1 -- 172.21.15.130:0/567570566 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274801d070 con 0x7f27580ffee0 2023-09-12T16:03:01.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275819ab10 con 0x7f27580ffee0 2023-09-12T16:03:01.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f274f7fe700 1 -- 172.21.15.130:0/567570566 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2748005510 con 0x7f27580ffee0 2023-09-12T16:03:01.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f274f7fe700 1 -- 172.21.15.130:0/567570566 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748004bd0 con 0x7f27580ffee0 2023-09-12T16:03:01.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.542+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27581a23d0 con 0x7f27580ffee0 2023-09-12T16:03:01.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.543+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275804e300 con 0x7f27580ffee0 2023-09-12T16:03:01.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.544+0000 7f274f7fe700 1 -- 172.21.15.130:0/567570566 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f2748004d30 con 0x7f27580ffee0 2023-09-12T16:03:01.078 INFO:teuthology.orchestra.run.smithi130.stdout:292057776153 2023-09-12T16:03:01.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.545+0000 7f274f7fe700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f27440a06d0 0x7f27440a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.545+0000 7f275e382700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f27440a06d0 0x7f27440a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.545+0000 7f274f7fe700 1 -- 172.21.15.130:0/567570566 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f27480ccdc0 con 0x7f27580ffee0 2023-09-12T16:03:01.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.545+0000 7f275e382700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f27440a06d0 0x7f27440a2b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2758069c10 tx=0x7f275400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.549+0000 7f274f7fe700 1 -- 172.21.15.130:0/567570566 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2748096510 con 0x7f27580ffee0 2023-09-12T16:03:01.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.748+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f275810f450 con 0x7f27580ffee0 2023-09-12T16:03:01.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.748+0000 7f274f7fe700 1 -- 172.21.15.130:0/567570566 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f2748099b30 con 0x7f27580ffee0 2023-09-12T16:03:01.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f27440a06d0 msgr2=0x7f27440a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f27440a06d0 0x7f27440a2b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2758069c10 tx=0x7f275400a380 comp rx=0 tx=0).stop 2023-09-12T16:03:01.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ffee0 msgr2=0x7f2758195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ffee0 0x7f2758195ae0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27480098e0 tx=0x7f274800f670 comp rx=0 tx=0).stop 2023-09-12T16:03:01.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 shutdown_connections 2023-09-12T16:03:01.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f27440a06d0 0x7f27440a2b90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2758104530 0x7f2758199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27580ff520 0x7f27581955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 --2- 172.21.15.130:0/567570566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27580ffee0 0x7f2758195ae0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.750+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 >> 172.21.15.130:0/567570566 conn(0x7f2758076600 msgr2=0x7f27580fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.751+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 shutdown_connections 2023-09-12T16:03:01.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:00.751+0000 7f27605e6700 1 -- 172.21.15.130:0/567570566 wait complete. 2023-09-12T16:03:01.083 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789997 got 158913789998 for osd.5 2023-09-12T16:03:01.083 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:01.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.127+0000 7f1e47068700 1 -- 172.21.15.130:0/232882482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e400ffec0 msgr2=0x7f1e401002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.127+0000 7f1e47068700 1 --2- 172.21.15.130:0/232882482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e400ffec0 0x7f1e401002a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1e340098f0 tx=0x7f1e34009c00 comp rx=0 tx=0).stop 2023-09-12T16:03:01.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.127+0000 7f1e47068700 1 -- 172.21.15.130:0/232882482 shutdown_connections 2023-09-12T16:03:01.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.127+0000 7f1e47068700 1 --2- 172.21.15.130:0/232882482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e401048e0 0x7f1e4010c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.127+0000 7f1e47068700 1 --2- 172.21.15.130:0/232882482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e400ffec0 0x7f1e401002a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.127+0000 7f1e47068700 1 --2- 172.21.15.130:0/232882482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e40100870 0x7f1e401041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.127+0000 7f1e47068700 1 -- 172.21.15.130:0/232882482 >> 172.21.15.130:0/232882482 conn(0x7f1e40075a70 msgr2=0x7f1e400fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.127+0000 7f1e47068700 1 -- 172.21.15.130:0/232882482 shutdown_connections 2023-09-12T16:03:01.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.127+0000 7f1e47068700 1 -- 172.21.15.130:0/232882482 wait complete. 2023-09-12T16:03:01.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.128+0000 7f1e47068700 1 Processor -- start 2023-09-12T16:03:01.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.128+0000 7f1e47068700 1 -- start start 2023-09-12T16:03:01.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.128+0000 7f1e47068700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e400ffec0 0x7f1e40197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.128+0000 7f1e47068700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e40100870 0x7f1e40197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.128+0000 7f1e47068700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e401048e0 0x7f1e401a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.128+0000 7f1e47068700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40198380 con 0x7f1e40100870 2023-09-12T16:03:01.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.128+0000 7f1e47068700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401a2af0 con 0x7f1e400ffec0 2023-09-12T16:03:01.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.128+0000 7f1e47068700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401a2c60 con 0x7f1e401048e0 2023-09-12T16:03:01.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e40100870 0x7f1e40197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e40100870 0x7f1e40197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54204/0 (socket says 172.21.15.130:54204) 2023-09-12T16:03:01.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3ffff700 1 -- 172.21.15.130:0/96080839 learned_addr learned my addr 172.21.15.130:0/96080839 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:01.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e45605700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e401048e0 0x7f1e401a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e44e04700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e400ffec0 0x7f1e40197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3ffff700 1 -- 172.21.15.130:0/96080839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e401048e0 msgr2=0x7f1e401a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3ffff700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e401048e0 0x7f1e401a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3ffff700 1 -- 172.21.15.130:0/96080839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e400ffec0 msgr2=0x7f1e40197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3ffff700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e400ffec0 0x7f1e40197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3ffff700 1 -- 172.21.15.130:0/96080839 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e34009520 con 0x7f1e40100870 2023-09-12T16:03:01.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e45605700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e401048e0 0x7f1e401a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:01.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e44e04700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e400ffec0 0x7f1e40197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:01.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3ffff700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e40100870 0x7f1e40197a70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00e9c0 tx=0x7f1e2c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3dffb700 1 -- 172.21.15.130:0/96080839 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00cbd0 con 0x7f1e40100870 2023-09-12T16:03:01.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3dffb700 1 -- 172.21.15.130:0/96080839 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e2c00cd30 con 0x7f1e40100870 2023-09-12T16:03:01.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e401a2f40 con 0x7f1e40100870 2023-09-12T16:03:01.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e3dffb700 1 -- 172.21.15.130:0/96080839 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c005580 con 0x7f1e40100870 2023-09-12T16:03:01.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.129+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e401a33b0 con 0x7f1e40100870 2023-09-12T16:03:01.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.130+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4004e300 con 0x7f1e40100870 2023-09-12T16:03:01.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.131+0000 7f1e3dffb700 1 -- 172.21.15.130:0/96080839 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f1e2c004750 con 0x7f1e40100870 2023-09-12T16:03:01.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.132+0000 7f1e3dffb700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1e300a06d0 0x7f1e300a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.132+0000 7f1e44e04700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1e300a06d0 0x7f1e300a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.132+0000 7f1e3dffb700 1 -- 172.21.15.130:0/96080839 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1e2c014070 con 0x7f1e40100870 2023-09-12T16:03:01.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.132+0000 7f1e44e04700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1e300a06d0 0x7f1e300a2b90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c010 tx=0x7f1e34005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.136+0000 7f1e3dffb700 1 -- 172.21.15.130:0/96080839 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1e2c0959b0 con 0x7f1e40100870 2023-09-12T16:03:01.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.192+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f080c1108a0 con 0x7f080c106660 2023-09-12T16:03:01.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.192+0000 7f07fb7fe700 1 -- 172.21.15.130:0/1954874819 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f080409a630 con 0x7f080c106660 2023-09-12T16:03:01.193 INFO:teuthology.orchestra.run.smithi130.stdout:107374182456 2023-09-12T16:03:01.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07f40a0750 msgr2=0x7f07f40a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07f40a0750 0x7f07f40a2c10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0800005950 tx=0x7f080000b410 comp rx=0 tx=0).stop 2023-09-12T16:03:01.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c106660 msgr2=0x7f080c0733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c106660 0x7f080c0733c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0804009fd0 tx=0x7f080400bce0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 shutdown_connections 2023-09-12T16:03:01.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07f40a0750 0x7f07f40a2c10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f080c106660 0x7f080c0733c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f080c100660 0x7f080c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 --2- 172.21.15.130:0/1954874819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f080c101020 0x7f080c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 >> 172.21.15.130:0/1954874819 conn(0x7f080c0fc150 msgr2=0x7f080c0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 shutdown_connections 2023-09-12T16:03:01.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.195+0000 7f0810c78700 1 -- 172.21.15.130:0/1954874819 wait complete. 2023-09-12T16:03:01.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:00 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/567570566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-12T16:03:01.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:00 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/567570566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-12T16:03:01.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.354+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1e400ff140 con 0x7f1e40100870 2023-09-12T16:03:01.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.354+0000 7f1e3dffb700 1 -- 172.21.15.130:0/96080839 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e2c098fd0 con 0x7f1e40100870 2023-09-12T16:03:01.354 INFO:teuthology.orchestra.run.smithi130.stdout:210453397544 2023-09-12T16:03:01.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.357+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1e300a06d0 msgr2=0x7f1e300a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.357+0000 7f1e47068700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1e300a06d0 0x7f1e300a2b90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c010 tx=0x7f1e34005b40 comp rx=0 tx=0).stop 2023-09-12T16:03:01.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.357+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e40100870 msgr2=0x7f1e40197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.357+0000 7f1e47068700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e40100870 0x7f1e40197a70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00e9c0 tx=0x7f1e2c00ed80 comp rx=0 tx=0).stop 2023-09-12T16:03:01.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.358+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 shutdown_connections 2023-09-12T16:03:01.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.358+0000 7f1e47068700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1e300a06d0 0x7f1e300a2b90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.358+0000 7f1e47068700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e401048e0 0x7f1e401a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.358+0000 7f1e47068700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e400ffec0 0x7f1e40197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.358+0000 7f1e47068700 1 --2- 172.21.15.130:0/96080839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e40100870 0x7f1e40197a70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.358+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 >> 172.21.15.130:0/96080839 conn(0x7f1e40075a70 msgr2=0x7f1e40103190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.358+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 shutdown_connections 2023-09-12T16:03:01.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.358+0000 7f1e47068700 1 -- 172.21.15.130:0/96080839 wait complete. 2023-09-12T16:03:01.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.418+0000 7f892a262700 1 -- 172.21.15.130:0/3527855844 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89240ffee0 msgr2=0x7f892410c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.418+0000 7f892a262700 1 --2- 172.21.15.130:0/3527855844 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89240ffee0 0x7f892410c3f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b0a0 tx=0x7f891c00b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.418+0000 7f892a262700 1 -- 172.21.15.130:0/3527855844 shutdown_connections 2023-09-12T16:03:01.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.418+0000 7f892a262700 1 --2- 172.21.15.130:0/3527855844 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89240ffee0 0x7f892410c3f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.418+0000 7f892a262700 1 --2- 172.21.15.130:0/3527855844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8924104530 0x7f8924104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.418+0000 7f892a262700 1 --2- 172.21.15.130:0/3527855844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89240ff520 0x7f89240ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.418+0000 7f892a262700 1 -- 172.21.15.130:0/3527855844 >> 172.21.15.130:0/3527855844 conn(0x7f8924076600 msgr2=0x7f8924076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.418+0000 7f892a262700 1 -- 172.21.15.130:0/3527855844 shutdown_connections 2023-09-12T16:03:01.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.418+0000 7f892a262700 1 -- 172.21.15.130:0/3527855844 wait complete. 2023-09-12T16:03:01.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f892a262700 1 Processor -- start 2023-09-12T16:03:01.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f892a262700 1 -- start start 2023-09-12T16:03:01.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f892a262700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89240ff520 0x7f8924101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f892a262700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89240ffee0 0x7f89241016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f892a262700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8924104530 0x7f8924101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f892a262700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89241a4c90 con 0x7f8924104530 2023-09-12T16:03:01.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f892a262700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89241a4e00 con 0x7f89240ffee0 2023-09-12T16:03:01.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f892a262700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89241a4fc0 con 0x7f89240ff520 2023-09-12T16:03:01.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f8923fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8924104530 0x7f8924101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f8923fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8924104530 0x7f8924101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54220/0 (socket says 172.21.15.130:54220) 2023-09-12T16:03:01.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f8923fff700 1 -- 172.21.15.130:0/2425838215 learned_addr learned my addr 172.21.15.130:0/2425838215 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:01.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f89237fe700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89240ff520 0x7f8924101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.419+0000 7f8922ffd700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89240ffee0 0x7f89241016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.420+0000 7f8923fff700 1 -- 172.21.15.130:0/2425838215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89240ff520 msgr2=0x7f8924101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.420+0000 7f8923fff700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89240ff520 0x7f8924101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.420+0000 7f8923fff700 1 -- 172.21.15.130:0/2425838215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89240ffee0 msgr2=0x7f89241016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.420+0000 7f8923fff700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89240ffee0 0x7f89241016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.420+0000 7f8923fff700 1 -- 172.21.15.130:0/2425838215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c009d00 con 0x7f8924104530 2023-09-12T16:03:01.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.420+0000 7f89237fe700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89240ff520 0x7f8924101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:03:01.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.420+0000 7f8922ffd700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89240ffee0 0x7f89241016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:03:01.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.420+0000 7f8923fff700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8924104530 0x7f8924101c40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b830 tx=0x7f891c009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.421+0000 7f8920ff9700 1 -- 172.21.15.130:0/2425838215 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c00e040 con 0x7f8924104530 2023-09-12T16:03:01.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.421+0000 7f8920ff9700 1 -- 172.21.15.130:0/2425838215 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891c004ce0 con 0x7f8924104530 2023-09-12T16:03:01.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.421+0000 7f8920ff9700 1 -- 172.21.15.130:0/2425838215 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c01d480 con 0x7f8924104530 2023-09-12T16:03:01.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.421+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89241a5240 con 0x7f8924104530 2023-09-12T16:03:01.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.421+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89241a5660 con 0x7f8924104530 2023-09-12T16:03:01.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.423+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892404e300 con 0x7f8924104530 2023-09-12T16:03:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.425+0000 7f8920ff9700 1 -- 172.21.15.130:0/2425838215 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f891c004000 con 0x7f8924104530 2023-09-12T16:03:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.426+0000 7f8920ff9700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f890c0a0820 0x7f890c0a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.426+0000 7f8920ff9700 1 -- 172.21.15.130:0/2425838215 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f891c0cdc20 con 0x7f8924104530 2023-09-12T16:03:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.426+0000 7f89237fe700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f890c0a0820 0x7f890c0a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.426+0000 7f89237fe700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f890c0a0820 0x7f890c0a2ce0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8924069c10 tx=0x7f891400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.431+0000 7f8920ff9700 1 -- 172.21.15.130:0/2425838215 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f891c097390 con 0x7f8924104530 2023-09-12T16:03:01.494 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:01.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.467+0000 7f572db7f700 1 -- 172.21.15.130:0/1071486817 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f57280ff4b0 msgr2=0x7f572810d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.467+0000 7f572db7f700 1 --2- 172.21.15.130:0/1071486817 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f57280ff4b0 0x7f572810d4e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f572000b0a0 tx=0x7f572000b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.467+0000 7f572db7f700 1 -- 172.21.15.130:0/1071486817 shutdown_connections 2023-09-12T16:03:01.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.467+0000 7f572db7f700 1 --2- 172.21.15.130:0/1071486817 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f57280ff4b0 0x7f572810d4e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.467+0000 7f572db7f700 1 --2- 172.21.15.130:0/1071486817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5728106610 0x7f57281069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.467+0000 7f572db7f700 1 --2- 172.21.15.130:0/1071486817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57280feaf0 0x7f57280fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.467+0000 7f572db7f700 1 -- 172.21.15.130:0/1071486817 >> 172.21.15.130:0/1071486817 conn(0x7f5728075a70 msgr2=0x7f57280fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.467+0000 7f572db7f700 1 -- 172.21.15.130:0/1071486817 shutdown_connections 2023-09-12T16:03:01.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.467+0000 7f572db7f700 1 -- 172.21.15.130:0/1071486817 wait complete. 2023-09-12T16:03:01.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.468+0000 7f572db7f700 1 Processor -- start 2023-09-12T16:03:01.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.468+0000 7f572db7f700 1 -- start start 2023-09-12T16:03:01.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f572db7f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57280feaf0 0x7f572819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f572db7f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57280ff4b0 0x7f572819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f572db7f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5728106610 0x7f57281966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f572db7f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5728196d20 con 0x7f5728106610 2023-09-12T16:03:01.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f572db7f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5728196e90 con 0x7f57280feaf0 2023-09-12T16:03:01.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f572db7f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5728197050 con 0x7f57280ff4b0 2023-09-12T16:03:01.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5727fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5728106610 0x7f57281966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5727fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5728106610 0x7f57281966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54234/0 (socket says 172.21.15.130:54234) 2023-09-12T16:03:01.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5727fff700 1 -- 172.21.15.130:0/2310273573 learned_addr learned my addr 172.21.15.130:0/2310273573 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:01.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5726ffd700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57280ff4b0 0x7f572819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f57277fe700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57280feaf0 0x7f572819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5727fff700 1 -- 172.21.15.130:0/2310273573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57280ff4b0 msgr2=0x7f572819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5727fff700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57280ff4b0 0x7f572819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5727fff700 1 -- 172.21.15.130:0/2310273573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57280feaf0 msgr2=0x7f572819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5727fff700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57280feaf0 0x7f572819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5727fff700 1 -- 172.21.15.130:0/2310273573 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720009d00 con 0x7f5728106610 2023-09-12T16:03:01.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f57277fe700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57280feaf0 0x7f572819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:01.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5727fff700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5728106610 0x7f57281966b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f572000b780 tx=0x7f5720007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5724ff9700 1 -- 172.21.15.130:0/2310273573 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572000e040 con 0x7f5728106610 2023-09-12T16:03:01.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57281972d0 con 0x7f5728106610 2023-09-12T16:03:01.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5724ff9700 1 -- 172.21.15.130:0/2310273573 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f572000be40 con 0x7f5728106610 2023-09-12T16:03:01.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f5724ff9700 1 -- 172.21.15.130:0/2310273573 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57200052a0 con 0x7f5728106610 2023-09-12T16:03:01.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.469+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57281a3620 con 0x7f5728106610 2023-09-12T16:03:01.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.470+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572804e300 con 0x7f5728106610 2023-09-12T16:03:01.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.472+0000 7f5724ff9700 1 -- 172.21.15.130:0/2310273573 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f572001d470 con 0x7f5728106610 2023-09-12T16:03:01.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.473+0000 7f5724ff9700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f57100a06d0 0x7f57100a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:01.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.473+0000 7f5724ff9700 1 -- 172.21.15.130:0/2310273573 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f57200cdaf0 con 0x7f5728106610 2023-09-12T16:03:01.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.473+0000 7f57277fe700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f57100a06d0 0x7f57100a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:01.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.473+0000 7f57277fe700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f57100a06d0 0x7f57100a2b90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5718009bc0 tx=0x7f5718009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:01.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.476+0000 7f5724ff9700 1 -- 172.21.15.130:0/2310273573 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f57200971e0 con 0x7f5728106610 2023-09-12T16:03:01.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.648+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f892410f520 con 0x7f8924104530 2023-09-12T16:03:01.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.649+0000 7f8920ff9700 1 -- 172.21.15.130:0/2425838215 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f891c021020 con 0x7f8924104530 2023-09-12T16:03:01.649 INFO:teuthology.orchestra.run.smithi130.stdout:188978561068 2023-09-12T16:03:01.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f890c0a0820 msgr2=0x7f890c0a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f890c0a0820 0x7f890c0a2ce0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8924069c10 tx=0x7f891400b410 comp rx=0 tx=0).stop 2023-09-12T16:03:01.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8924104530 msgr2=0x7f8924101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8924104530 0x7f8924101c40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b830 tx=0x7f891c009450 comp rx=0 tx=0).stop 2023-09-12T16:03:01.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 shutdown_connections 2023-09-12T16:03:01.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f890c0a0820 0x7f890c0a2ce0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8924104530 0x7f8924101c40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89240ff520 0x7f8924101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 --2- 172.21.15.130:0/2425838215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89240ffee0 0x7f89241016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 >> 172.21.15.130:0/2425838215 conn(0x7f8924076600 msgr2=0x7f8924068120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 shutdown_connections 2023-09-12T16:03:01.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.651+0000 7f892a262700 1 -- 172.21.15.130:0/2425838215 wait complete. 2023-09-12T16:03:01.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.686+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5728197bc0 con 0x7f5728106610 2023-09-12T16:03:01.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.687+0000 7f5724ff9700 1 -- 172.21.15.130:0/2310273573 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5720004c50 con 0x7f5728106610 2023-09-12T16:03:01.687 INFO:teuthology.orchestra.run.smithi130.stdout:34359738436 2023-09-12T16:03:01.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f57100a06d0 msgr2=0x7f57100a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f57100a06d0 0x7f57100a2b90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5718009bc0 tx=0x7f5718009260 comp rx=0 tx=0).stop 2023-09-12T16:03:01.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5728106610 msgr2=0x7f57281966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:01.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5728106610 0x7f57281966b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f572000b780 tx=0x7f5720007990 comp rx=0 tx=0).stop 2023-09-12T16:03:01.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 shutdown_connections 2023-09-12T16:03:01.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f57100a06d0 0x7f57100a2b90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5728106610 0x7f57281966b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57280feaf0 0x7f572819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 --2- 172.21.15.130:0/2310273573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57280ff4b0 0x7f572819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:01.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.689+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 >> 172.21.15.130:0/2310273573 conn(0x7f5728075a70 msgr2=0x7f57281041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:01.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.690+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 shutdown_connections 2023-09-12T16:03:01.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:01.690+0000 7f572db7f700 1 -- 172.21.15.130:0/2310273573 wait complete. 2023-09-12T16:03:01.890 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776151 got 292057776153 for osd.10 2023-09-12T16:03:01.890 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:02.154 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:01 smithi130 ceph-mon[98670]: pgmap v293: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 2 op/s; 5.9 KiB/s, 0 objects/s recovering 2023-09-12T16:03:02.154 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1954874819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-12T16:03:02.154 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/96080839' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-12T16:03:02.154 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2425838215' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-12T16:03:02.155 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:01 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2310273573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-12T16:03:02.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:01 smithi159 ceph-mon[102522]: pgmap v293: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 2 op/s; 5.9 KiB/s, 0 objects/s recovering 2023-09-12T16:03:02.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1954874819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-12T16:03:02.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/96080839' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-12T16:03:02.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2425838215' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-12T16:03:02.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:01 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2310273573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-12T16:03:02.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:01 smithi203 ceph-mon[103431]: pgmap v293: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 2 op/s; 5.9 KiB/s, 0 objects/s recovering 2023-09-12T16:03:02.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1954874819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-12T16:03:02.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/96080839' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-12T16:03:02.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2425838215' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-12T16:03:02.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:01 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2310273573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-12T16:03:02.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738434 got 34359738436 for osd.0 2023-09-12T16:03:02.395 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:02.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561065 got 188978561068 for osd.6 2023-09-12T16:03:02.439 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:02.815 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397541 got 210453397544 for osd.7 2023-09-12T16:03:02.816 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:02.834 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182453 got 107374182456 for osd.3 2023-09-12T16:03:02.835 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:03.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.997+0000 7fcd19d18700 1 -- 172.21.15.130:0/3467399840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 msgr2=0x7fcd140ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:03.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.997+0000 7fcd19d18700 1 --2- 172.21.15.130:0/3467399840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 0x7fcd140ff9a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009990 tx=0x7fcd08009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.997+0000 7fcd19d18700 1 -- 172.21.15.130:0/3467399840 shutdown_connections 2023-09-12T16:03:03.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.997+0000 7fcd19d18700 1 --2- 172.21.15.130:0/3467399840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd140ffee0 0x7fcd1410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.997+0000 7fcd19d18700 1 --2- 172.21.15.130:0/3467399840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd14104530 0x7fcd14104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.997+0000 7fcd19d18700 1 --2- 172.21.15.130:0/3467399840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 0x7fcd140ff9a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.997+0000 7fcd19d18700 1 -- 172.21.15.130:0/3467399840 >> 172.21.15.130:0/3467399840 conn(0x7fcd14076600 msgr2=0x7fcd14076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:03.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.997+0000 7fcd19d18700 1 -- 172.21.15.130:0/3467399840 shutdown_connections 2023-09-12T16:03:03.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.997+0000 7fcd19d18700 1 -- 172.21.15.130:0/3467399840 wait complete. 2023-09-12T16:03:03.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd19d18700 1 Processor -- start 2023-09-12T16:03:03.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd19d18700 1 -- start start 2023-09-12T16:03:03.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd19d18700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 0x7fcd1419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:03.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd19d18700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd140ffee0 0x7fcd1419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:03.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd19d18700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd14104530 0x7fcd14198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:03.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd19d18700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14199540 con 0x7fcd140ff520 2023-09-12T16:03:03.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd19d18700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd141996b0 con 0x7fcd14104530 2023-09-12T16:03:03.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd19d18700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14199870 con 0x7fcd140ffee0 2023-09-12T16:03:03.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd137fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 0x7fcd1419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:03.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd137fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 0x7fcd1419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54244/0 (socket says 172.21.15.130:54244) 2023-09-12T16:03:03.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd137fe700 1 -- 172.21.15.130:0/2560920438 learned_addr learned my addr 172.21.15.130:0/2560920438 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:03.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd12ffd700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd140ffee0 0x7fcd1419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:03.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.998+0000 7fcd13fff700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd14104530 0x7fcd14198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:03.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.999+0000 7fcd137fe700 1 -- 172.21.15.130:0/2560920438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd140ffee0 msgr2=0x7fcd1419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:03.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.999+0000 7fcd137fe700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd140ffee0 0x7fcd1419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.999+0000 7fcd137fe700 1 -- 172.21.15.130:0/2560920438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd14104530 msgr2=0x7fcd14198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:03.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.999+0000 7fcd137fe700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd14104530 0x7fcd14198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.999+0000 7fcd137fe700 1 -- 172.21.15.130:0/2560920438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd080095f0 con 0x7fcd140ff520 2023-09-12T16:03:03.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:02.999+0000 7fcd13fff700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd14104530 0x7fcd14198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:03.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.000+0000 7fcd137fe700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 0x7fcd1419e660 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400c8f0 tx=0x7fcd0400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:03.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.000+0000 7fcd10ff9700 1 -- 172.21.15.130:0/2560920438 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04007940 con 0x7fcd140ff520 2023-09-12T16:03:03.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.000+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd14199b50 con 0x7fcd140ff520 2023-09-12T16:03:03.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.000+0000 7fcd10ff9700 1 -- 172.21.15.130:0/2560920438 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd0400f450 con 0x7fcd140ff520 2023-09-12T16:03:03.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.000+0000 7fcd10ff9700 1 -- 172.21.15.130:0/2560920438 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04018660 con 0x7fcd140ff520 2023-09-12T16:03:03.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.000+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd141a5650 con 0x7fcd140ff520 2023-09-12T16:03:03.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.001+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1404e300 con 0x7fcd140ff520 2023-09-12T16:03:03.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.002+0000 7fcd10ff9700 1 -- 172.21.15.130:0/2560920438 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fcd0400f5c0 con 0x7fcd140ff520 2023-09-12T16:03:03.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.003+0000 7fcd10ff9700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fccfc0a06d0 0x7fccfc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:03.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.003+0000 7fcd12ffd700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fccfc0a06d0 0x7fccfc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:03.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.003+0000 7fcd10ff9700 1 -- 172.21.15.130:0/2560920438 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcd040cbf00 con 0x7fcd140ff520 2023-09-12T16:03:03.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.004+0000 7fcd12ffd700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fccfc0a06d0 0x7fccfc0a2b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd1410c5a0 tx=0x7fcd08005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:03.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.008+0000 7fcd10ff9700 1 -- 172.21.15.130:0/2560920438 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcd040955c0 con 0x7fcd140ff520 2023-09-12T16:03:03.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.206+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fcd1410f510 con 0x7fcd140ff520 2023-09-12T16:03:03.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.207+0000 7fcd10ff9700 1 -- 172.21.15.130:0/2560920438 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd04098be0 con 0x7fcd140ff520 2023-09-12T16:03:03.207 INFO:teuthology.orchestra.run.smithi130.stdout:270582939680 2023-09-12T16:03:03.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fccfc0a06d0 msgr2=0x7fccfc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:03.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fccfc0a06d0 0x7fccfc0a2b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd1410c5a0 tx=0x7fcd08005fd0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 msgr2=0x7fcd1419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:03.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 0x7fcd1419e660 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400c8f0 tx=0x7fcd0400cc00 comp rx=0 tx=0).stop 2023-09-12T16:03:03.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 shutdown_connections 2023-09-12T16:03:03.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fccfc0a06d0 0x7fccfc0a2b90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd14104530 0x7fcd14198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd140ff520 0x7fcd1419e660 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 --2- 172.21.15.130:0/2560920438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd140ffee0 0x7fcd1419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:03.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 >> 172.21.15.130:0/2560920438 conn(0x7fcd14076600 msgr2=0x7fcd141010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:03.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 shutdown_connections 2023-09-12T16:03:03.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:03.209+0000 7fcd19d18700 1 -- 172.21.15.130:0/2560920438 wait complete. 2023-09-12T16:03:03.768 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939677 got 270582939680 for osd.9 2023-09-12T16:03:03.768 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:03.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:03 smithi130 ceph-mon[98670]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 187 B/s wr, 4 op/s; 5.3 KiB/s, 0 objects/s recovering 2023-09-12T16:03:03.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:03 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2560920438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-12T16:03:04.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.013+0000 7fe4a6750700 1 -- 172.21.15.130:0/299820349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 msgr2=0x7fe4a0104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:04.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.013+0000 7fe4a6750700 1 --2- 172.21.15.130:0/299820349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 0x7fe4a0104910 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe490009990 tx=0x7fe490009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.013+0000 7fe4a6750700 1 -- 172.21.15.130:0/299820349 shutdown_connections 2023-09-12T16:03:04.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.013+0000 7fe4a6750700 1 --2- 172.21.15.130:0/299820349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4a00ffee0 0x7fe4a010c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.013+0000 7fe4a6750700 1 --2- 172.21.15.130:0/299820349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 0x7fe4a0104910 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.013+0000 7fe4a6750700 1 --2- 172.21.15.130:0/299820349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a00ff520 0x7fe4a00ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.013+0000 7fe4a6750700 1 -- 172.21.15.130:0/299820349 >> 172.21.15.130:0/299820349 conn(0x7fe4a0076600 msgr2=0x7fe4a0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:04.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.013+0000 7fe4a6750700 1 -- 172.21.15.130:0/299820349 shutdown_connections 2023-09-12T16:03:04.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.013+0000 7fe4a6750700 1 -- 172.21.15.130:0/299820349 wait complete. 2023-09-12T16:03:04.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.014+0000 7fe4a6750700 1 Processor -- start 2023-09-12T16:03:04.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.014+0000 7fe4a6750700 1 -- start start 2023-09-12T16:03:04.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.014+0000 7fe4a6750700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4a00ff520 0x7fe4a01974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:04.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.014+0000 7fe4a6750700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a00ffee0 0x7fe4a0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:04.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.014+0000 7fe4a6750700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 0x7fe4a01a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:04.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.014+0000 7fe4a6750700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0198340 con 0x7fe4a0104530 2023-09-12T16:03:04.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.014+0000 7fe4a6750700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a01a2ab0 con 0x7fe4a00ffee0 2023-09-12T16:03:04.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.014+0000 7fe4a6750700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a01a2c20 con 0x7fe4a00ff520 2023-09-12T16:03:04.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a4ced700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 0x7fe4a01a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:04.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a4ced700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 0x7fe4a01a2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54262/0 (socket says 172.21.15.130:54262) 2023-09-12T16:03:04.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a4ced700 1 -- 172.21.15.130:0/2128474034 learned_addr learned my addr 172.21.15.130:0/2128474034 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:04.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe49ffff700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4a00ff520 0x7fe4a01974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:04.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe49f7fe700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a00ffee0 0x7fe4a0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:04.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a4ced700 1 -- 172.21.15.130:0/2128474034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4a00ff520 msgr2=0x7fe4a01974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:04.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a4ced700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4a00ff520 0x7fe4a01974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a4ced700 1 -- 172.21.15.130:0/2128474034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a00ffee0 msgr2=0x7fe4a0197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:04.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a4ced700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a00ffee0 0x7fe4a0197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a4ced700 1 -- 172.21.15.130:0/2128474034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900095f0 con 0x7fe4a0104530 2023-09-12T16:03:04.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a4ced700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 0x7fe4a01a2570 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe49800ab50 tx=0x7fe49800ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:04.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe49d7fa700 1 -- 172.21.15.130:0/2128474034 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49800f070 con 0x7fe4a0104530 2023-09-12T16:03:04.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a01a2f00 con 0x7fe4a0104530 2023-09-12T16:03:04.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe49d7fa700 1 -- 172.21.15.130:0/2128474034 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe498007a80 con 0x7fe4a0104530 2023-09-12T16:03:04.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe49d7fa700 1 -- 172.21.15.130:0/2128474034 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498005280 con 0x7fe4a0104530 2023-09-12T16:03:04.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.015+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a01a3420 con 0x7fe4a0104530 2023-09-12T16:03:04.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.016+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a004e300 con 0x7fe4a0104530 2023-09-12T16:03:04.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.017+0000 7fe49d7fa700 1 -- 172.21.15.130:0/2128474034 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fe49800c620 con 0x7fe4a0104530 2023-09-12T16:03:04.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.018+0000 7fe49d7fa700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4880a06d0 0x7fe4880a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:04.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.018+0000 7fe49ffff700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4880a06d0 0x7fe4880a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:04.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.018+0000 7fe49d7fa700 1 -- 172.21.15.130:0/2128474034 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe4980cbae0 con 0x7fe4a0104530 2023-09-12T16:03:04.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.018+0000 7fe49ffff700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4880a06d0 0x7fe4880a2b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe49000b5c0 tx=0x7fe490005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:04.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.022+0000 7fe49d7fa700 1 -- 172.21.15.130:0/2128474034 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe4980951d0 con 0x7fe4a0104530 2023-09-12T16:03:04.211 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.211+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe4a010f6c0 con 0x7fe4a0104530 2023-09-12T16:03:04.211 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.212+0000 7fe49d7fa700 1 -- 172.21.15.130:0/2128474034 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4980987f0 con 0x7fe4a0104530 2023-09-12T16:03:04.212 INFO:teuthology.orchestra.run.smithi130.stdout:55834574912 2023-09-12T16:03:04.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4880a06d0 msgr2=0x7fe4880a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:04.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4880a06d0 0x7fe4880a2b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe49000b5c0 tx=0x7fe490005fd0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 msgr2=0x7fe4a01a2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:04.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 0x7fe4a01a2570 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe49800ab50 tx=0x7fe49800ae60 comp rx=0 tx=0).stop 2023-09-12T16:03:04.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 shutdown_connections 2023-09-12T16:03:04.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4880a06d0 0x7fe4880a2b90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4a0104530 0x7fe4a01a2570 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4a00ff520 0x7fe4a01974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 --2- 172.21.15.130:0/2128474034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a00ffee0 0x7fe4a0197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:04.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 >> 172.21.15.130:0/2128474034 conn(0x7fe4a0076600 msgr2=0x7fe4a00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:04.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 shutdown_connections 2023-09-12T16:03:04.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:04.214+0000 7fe4a6750700 1 -- 172.21.15.130:0/2128474034 wait complete. 2023-09-12T16:03:04.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:03 smithi159 ceph-mon[102522]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 187 B/s wr, 4 op/s; 5.3 KiB/s, 0 objects/s recovering 2023-09-12T16:03:04.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:03 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2560920438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-12T16:03:04.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:03 smithi203 ceph-mon[103431]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 187 B/s wr, 4 op/s; 5.3 KiB/s, 0 objects/s recovering 2023-09-12T16:03:04.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:03 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2560920438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-12T16:03:04.578 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574910 got 55834574912 for osd.1 2023-09-12T16:03:04.579 DEBUG:teuthology.parallel:result is None 2023-09-12T16:03:04.579 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-09-12T16:03:04.580 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph pg dump --format=json 2023-09-12T16:03:05.136 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:05.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:04 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2128474034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-12T16:03:05.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:04 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2128474034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-12T16:03:05.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:04 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2128474034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-12T16:03:06.134 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:05 smithi130 ceph-mon[98670]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 170 B/s wr, 4 op/s; 4.9 KiB/s, 0 objects/s recovering 2023-09-12T16:03:06.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:05 smithi159 ceph-mon[102522]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 170 B/s wr, 4 op/s; 4.9 KiB/s, 0 objects/s recovering 2023-09-12T16:03:06.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:05 smithi203 ceph-mon[103431]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 170 B/s wr, 4 op/s; 4.9 KiB/s, 0 objects/s recovering 2023-09-12T16:03:06.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.346+0000 7f3a1f935700 1 -- 172.21.15.130:0/2900939974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 msgr2=0x7f3a181069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:06.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.346+0000 7f3a1f935700 1 --2- 172.21.15.130:0/2900939974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 0x7f3a181069f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009990 tx=0x7f3a08009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.346+0000 7f3a1f935700 1 -- 172.21.15.130:0/2900939974 shutdown_connections 2023-09-12T16:03:06.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.346+0000 7f3a1f935700 1 --2- 172.21.15.130:0/2900939974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a180ff530 0x7f3a1810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.346+0000 7f3a1f935700 1 --2- 172.21.15.130:0/2900939974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 0x7f3a181069f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.346+0000 7f3a1f935700 1 --2- 172.21.15.130:0/2900939974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a180feb70 0x7f3a180feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.346+0000 7f3a1f935700 1 -- 172.21.15.130:0/2900939974 >> 172.21.15.130:0/2900939974 conn(0x7f3a18075a70 msgr2=0x7f3a180fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:06.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.346+0000 7f3a1f935700 1 -- 172.21.15.130:0/2900939974 shutdown_connections 2023-09-12T16:03:06.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.346+0000 7f3a1f935700 1 -- 172.21.15.130:0/2900939974 wait complete. 2023-09-12T16:03:06.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.347+0000 7f3a1f935700 1 Processor -- start 2023-09-12T16:03:06.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.347+0000 7f3a1f935700 1 -- start start 2023-09-12T16:03:06.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.347+0000 7f3a1f935700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a180feb70 0x7f3a181966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:06.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.347+0000 7f3a1f935700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a180ff530 0x7f3a18196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:06.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.347+0000 7f3a1f935700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 0x7f3a18199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:06.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.347+0000 7f3a1f935700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1819a5f0 con 0x7f3a18106610 2023-09-12T16:03:06.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1f935700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1819a760 con 0x7f3a180ff530 2023-09-12T16:03:06.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1f935700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1819a920 con 0x7f3a180feb70 2023-09-12T16:03:06.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ded2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 0x7f3a18199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:06.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ded2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 0x7f3a18199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54274/0 (socket says 172.21.15.130:54274) 2023-09-12T16:03:06.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ded2700 1 -- 172.21.15.130:0/309767662 learned_addr learned my addr 172.21.15.130:0/309767662 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:06.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1d6d1700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a180feb70 0x7f3a181966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:06.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ced0700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a180ff530 0x7f3a18196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:06.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ded2700 1 -- 172.21.15.130:0/309767662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a180feb70 msgr2=0x7f3a181966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:06.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ded2700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a180feb70 0x7f3a181966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ded2700 1 -- 172.21.15.130:0/309767662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a180ff530 msgr2=0x7f3a18196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:06.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ded2700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a180ff530 0x7f3a18196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ded2700 1 -- 172.21.15.130:0/309767662 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a080095f0 con 0x7f3a18106610 2023-09-12T16:03:06.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1ded2700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 0x7f3a18199ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3a10007ee0 tx=0x7f3a1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:06.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a0e7fc700 1 -- 172.21.15.130:0/309767662 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10017070 con 0x7f3a18106610 2023-09-12T16:03:06.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a0e7fc700 1 -- 172.21.15.130:0/309767662 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1000f040 con 0x7f3a18106610 2023-09-12T16:03:06.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1819ac00 con 0x7f3a18106610 2023-09-12T16:03:06.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a0e7fc700 1 -- 172.21.15.130:0/309767662 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10005580 con 0x7f3a18106610 2023-09-12T16:03:06.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.348+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a181a3550 con 0x7f3a18106610 2023-09-12T16:03:06.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.349+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1804e300 con 0x7f3a18106610 2023-09-12T16:03:06.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.350+0000 7f3a0e7fc700 1 -- 172.21.15.130:0/309767662 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f3a1000db20 con 0x7f3a18106610 2023-09-12T16:03:06.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.351+0000 7f3a0e7fc700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3a040a0750 0x7f3a040a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:06.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.351+0000 7f3a1d6d1700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3a040a0750 0x7f3a040a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:06.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.351+0000 7f3a0e7fc700 1 -- 172.21.15.130:0/309767662 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3a100cb680 con 0x7f3a18106610 2023-09-12T16:03:06.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.351+0000 7f3a1d6d1700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3a040a0750 0x7f3a040a2c10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b5c0 tx=0x7f3a08005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:06.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.355+0000 7f3a0e7fc700 1 -- 172.21.15.130:0/309767662 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3a10094d40 con 0x7f3a18106610 2023-09-12T16:03:06.561 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.561+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a18110740 con 0x7f3a040a0750 2023-09-12T16:03:06.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.577+0000 7f3a0e7fc700 1 -- 172.21.15.130:0/309767662 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423331 (secure 0 0 0) 0x7f3a18110740 con 0x7f3a040a0750 2023-09-12T16:03:06.582 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:06.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3a040a0750 msgr2=0x7f3a040a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:06.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3a040a0750 0x7f3a040a2c10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b5c0 tx=0x7f3a08005fd0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 msgr2=0x7f3a18199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:06.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 0x7f3a18199ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3a10007ee0 tx=0x7f3a1000d3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 shutdown_connections 2023-09-12T16:03:06.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3a040a0750 0x7f3a040a2c10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a18106610 0x7f3a18199ff0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a180feb70 0x7f3a181966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 --2- 172.21.15.130:0/309767662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a180ff530 0x7f3a18196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:06.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 >> 172.21.15.130:0/309767662 conn(0x7f3a18075a70 msgr2=0x7f3a18103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:06.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 shutdown_connections 2023-09-12T16:03:06.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:06.581+0000 7f3a1f935700 1 -- 172.21.15.130:0/309767662 wait complete. 2023-09-12T16:03:06.588 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2023-09-12T16:03:06.996 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-09-12T16:03:04.771432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655889,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":358,"num_read_kb":291,"num_write":337,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1240551,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335968,"kb_used_data":11696,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605856,"statfs":{"total":1151940427776,"available":1151596396544,"internally_reserved":0,"allocated":11976704,"data_stored":7814264,"data_compressed":363759,"data_compressed_allocated":491520,"data_compressed_original":2514944,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":25,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":59815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003428"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025907+0000","last_change":"2023-09-12T16:02:05.039617+0000","last_active":"2023-09-12T16:02:51.025907+0000","last_peered":"2023-09-12T16:02:51.025907+0000","last_clean":"2023-09-12T16:02:51.025907+0000","last_became_active":"2023-09-12T16:02:05.039264+0000","last_became_peered":"2023-09-12T16:02:05.039264+0000","last_unstale":"2023-09-12T16:02:51.025907+0000","last_undegraded":"2023-09-12T16:02:51.025907+0000","last_fullsized":"2023-09-12T16:02:51.025907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:37:01.381406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829291+0000","last_change":"2023-09-12T16:02:05.033151+0000","last_active":"2023-09-12T16:02:50.829291+0000","last_peered":"2023-09-12T16:02:50.829291+0000","last_clean":"2023-09-12T16:02:50.829291+0000","last_became_active":"2023-09-12T16:02:05.032866+0000","last_became_peered":"2023-09-12T16:02:05.032866+0000","last_unstale":"2023-09-12T16:02:50.829291+0000","last_undegraded":"2023-09-12T16:02:50.829291+0000","last_fullsized":"2023-09-12T16:02:50.829291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:35.921428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.032478+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.032124+0000","last_became_peered":"2023-09-12T16:02:05.032124+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:58:54.112577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835786+0000","last_change":"2023-09-12T16:02:05.033088+0000","last_active":"2023-09-12T16:02:50.835786+0000","last_peered":"2023-09-12T16:02:50.835786+0000","last_clean":"2023-09-12T16:02:50.835786+0000","last_became_active":"2023-09-12T16:02:05.032332+0000","last_became_peered":"2023-09-12T16:02:05.032332+0000","last_unstale":"2023-09-12T16:02:50.835786+0000","last_undegraded":"2023-09-12T16:02:50.835786+0000","last_fullsized":"2023-09-12T16:02:50.835786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:25:38.751961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:05.036612+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:05.036215+0000","last_became_peered":"2023-09-12T16:02:05.036215+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:15:04.561751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832126+0000","last_change":"2023-09-12T16:02:05.043816+0000","last_active":"2023-09-12T16:02:50.832126+0000","last_peered":"2023-09-12T16:02:50.832126+0000","last_clean":"2023-09-12T16:02:50.832126+0000","last_became_active":"2023-09-12T16:02:05.043599+0000","last_became_peered":"2023-09-12T16:02:05.043599+0000","last_unstale":"2023-09-12T16:02:50.832126+0000","last_undegraded":"2023-09-12T16:02:50.832126+0000","last_fullsized":"2023-09-12T16:02:50.832126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:32:00.286196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829345+0000","last_change":"2023-09-12T16:02:05.031476+0000","last_active":"2023-09-12T16:02:50.829345+0000","last_peered":"2023-09-12T16:02:50.829345+0000","last_clean":"2023-09-12T16:02:50.829345+0000","last_became_active":"2023-09-12T16:02:05.031150+0000","last_became_peered":"2023-09-12T16:02:05.031150+0000","last_unstale":"2023-09-12T16:02:50.829345+0000","last_undegraded":"2023-09-12T16:02:50.829345+0000","last_fullsized":"2023-09-12T16:02:50.829345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:22:27.405505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831825+0000","last_change":"2023-09-12T16:02:05.031981+0000","last_active":"2023-09-12T16:02:50.831825+0000","last_peered":"2023-09-12T16:02:50.831825+0000","last_clean":"2023-09-12T16:02:50.831825+0000","last_became_active":"2023-09-12T16:02:05.031692+0000","last_became_peered":"2023-09-12T16:02:05.031692+0000","last_unstale":"2023-09-12T16:02:50.831825+0000","last_undegraded":"2023-09-12T16:02:50.831825+0000","last_fullsized":"2023-09-12T16:02:50.831825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:51:00.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838117+0000","last_change":"2023-09-12T16:02:05.029898+0000","last_active":"2023-09-12T16:02:50.838117+0000","last_peered":"2023-09-12T16:02:50.838117+0000","last_clean":"2023-09-12T16:02:50.838117+0000","last_became_active":"2023-09-12T16:02:05.029731+0000","last_became_peered":"2023-09-12T16:02:05.029731+0000","last_unstale":"2023-09-12T16:02:50.838117+0000","last_undegraded":"2023-09-12T16:02:50.838117+0000","last_fullsized":"2023-09-12T16:02:50.838117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:16:31.194561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833228+0000","last_change":"2023-09-12T16:02:05.044846+0000","last_active":"2023-09-12T16:02:50.833228+0000","last_peered":"2023-09-12T16:02:50.833228+0000","last_clean":"2023-09-12T16:02:50.833228+0000","last_became_active":"2023-09-12T16:02:05.044430+0000","last_became_peered":"2023-09-12T16:02:05.044430+0000","last_unstale":"2023-09-12T16:02:50.833228+0000","last_undegraded":"2023-09-12T16:02:50.833228+0000","last_fullsized":"2023-09-12T16:02:50.833228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:15.999080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830392+0000","last_change":"2023-09-12T16:02:05.030285+0000","last_active":"2023-09-12T16:02:50.830392+0000","last_peered":"2023-09-12T16:02:50.830392+0000","last_clean":"2023-09-12T16:02:50.830392+0000","last_became_active":"2023-09-12T16:02:05.029585+0000","last_became_peered":"2023-09-12T16:02:05.029585+0000","last_unstale":"2023-09-12T16:02:50.830392+0000","last_undegraded":"2023-09-12T16:02:50.830392+0000","last_fullsized":"2023-09-12T16:02:50.830392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:52:27.471412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835453+0000","last_change":"2023-09-12T16:02:05.037258+0000","last_active":"2023-09-12T16:02:50.835453+0000","last_peered":"2023-09-12T16:02:50.835453+0000","last_clean":"2023-09-12T16:02:50.835453+0000","last_became_active":"2023-09-12T16:02:05.036941+0000","last_became_peered":"2023-09-12T16:02:05.036941+0000","last_unstale":"2023-09-12T16:02:50.835453+0000","last_undegraded":"2023-09-12T16:02:50.835453+0000","last_fullsized":"2023-09-12T16:02:50.835453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:04:17.998927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835219+0000","last_change":"2023-09-12T16:02:05.041986+0000","last_active":"2023-09-12T16:02:50.835219+0000","last_peered":"2023-09-12T16:02:50.835219+0000","last_clean":"2023-09-12T16:02:50.835219+0000","last_became_active":"2023-09-12T16:02:05.041584+0000","last_became_peered":"2023-09-12T16:02:05.041584+0000","last_unstale":"2023-09-12T16:02:50.835219+0000","last_undegraded":"2023-09-12T16:02:50.835219+0000","last_fullsized":"2023-09-12T16:02:50.835219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:39.065525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828785+0000","last_change":"2023-09-12T16:02:05.030405+0000","last_active":"2023-09-12T16:02:50.828785+0000","last_peered":"2023-09-12T16:02:50.828785+0000","last_clean":"2023-09-12T16:02:50.828785+0000","last_became_active":"2023-09-12T16:02:05.029488+0000","last_became_peered":"2023-09-12T16:02:05.029488+0000","last_unstale":"2023-09-12T16:02:50.828785+0000","last_undegraded":"2023-09-12T16:02:50.828785+0000","last_fullsized":"2023-09-12T16:02:50.828785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.646069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829401+0000","last_change":"2023-09-12T16:02:05.032576+0000","last_active":"2023-09-12T16:02:50.829401+0000","last_peered":"2023-09-12T16:02:50.829401+0000","last_clean":"2023-09-12T16:02:50.829401+0000","last_became_active":"2023-09-12T16:02:05.032277+0000","last_became_peered":"2023-09-12T16:02:05.032277+0000","last_unstale":"2023-09-12T16:02:50.829401+0000","last_undegraded":"2023-09-12T16:02:50.829401+0000","last_fullsized":"2023-09-12T16:02:50.829401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:47:32.734830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836039+0000","last_change":"2023-09-12T16:02:05.023468+0000","last_active":"2023-09-12T16:02:50.836039+0000","last_peered":"2023-09-12T16:02:50.836039+0000","last_clean":"2023-09-12T16:02:50.836039+0000","last_became_active":"2023-09-12T16:02:05.023132+0000","last_became_peered":"2023-09-12T16:02:05.023132+0000","last_unstale":"2023-09-12T16:02:50.836039+0000","last_undegraded":"2023-09-12T16:02:50.836039+0000","last_fullsized":"2023-09-12T16:02:50.836039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:21:49.378779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025810+0000","last_change":"2023-09-12T16:02:05.039910+0000","last_active":"2023-09-12T16:02:51.025810+0000","last_peered":"2023-09-12T16:02:51.025810+0000","last_clean":"2023-09-12T16:02:51.025810+0000","last_became_active":"2023-09-12T16:02:05.039605+0000","last_became_peered":"2023-09-12T16:02:05.039605+0000","last_unstale":"2023-09-12T16:02:51.025810+0000","last_undegraded":"2023-09-12T16:02:51.025810+0000","last_fullsized":"2023-09-12T16:02:51.025810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:19:58.823471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833500+0000","last_change":"2023-09-12T16:02:05.031667+0000","last_active":"2023-09-12T16:02:50.833500+0000","last_peered":"2023-09-12T16:02:50.833500+0000","last_clean":"2023-09-12T16:02:50.833500+0000","last_became_active":"2023-09-12T16:02:05.030886+0000","last_became_peered":"2023-09-12T16:02:05.030886+0000","last_unstale":"2023-09-12T16:02:50.833500+0000","last_undegraded":"2023-09-12T16:02:50.833500+0000","last_fullsized":"2023-09-12T16:02:50.833500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:24:02.312902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836131+0000","last_change":"2023-09-12T16:02:05.037607+0000","last_active":"2023-09-12T16:02:50.836131+0000","last_peered":"2023-09-12T16:02:50.836131+0000","last_clean":"2023-09-12T16:02:50.836131+0000","last_became_active":"2023-09-12T16:02:05.037095+0000","last_became_peered":"2023-09-12T16:02:05.037095+0000","last_unstale":"2023-09-12T16:02:50.836131+0000","last_undegraded":"2023-09-12T16:02:50.836131+0000","last_fullsized":"2023-09-12T16:02:50.836131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:17:04.886737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834805+0000","last_change":"2023-09-12T16:02:09.292093+0000","last_active":"2023-09-12T16:02:50.834805+0000","last_peered":"2023-09-12T16:02:50.834805+0000","last_clean":"2023-09-12T16:02:50.834805+0000","last_became_active":"2023-09-12T16:02:09.291602+0000","last_became_peered":"2023-09-12T16:02:09.291602+0000","last_unstale":"2023-09-12T16:02:50.834805+0000","last_undegraded":"2023-09-12T16:02:50.834805+0000","last_fullsized":"2023-09-12T16:02:50.834805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:36:23.089498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.857296+0000","last_change":"2023-09-12T16:02:51.857296+0000","last_active":"2023-09-12T16:02:51.857296+0000","last_peered":"2023-09-12T16:02:51.857296+0000","last_clean":"2023-09-12T16:02:51.857296+0000","last_became_active":"2023-09-12T16:02:51.843961+0000","last_became_peered":"2023-09-12T16:02:51.843961+0000","last_unstale":"2023-09-12T16:02:51.857296+0000","last_undegraded":"2023-09-12T16:02:51.857296+0000","last_fullsized":"2023-09-12T16:02:51.857296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:55:57.938037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832962+0000","last_change":"2023-09-12T16:02:05.032191+0000","last_active":"2023-09-12T16:02:50.832962+0000","last_peered":"2023-09-12T16:02:50.832962+0000","last_clean":"2023-09-12T16:02:50.832962+0000","last_became_active":"2023-09-12T16:02:05.031888+0000","last_became_peered":"2023-09-12T16:02:05.031888+0000","last_unstale":"2023-09-12T16:02:50.832962+0000","last_undegraded":"2023-09-12T16:02:50.832962+0000","last_fullsized":"2023-09-12T16:02:50.832962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:50:15.775078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834369+0000","last_change":"2023-09-12T16:02:09.290650+0000","last_active":"2023-09-12T16:02:50.834369+0000","last_peered":"2023-09-12T16:02:50.834369+0000","last_clean":"2023-09-12T16:02:50.834369+0000","last_became_active":"2023-09-12T16:02:09.290205+0000","last_became_peered":"2023-09-12T16:02:09.290205+0000","last_unstale":"2023-09-12T16:02:50.834369+0000","last_undegraded":"2023-09-12T16:02:50.834369+0000","last_fullsized":"2023-09-12T16:02:50.834369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:44:04.930063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833124+0000","last_change":"2023-09-12T16:02:04.032506+0000","last_active":"2023-09-12T16:02:50.833124+0000","last_peered":"2023-09-12T16:02:50.833124+0000","last_clean":"2023-09-12T16:02:50.833124+0000","last_became_active":"2023-09-12T16:02:04.032047+0000","last_became_peered":"2023-09-12T16:02:04.032047+0000","last_unstale":"2023-09-12T16:02:50.833124+0000","last_undegraded":"2023-09-12T16:02:50.833124+0000","last_fullsized":"2023-09-12T16:02:50.833124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:30:22.664465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.025692+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.025370+0000","last_became_peered":"2023-09-12T16:02:05.025370+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.305375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.068789+0000","last_change":"2023-09-12T16:02:52.068789+0000","last_active":"2023-09-12T16:02:52.068789+0000","last_peered":"2023-09-12T16:02:52.068789+0000","last_clean":"2023-09-12T16:02:52.068789+0000","last_became_active":"2023-09-12T16:02:52.068276+0000","last_became_peered":"2023-09-12T16:02:52.068276+0000","last_unstale":"2023-09-12T16:02:52.068789+0000","last_undegraded":"2023-09-12T16:02:52.068789+0000","last_fullsized":"2023-09-12T16:02:52.068789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:19:19.009173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831481+0000","last_change":"2023-09-12T16:02:04.032079+0000","last_active":"2023-09-12T16:02:50.831481+0000","last_peered":"2023-09-12T16:02:50.831481+0000","last_clean":"2023-09-12T16:02:50.831481+0000","last_became_active":"2023-09-12T16:02:04.031835+0000","last_became_peered":"2023-09-12T16:02:04.031835+0000","last_unstale":"2023-09-12T16:02:50.831481+0000","last_undegraded":"2023-09-12T16:02:50.831481+0000","last_fullsized":"2023-09-12T16:02:50.831481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:48:33.935012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837888+0000","last_change":"2023-09-12T16:02:05.037671+0000","last_active":"2023-09-12T16:02:50.837888+0000","last_peered":"2023-09-12T16:02:50.837888+0000","last_clean":"2023-09-12T16:02:50.837888+0000","last_became_active":"2023-09-12T16:02:05.037307+0000","last_became_peered":"2023-09-12T16:02:05.037307+0000","last_unstale":"2023-09-12T16:02:50.837888+0000","last_undegraded":"2023-09-12T16:02:50.837888+0000","last_fullsized":"2023-09-12T16:02:50.837888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:47:23.836612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834100+0000","last_change":"2023-09-12T16:02:09.289165+0000","last_active":"2023-09-12T16:02:50.834100+0000","last_peered":"2023-09-12T16:02:50.834100+0000","last_clean":"2023-09-12T16:02:50.834100+0000","last_became_active":"2023-09-12T16:02:09.288903+0000","last_became_peered":"2023-09-12T16:02:09.288903+0000","last_unstale":"2023-09-12T16:02:50.834100+0000","last_undegraded":"2023-09-12T16:02:50.834100+0000","last_fullsized":"2023-09-12T16:02:50.834100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:27:09.160822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831262+0000","last_change":"2023-09-12T16:02:04.027279+0000","last_active":"2023-09-12T16:02:50.831262+0000","last_peered":"2023-09-12T16:02:50.831262+0000","last_clean":"2023-09-12T16:02:50.831262+0000","last_became_active":"2023-09-12T16:02:04.026975+0000","last_became_peered":"2023-09-12T16:02:04.026975+0000","last_unstale":"2023-09-12T16:02:50.831262+0000","last_undegraded":"2023-09-12T16:02:50.831262+0000","last_fullsized":"2023-09-12T16:02:50.831262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:45.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832885+0000","last_change":"2023-09-12T16:02:05.027582+0000","last_active":"2023-09-12T16:02:50.832885+0000","last_peered":"2023-09-12T16:02:50.832885+0000","last_clean":"2023-09-12T16:02:50.832885+0000","last_became_active":"2023-09-12T16:02:05.027236+0000","last_became_peered":"2023-09-12T16:02:05.027236+0000","last_unstale":"2023-09-12T16:02:50.832885+0000","last_undegraded":"2023-09-12T16:02:50.832885+0000","last_fullsized":"2023-09-12T16:02:50.832885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:06:24.812731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836055+0000","last_change":"2023-09-12T16:02:09.284381+0000","last_active":"2023-09-12T16:02:50.836055+0000","last_peered":"2023-09-12T16:02:50.836055+0000","last_clean":"2023-09-12T16:02:50.836055+0000","last_became_active":"2023-09-12T16:02:09.283947+0000","last_became_peered":"2023-09-12T16:02:09.283947+0000","last_unstale":"2023-09-12T16:02:50.836055+0000","last_undegraded":"2023-09-12T16:02:50.836055+0000","last_fullsized":"2023-09-12T16:02:50.836055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:05:24.803781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":65,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:00.712508+0000","last_change":"2023-09-12T16:02:04.027693+0000","last_active":"2023-09-12T16:03:00.712508+0000","last_peered":"2023-09-12T16:03:00.712508+0000","last_clean":"2023-09-12T16:03:00.712508+0000","last_became_active":"2023-09-12T16:02:04.027315+0000","last_became_peered":"2023-09-12T16:02:04.027315+0000","last_unstale":"2023-09-12T16:03:00.712508+0000","last_undegraded":"2023-09-12T16:03:00.712508+0000","last_fullsized":"2023-09-12T16:03:00.712508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:32:21.756802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831054+0000","last_change":"2023-09-12T16:02:05.031673+0000","last_active":"2023-09-12T16:02:50.831054+0000","last_peered":"2023-09-12T16:02:50.831054+0000","last_clean":"2023-09-12T16:02:50.831054+0000","last_became_active":"2023-09-12T16:02:05.031364+0000","last_became_peered":"2023-09-12T16:02:05.031364+0000","last_unstale":"2023-09-12T16:02:50.831054+0000","last_undegraded":"2023-09-12T16:02:50.831054+0000","last_fullsized":"2023-09-12T16:02:50.831054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:48:05.222530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835641+0000","last_change":"2023-09-12T16:02:09.288536+0000","last_active":"2023-09-12T16:02:50.835641+0000","last_peered":"2023-09-12T16:02:50.835641+0000","last_clean":"2023-09-12T16:02:50.835641+0000","last_became_active":"2023-09-12T16:02:09.288062+0000","last_became_peered":"2023-09-12T16:02:09.288062+0000","last_unstale":"2023-09-12T16:02:50.835641+0000","last_undegraded":"2023-09-12T16:02:50.835641+0000","last_fullsized":"2023-09-12T16:02:50.835641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:39:28.935563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832024+0000","last_change":"2023-09-12T16:02:04.026298+0000","last_active":"2023-09-12T16:02:50.832024+0000","last_peered":"2023-09-12T16:02:50.832024+0000","last_clean":"2023-09-12T16:02:50.832024+0000","last_became_active":"2023-09-12T16:02:04.025975+0000","last_became_peered":"2023-09-12T16:02:04.025975+0000","last_unstale":"2023-09-12T16:02:50.832024+0000","last_undegraded":"2023-09-12T16:02:50.832024+0000","last_fullsized":"2023-09-12T16:02:50.832024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:12:04.338497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832502+0000","last_change":"2023-09-12T16:02:05.028337+0000","last_active":"2023-09-12T16:02:50.832502+0000","last_peered":"2023-09-12T16:02:50.832502+0000","last_clean":"2023-09-12T16:02:50.832502+0000","last_became_active":"2023-09-12T16:02:05.028036+0000","last_became_peered":"2023-09-12T16:02:05.028036+0000","last_unstale":"2023-09-12T16:02:50.832502+0000","last_undegraded":"2023-09-12T16:02:50.832502+0000","last_fullsized":"2023-09-12T16:02:50.832502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:39:09.387191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025809+0000","last_change":"2023-09-12T16:02:09.290950+0000","last_active":"2023-09-12T16:02:51.025809+0000","last_peered":"2023-09-12T16:02:51.025809+0000","last_clean":"2023-09-12T16:02:51.025809+0000","last_became_active":"2023-09-12T16:02:09.290477+0000","last_became_peered":"2023-09-12T16:02:09.290477+0000","last_unstale":"2023-09-12T16:02:51.025809+0000","last_undegraded":"2023-09-12T16:02:51.025809+0000","last_fullsized":"2023-09-12T16:02:51.025809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:53:45.907575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834845+0000","last_change":"2023-09-12T16:02:04.026025+0000","last_active":"2023-09-12T16:02:50.834845+0000","last_peered":"2023-09-12T16:02:50.834845+0000","last_clean":"2023-09-12T16:02:50.834845+0000","last_became_active":"2023-09-12T16:02:04.025722+0000","last_became_peered":"2023-09-12T16:02:04.025722+0000","last_unstale":"2023-09-12T16:02:50.834845+0000","last_undegraded":"2023-09-12T16:02:50.834845+0000","last_fullsized":"2023-09-12T16:02:50.834845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:29:58.357960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831920+0000","last_change":"2023-09-12T16:02:05.028461+0000","last_active":"2023-09-12T16:02:50.831920+0000","last_peered":"2023-09-12T16:02:50.831920+0000","last_clean":"2023-09-12T16:02:50.831920+0000","last_became_active":"2023-09-12T16:02:05.028176+0000","last_became_peered":"2023-09-12T16:02:05.028176+0000","last_unstale":"2023-09-12T16:02:50.831920+0000","last_undegraded":"2023-09-12T16:02:50.831920+0000","last_fullsized":"2023-09-12T16:02:50.831920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:16:48.652513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829138+0000","last_change":"2023-09-12T16:02:09.295245+0000","last_active":"2023-09-12T16:02:50.829138+0000","last_peered":"2023-09-12T16:02:50.829138+0000","last_clean":"2023-09-12T16:02:50.829138+0000","last_became_active":"2023-09-12T16:02:09.294872+0000","last_became_peered":"2023-09-12T16:02:09.294872+0000","last_unstale":"2023-09-12T16:02:50.829138+0000","last_undegraded":"2023-09-12T16:02:50.829138+0000","last_fullsized":"2023-09-12T16:02:50.829138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:43:49.462317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832140+0000","last_change":"2023-09-12T16:02:04.030815+0000","last_active":"2023-09-12T16:02:50.832140+0000","last_peered":"2023-09-12T16:02:50.832140+0000","last_clean":"2023-09-12T16:02:50.832140+0000","last_became_active":"2023-09-12T16:02:04.030435+0000","last_became_peered":"2023-09-12T16:02:04.030435+0000","last_unstale":"2023-09-12T16:02:50.832140+0000","last_undegraded":"2023-09-12T16:02:50.832140+0000","last_fullsized":"2023-09-12T16:02:50.832140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:39:59.661143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832565+0000","last_change":"2023-09-12T16:02:05.031155+0000","last_active":"2023-09-12T16:02:50.832565+0000","last_peered":"2023-09-12T16:02:50.832565+0000","last_clean":"2023-09-12T16:02:50.832565+0000","last_became_active":"2023-09-12T16:02:05.030803+0000","last_became_peered":"2023-09-12T16:02:05.030803+0000","last_unstale":"2023-09-12T16:02:50.832565+0000","last_undegraded":"2023-09-12T16:02:50.832565+0000","last_fullsized":"2023-09-12T16:02:50.832565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:55:21.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832473+0000","last_change":"2023-09-12T16:02:09.286445+0000","last_active":"2023-09-12T16:02:50.832473+0000","last_peered":"2023-09-12T16:02:50.832473+0000","last_clean":"2023-09-12T16:02:50.832473+0000","last_became_active":"2023-09-12T16:02:09.286049+0000","last_became_peered":"2023-09-12T16:02:09.286049+0000","last_unstale":"2023-09-12T16:02:50.832473+0000","last_undegraded":"2023-09-12T16:02:50.832473+0000","last_fullsized":"2023-09-12T16:02:50.832473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:43:13.943984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":59,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:59.710267+0000","last_change":"2023-09-12T16:02:04.035341+0000","last_active":"2023-09-12T16:02:59.710267+0000","last_peered":"2023-09-12T16:02:59.710267+0000","last_clean":"2023-09-12T16:02:59.710267+0000","last_became_active":"2023-09-12T16:02:04.035092+0000","last_became_peered":"2023-09-12T16:02:04.035092+0000","last_unstale":"2023-09-12T16:02:59.710267+0000","last_undegraded":"2023-09-12T16:02:59.710267+0000","last_fullsized":"2023-09-12T16:02:59.710267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:33:00.784401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838363+0000","last_change":"2023-09-12T16:02:05.025229+0000","last_active":"2023-09-12T16:02:50.838363+0000","last_peered":"2023-09-12T16:02:50.838363+0000","last_clean":"2023-09-12T16:02:50.838363+0000","last_became_active":"2023-09-12T16:02:05.024963+0000","last_became_peered":"2023-09-12T16:02:05.024963+0000","last_unstale":"2023-09-12T16:02:50.838363+0000","last_undegraded":"2023-09-12T16:02:50.838363+0000","last_fullsized":"2023-09-12T16:02:50.838363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:46:04.318082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832595+0000","last_change":"2023-09-12T16:02:09.295011+0000","last_active":"2023-09-12T16:02:50.832595+0000","last_peered":"2023-09-12T16:02:50.832595+0000","last_clean":"2023-09-12T16:02:50.832595+0000","last_became_active":"2023-09-12T16:02:09.294617+0000","last_became_peered":"2023-09-12T16:02:09.294617+0000","last_unstale":"2023-09-12T16:02:50.832595+0000","last_undegraded":"2023-09-12T16:02:50.832595+0000","last_fullsized":"2023-09-12T16:02:50.832595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:57:53.902946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830613+0000","last_change":"2023-09-12T16:02:04.028086+0000","last_active":"2023-09-12T16:02:50.830613+0000","last_peered":"2023-09-12T16:02:50.830613+0000","last_clean":"2023-09-12T16:02:50.830613+0000","last_became_active":"2023-09-12T16:02:04.027798+0000","last_became_peered":"2023-09-12T16:02:04.027798+0000","last_unstale":"2023-09-12T16:02:50.830613+0000","last_undegraded":"2023-09-12T16:02:50.830613+0000","last_fullsized":"2023-09-12T16:02:50.830613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:11:16.921824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829002+0000","last_change":"2023-09-12T16:02:05.030691+0000","last_active":"2023-09-12T16:02:50.829002+0000","last_peered":"2023-09-12T16:02:50.829002+0000","last_clean":"2023-09-12T16:02:50.829002+0000","last_became_active":"2023-09-12T16:02:05.030379+0000","last_became_peered":"2023-09-12T16:02:05.030379+0000","last_unstale":"2023-09-12T16:02:50.829002+0000","last_undegraded":"2023-09-12T16:02:50.829002+0000","last_fullsized":"2023-09-12T16:02:50.829002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:15:05.221078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835660+0000","last_change":"2023-09-12T16:02:09.290703+0000","last_active":"2023-09-12T16:02:50.835660+0000","last_peered":"2023-09-12T16:02:50.835660+0000","last_clean":"2023-09-12T16:02:50.835660+0000","last_became_active":"2023-09-12T16:02:09.290227+0000","last_became_peered":"2023-09-12T16:02:09.290227+0000","last_unstale":"2023-09-12T16:02:50.835660+0000","last_undegraded":"2023-09-12T16:02:50.835660+0000","last_fullsized":"2023-09-12T16:02:50.835660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:07:06.182295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832824+0000","last_change":"2023-09-12T16:02:04.028630+0000","last_active":"2023-09-12T16:02:50.832824+0000","last_peered":"2023-09-12T16:02:50.832824+0000","last_clean":"2023-09-12T16:02:50.832824+0000","last_became_active":"2023-09-12T16:02:04.028319+0000","last_became_peered":"2023-09-12T16:02:04.028319+0000","last_unstale":"2023-09-12T16:02:50.832824+0000","last_undegraded":"2023-09-12T16:02:50.832824+0000","last_fullsized":"2023-09-12T16:02:50.832824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:13:43.349425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838071+0000","last_change":"2023-09-12T16:02:05.032392+0000","last_active":"2023-09-12T16:02:50.838071+0000","last_peered":"2023-09-12T16:02:50.838071+0000","last_clean":"2023-09-12T16:02:50.838071+0000","last_became_active":"2023-09-12T16:02:05.032099+0000","last_became_peered":"2023-09-12T16:02:05.032099+0000","last_unstale":"2023-09-12T16:02:50.838071+0000","last_undegraded":"2023-09-12T16:02:50.838071+0000","last_fullsized":"2023-09-12T16:02:50.838071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:46:03.108459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832745+0000","last_change":"2023-09-12T16:02:09.289573+0000","last_active":"2023-09-12T16:02:50.832745+0000","last_peered":"2023-09-12T16:02:50.832745+0000","last_clean":"2023-09-12T16:02:50.832745+0000","last_became_active":"2023-09-12T16:02:09.289179+0000","last_became_peered":"2023-09-12T16:02:09.289179+0000","last_unstale":"2023-09-12T16:02:50.832745+0000","last_undegraded":"2023-09-12T16:02:50.832745+0000","last_fullsized":"2023-09-12T16:02:50.832745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:52:35.905346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":149,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:59.856931+0000","last_change":"2023-09-12T16:02:51.851180+0000","last_active":"2023-09-12T16:02:59.856931+0000","last_peered":"2023-09-12T16:02:59.856931+0000","last_clean":"2023-09-12T16:02:59.856931+0000","last_became_active":"2023-09-12T16:02:51.841452+0000","last_became_peered":"2023-09-12T16:02:51.841452+0000","last_unstale":"2023-09-12T16:02:59.856931+0000","last_undegraded":"2023-09-12T16:02:59.856931+0000","last_fullsized":"2023-09-12T16:02:59.856931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:20:20.903637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60885,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":63,"num_read_kb":87,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":59233,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831192+0000","last_change":"2023-09-12T16:02:05.031409+0000","last_active":"2023-09-12T16:02:50.831192+0000","last_peered":"2023-09-12T16:02:50.831192+0000","last_clean":"2023-09-12T16:02:50.831192+0000","last_became_active":"2023-09-12T16:02:05.031110+0000","last_became_peered":"2023-09-12T16:02:05.031110+0000","last_unstale":"2023-09-12T16:02:50.831192+0000","last_undegraded":"2023-09-12T16:02:50.831192+0000","last_fullsized":"2023-09-12T16:02:50.831192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:10:54.482078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833098+0000","last_change":"2023-09-12T16:02:09.291493+0000","last_active":"2023-09-12T16:02:50.833098+0000","last_peered":"2023-09-12T16:02:50.833098+0000","last_clean":"2023-09-12T16:02:50.833098+0000","last_became_active":"2023-09-12T16:02:09.291044+0000","last_became_peered":"2023-09-12T16:02:09.291044+0000","last_unstale":"2023-09-12T16:02:50.833098+0000","last_undegraded":"2023-09-12T16:02:50.833098+0000","last_fullsized":"2023-09-12T16:02:50.833098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:34:30.269720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835682+0000","last_change":"2023-09-12T16:02:04.029283+0000","last_active":"2023-09-12T16:02:50.835682+0000","last_peered":"2023-09-12T16:02:50.835682+0000","last_clean":"2023-09-12T16:02:50.835682+0000","last_became_active":"2023-09-12T16:02:04.028959+0000","last_became_peered":"2023-09-12T16:02:04.028959+0000","last_unstale":"2023-09-12T16:02:50.835682+0000","last_undegraded":"2023-09-12T16:02:50.835682+0000","last_fullsized":"2023-09-12T16:02:50.835682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:28:52.750278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834981+0000","last_change":"2023-09-12T16:02:05.034140+0000","last_active":"2023-09-12T16:02:50.834981+0000","last_peered":"2023-09-12T16:02:50.834981+0000","last_clean":"2023-09-12T16:02:50.834981+0000","last_became_active":"2023-09-12T16:02:05.033840+0000","last_became_peered":"2023-09-12T16:02:05.033840+0000","last_unstale":"2023-09-12T16:02:50.834981+0000","last_undegraded":"2023-09-12T16:02:50.834981+0000","last_fullsized":"2023-09-12T16:02:50.834981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:59:55.522608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025577+0000","last_change":"2023-09-12T16:02:09.289998+0000","last_active":"2023-09-12T16:02:51.025577+0000","last_peered":"2023-09-12T16:02:51.025577+0000","last_clean":"2023-09-12T16:02:51.025577+0000","last_became_active":"2023-09-12T16:02:09.289561+0000","last_became_peered":"2023-09-12T16:02:09.289561+0000","last_unstale":"2023-09-12T16:02:51.025577+0000","last_undegraded":"2023-09-12T16:02:51.025577+0000","last_fullsized":"2023-09-12T16:02:51.025577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:09:47.327213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832456+0000","last_change":"2023-09-12T16:02:04.029883+0000","last_active":"2023-09-12T16:02:50.832456+0000","last_peered":"2023-09-12T16:02:50.832456+0000","last_clean":"2023-09-12T16:02:50.832456+0000","last_became_active":"2023-09-12T16:02:04.029547+0000","last_became_peered":"2023-09-12T16:02:04.029547+0000","last_unstale":"2023-09-12T16:02:50.832456+0000","last_undegraded":"2023-09-12T16:02:50.832456+0000","last_fullsized":"2023-09-12T16:02:50.832456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:47:46.478325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846042+0000","last_change":"2023-09-12T16:02:51.846042+0000","last_active":"2023-09-12T16:02:51.846042+0000","last_peered":"2023-09-12T16:02:51.846042+0000","last_clean":"2023-09-12T16:02:51.846042+0000","last_became_active":"2023-09-12T16:02:51.845572+0000","last_became_peered":"2023-09-12T16:02:51.845572+0000","last_unstale":"2023-09-12T16:02:51.846042+0000","last_undegraded":"2023-09-12T16:02:51.846042+0000","last_fullsized":"2023-09-12T16:02:51.846042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:55.335485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069758+0000","last_change":"2023-09-12T16:02:52.069157+0000","last_active":"2023-09-12T16:02:52.069758+0000","last_peered":"2023-09-12T16:02:52.069758+0000","last_clean":"2023-09-12T16:02:52.069758+0000","last_became_active":"2023-09-12T16:02:52.068600+0000","last_became_peered":"2023-09-12T16:02:52.068600+0000","last_unstale":"2023-09-12T16:02:52.069758+0000","last_undegraded":"2023-09-12T16:02:52.069758+0000","last_fullsized":"2023-09-12T16:02:52.069758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:28:44.160612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":64,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:00.711723+0000","last_change":"2023-09-12T16:02:04.025723+0000","last_active":"2023-09-12T16:03:00.711723+0000","last_peered":"2023-09-12T16:03:00.711723+0000","last_clean":"2023-09-12T16:03:00.711723+0000","last_became_active":"2023-09-12T16:02:04.025432+0000","last_became_peered":"2023-09-12T16:02:04.025432+0000","last_unstale":"2023-09-12T16:03:00.711723+0000","last_undegraded":"2023-09-12T16:03:00.711723+0000","last_fullsized":"2023-09-12T16:03:00.711723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:56.279910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.839291+0000","last_change":"2023-09-12T16:02:51.839291+0000","last_active":"2023-09-12T16:02:51.839291+0000","last_peered":"2023-09-12T16:02:51.839291+0000","last_clean":"2023-09-12T16:02:51.839291+0000","last_became_active":"2023-09-12T16:02:51.838968+0000","last_became_peered":"2023-09-12T16:02:51.838968+0000","last_unstale":"2023-09-12T16:02:51.839291+0000","last_undegraded":"2023-09-12T16:02:51.839291+0000","last_fullsized":"2023-09-12T16:02:51.839291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:21:12.542019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835124+0000","last_change":"2023-09-12T16:02:09.286447+0000","last_active":"2023-09-12T16:02:50.835124+0000","last_peered":"2023-09-12T16:02:50.835124+0000","last_clean":"2023-09-12T16:02:50.835124+0000","last_became_active":"2023-09-12T16:02:09.285805+0000","last_became_peered":"2023-09-12T16:02:09.285805+0000","last_unstale":"2023-09-12T16:02:50.835124+0000","last_undegraded":"2023-09-12T16:02:50.835124+0000","last_fullsized":"2023-09-12T16:02:50.835124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:35:05.518872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830710+0000","last_change":"2023-09-12T16:02:04.032527+0000","last_active":"2023-09-12T16:02:50.830710+0000","last_peered":"2023-09-12T16:02:50.830710+0000","last_clean":"2023-09-12T16:02:50.830710+0000","last_became_active":"2023-09-12T16:02:04.032264+0000","last_became_peered":"2023-09-12T16:02:04.032264+0000","last_unstale":"2023-09-12T16:02:50.830710+0000","last_undegraded":"2023-09-12T16:02:50.830710+0000","last_fullsized":"2023-09-12T16:02:50.830710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:53:52.433549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833597+0000","last_change":"2023-09-12T16:02:05.029281+0000","last_active":"2023-09-12T16:02:50.833597+0000","last_peered":"2023-09-12T16:02:50.833597+0000","last_clean":"2023-09-12T16:02:50.833597+0000","last_became_active":"2023-09-12T16:02:05.028919+0000","last_became_peered":"2023-09-12T16:02:05.028919+0000","last_unstale":"2023-09-12T16:02:50.833597+0000","last_undegraded":"2023-09-12T16:02:50.833597+0000","last_fullsized":"2023-09-12T16:02:50.833597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:26.667060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069744+0000","last_change":"2023-09-12T16:02:52.069336+0000","last_active":"2023-09-12T16:02:52.069744+0000","last_peered":"2023-09-12T16:02:52.069744+0000","last_clean":"2023-09-12T16:02:52.069744+0000","last_became_active":"2023-09-12T16:02:52.068805+0000","last_became_peered":"2023-09-12T16:02:52.068805+0000","last_unstale":"2023-09-12T16:02:52.069744+0000","last_undegraded":"2023-09-12T16:02:52.069744+0000","last_fullsized":"2023-09-12T16:02:52.069744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:09.095952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831444+0000","last_change":"2023-09-12T16:02:04.026373+0000","last_active":"2023-09-12T16:02:50.831444+0000","last_peered":"2023-09-12T16:02:50.831444+0000","last_clean":"2023-09-12T16:02:50.831444+0000","last_became_active":"2023-09-12T16:02:04.025907+0000","last_became_peered":"2023-09-12T16:02:04.025907+0000","last_unstale":"2023-09-12T16:02:50.831444+0000","last_undegraded":"2023-09-12T16:02:50.831444+0000","last_fullsized":"2023-09-12T16:02:50.831444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:33:15.152128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025705+0000","last_change":"2023-09-12T16:02:05.040143+0000","last_active":"2023-09-12T16:02:51.025705+0000","last_peered":"2023-09-12T16:02:51.025705+0000","last_clean":"2023-09-12T16:02:51.025705+0000","last_became_active":"2023-09-12T16:02:05.039749+0000","last_became_peered":"2023-09-12T16:02:05.039749+0000","last_unstale":"2023-09-12T16:02:51.025705+0000","last_undegraded":"2023-09-12T16:02:51.025705+0000","last_fullsized":"2023-09-12T16:02:51.025705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:49.561252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:09.841670+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:09.841203+0000","last_became_peered":"2023-09-12T16:02:09.841203+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:55:46.128996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833178+0000","last_change":"2023-09-12T16:02:04.027953+0000","last_active":"2023-09-12T16:02:50.833178+0000","last_peered":"2023-09-12T16:02:50.833178+0000","last_clean":"2023-09-12T16:02:50.833178+0000","last_became_active":"2023-09-12T16:02:04.027624+0000","last_became_peered":"2023-09-12T16:02:04.027624+0000","last_unstale":"2023-09-12T16:02:50.833178+0000","last_undegraded":"2023-09-12T16:02:50.833178+0000","last_fullsized":"2023-09-12T16:02:50.833178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:33:58.356358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832388+0000","last_change":"2023-09-12T16:02:05.036593+0000","last_active":"2023-09-12T16:02:50.832388+0000","last_peered":"2023-09-12T16:02:50.832388+0000","last_clean":"2023-09-12T16:02:50.832388+0000","last_became_active":"2023-09-12T16:02:05.036204+0000","last_became_peered":"2023-09-12T16:02:05.036204+0000","last_unstale":"2023-09-12T16:02:50.832388+0000","last_undegraded":"2023-09-12T16:02:50.832388+0000","last_fullsized":"2023-09-12T16:02:50.832388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:12:31.068837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835475+0000","last_change":"2023-09-12T16:02:09.292405+0000","last_active":"2023-09-12T16:02:50.835475+0000","last_peered":"2023-09-12T16:02:50.835475+0000","last_clean":"2023-09-12T16:02:50.835475+0000","last_became_active":"2023-09-12T16:02:09.291877+0000","last_became_peered":"2023-09-12T16:02:09.291877+0000","last_unstale":"2023-09-12T16:02:50.835475+0000","last_undegraded":"2023-09-12T16:02:50.835475+0000","last_fullsized":"2023-09-12T16:02:50.835475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:53:04.847310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829465+0000","last_change":"2023-09-12T16:02:04.034738+0000","last_active":"2023-09-12T16:02:50.829465+0000","last_peered":"2023-09-12T16:02:50.829465+0000","last_clean":"2023-09-12T16:02:50.829465+0000","last_became_active":"2023-09-12T16:02:04.034408+0000","last_became_peered":"2023-09-12T16:02:04.034408+0000","last_unstale":"2023-09-12T16:02:50.829465+0000","last_undegraded":"2023-09-12T16:02:50.829465+0000","last_fullsized":"2023-09-12T16:02:50.829465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:27.845770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832464+0000","last_change":"2023-09-12T16:02:05.044936+0000","last_active":"2023-09-12T16:02:50.832464+0000","last_peered":"2023-09-12T16:02:50.832464+0000","last_clean":"2023-09-12T16:02:50.832464+0000","last_became_active":"2023-09-12T16:02:05.044617+0000","last_became_peered":"2023-09-12T16:02:05.044617+0000","last_unstale":"2023-09-12T16:02:50.832464+0000","last_undegraded":"2023-09-12T16:02:50.832464+0000","last_fullsized":"2023-09-12T16:02:50.832464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:41:32.639331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834992+0000","last_change":"2023-09-12T16:02:09.841919+0000","last_active":"2023-09-12T16:02:50.834992+0000","last_peered":"2023-09-12T16:02:50.834992+0000","last_clean":"2023-09-12T16:02:50.834992+0000","last_became_active":"2023-09-12T16:02:09.841410+0000","last_became_peered":"2023-09-12T16:02:09.841410+0000","last_unstale":"2023-09-12T16:02:50.834992+0000","last_undegraded":"2023-09-12T16:02:50.834992+0000","last_fullsized":"2023-09-12T16:02:50.834992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:29:57.511693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.840116+0000","last_change":"2023-09-12T16:02:51.840116+0000","last_active":"2023-09-12T16:02:51.840116+0000","last_peered":"2023-09-12T16:02:51.840116+0000","last_clean":"2023-09-12T16:02:51.840116+0000","last_became_active":"2023-09-12T16:02:51.839698+0000","last_became_peered":"2023-09-12T16:02:51.839698+0000","last_unstale":"2023-09-12T16:02:51.840116+0000","last_undegraded":"2023-09-12T16:02:51.840116+0000","last_fullsized":"2023-09-12T16:02:51.840116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:29.475855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":63,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:02.714131+0000","last_change":"2023-09-12T16:02:04.032892+0000","last_active":"2023-09-12T16:03:02.714131+0000","last_peered":"2023-09-12T16:03:02.714131+0000","last_clean":"2023-09-12T16:03:02.714131+0000","last_became_active":"2023-09-12T16:02:04.032637+0000","last_became_peered":"2023-09-12T16:02:04.032637+0000","last_unstale":"2023-09-12T16:03:02.714131+0000","last_undegraded":"2023-09-12T16:03:02.714131+0000","last_fullsized":"2023-09-12T16:03:02.714131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:22:55.805000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830231+0000","last_change":"2023-09-12T16:02:09.285127+0000","last_active":"2023-09-12T16:02:50.830231+0000","last_peered":"2023-09-12T16:02:50.830231+0000","last_clean":"2023-09-12T16:02:50.830231+0000","last_became_active":"2023-09-12T16:02:09.284790+0000","last_became_peered":"2023-09-12T16:02:09.284790+0000","last_unstale":"2023-09-12T16:02:50.830231+0000","last_undegraded":"2023-09-12T16:02:50.830231+0000","last_fullsized":"2023-09-12T16:02:50.830231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:50:50.578069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832025+0000","last_change":"2023-09-12T16:02:05.027877+0000","last_active":"2023-09-12T16:02:50.832025+0000","last_peered":"2023-09-12T16:02:50.832025+0000","last_clean":"2023-09-12T16:02:50.832025+0000","last_became_active":"2023-09-12T16:02:05.027582+0000","last_became_peered":"2023-09-12T16:02:05.027582+0000","last_unstale":"2023-09-12T16:02:50.832025+0000","last_undegraded":"2023-09-12T16:02:50.832025+0000","last_fullsized":"2023-09-12T16:02:50.832025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:47.835790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835647+0000","last_change":"2023-09-12T16:02:04.024800+0000","last_active":"2023-09-12T16:02:50.835647+0000","last_peered":"2023-09-12T16:02:50.835647+0000","last_clean":"2023-09-12T16:02:50.835647+0000","last_became_active":"2023-09-12T16:02:04.024436+0000","last_became_peered":"2023-09-12T16:02:04.024436+0000","last_unstale":"2023-09-12T16:02:50.835647+0000","last_undegraded":"2023-09-12T16:02:50.835647+0000","last_fullsized":"2023-09-12T16:02:50.835647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:57:35.093322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829225+0000","last_change":"2023-09-12T16:01:04.599706+0000","last_active":"2023-09-12T16:02:50.829225+0000","last_peered":"2023-09-12T16:02:50.829225+0000","last_clean":"2023-09-12T16:02:50.829225+0000","last_became_active":"2023-09-12T16:01:04.599328+0000","last_became_peered":"2023-09-12T16:01:04.599328+0000","last_unstale":"2023-09-12T16:02:50.829225+0000","last_undegraded":"2023-09-12T16:02:50.829225+0000","last_fullsized":"2023-09-12T16:02:50.829225+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_clean_scrub_stamp":"2023-09-12T15:58:15.250648+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:57:56.846488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833462+0000","last_change":"2023-09-12T16:02:09.841960+0000","last_active":"2023-09-12T16:02:50.833462+0000","last_peered":"2023-09-12T16:02:50.833462+0000","last_clean":"2023-09-12T16:02:50.833462+0000","last_became_active":"2023-09-12T16:02:09.841279+0000","last_became_peered":"2023-09-12T16:02:09.841279+0000","last_unstale":"2023-09-12T16:02:50.833462+0000","last_undegraded":"2023-09-12T16:02:50.833462+0000","last_fullsized":"2023-09-12T16:02:50.833462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:17:19.419350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.847600+0000","last_change":"2023-09-12T16:02:51.847600+0000","last_active":"2023-09-12T16:02:51.847600+0000","last_peered":"2023-09-12T16:02:51.847600+0000","last_clean":"2023-09-12T16:02:51.847600+0000","last_became_active":"2023-09-12T16:02:51.847292+0000","last_became_peered":"2023-09-12T16:02:51.847292+0000","last_unstale":"2023-09-12T16:02:51.847600+0000","last_undegraded":"2023-09-12T16:02:51.847600+0000","last_fullsized":"2023-09-12T16:02:51.847600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:25:58.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829344+0000","last_change":"2023-09-12T16:02:04.029790+0000","last_active":"2023-09-12T16:02:50.829344+0000","last_peered":"2023-09-12T16:02:50.829344+0000","last_clean":"2023-09-12T16:02:50.829344+0000","last_became_active":"2023-09-12T16:02:04.029440+0000","last_became_peered":"2023-09-12T16:02:04.029440+0000","last_unstale":"2023-09-12T16:02:50.829344+0000","last_undegraded":"2023-09-12T16:02:50.829344+0000","last_fullsized":"2023-09-12T16:02:50.829344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:50:33.690628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025552+0000","last_change":"2023-09-12T16:02:09.289757+0000","last_active":"2023-09-12T16:02:51.025552+0000","last_peered":"2023-09-12T16:02:51.025552+0000","last_clean":"2023-09-12T16:02:51.025552+0000","last_became_active":"2023-09-12T16:02:09.289348+0000","last_became_peered":"2023-09-12T16:02:09.289348+0000","last_unstale":"2023-09-12T16:02:51.025552+0000","last_undegraded":"2023-09-12T16:02:51.025552+0000","last_fullsized":"2023-09-12T16:02:51.025552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:15:14.695521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831296+0000","last_change":"2023-09-12T16:02:04.033213+0000","last_active":"2023-09-12T16:02:50.831296+0000","last_peered":"2023-09-12T16:02:50.831296+0000","last_clean":"2023-09-12T16:02:50.831296+0000","last_became_active":"2023-09-12T16:02:04.032978+0000","last_became_peered":"2023-09-12T16:02:04.032978+0000","last_unstale":"2023-09-12T16:02:50.831296+0000","last_undegraded":"2023-09-12T16:02:50.831296+0000","last_fullsized":"2023-09-12T16:02:50.831296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:25:33.718954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835250+0000","last_change":"2023-09-12T16:02:05.043252+0000","last_active":"2023-09-12T16:02:50.835250+0000","last_peered":"2023-09-12T16:02:50.835250+0000","last_clean":"2023-09-12T16:02:50.835250+0000","last_became_active":"2023-09-12T16:02:05.042975+0000","last_became_peered":"2023-09-12T16:02:05.042975+0000","last_unstale":"2023-09-12T16:02:50.835250+0000","last_undegraded":"2023-09-12T16:02:50.835250+0000","last_fullsized":"2023-09-12T16:02:50.835250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:46:31.549547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831861+0000","last_change":"2023-09-12T16:02:09.841728+0000","last_active":"2023-09-12T16:02:50.831861+0000","last_peered":"2023-09-12T16:02:50.831861+0000","last_clean":"2023-09-12T16:02:50.831861+0000","last_became_active":"2023-09-12T16:02:09.841351+0000","last_became_peered":"2023-09-12T16:02:09.841351+0000","last_unstale":"2023-09-12T16:02:50.831861+0000","last_undegraded":"2023-09-12T16:02:50.831861+0000","last_fullsized":"2023-09-12T16:02:50.831861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:00:50.637272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833531+0000","last_change":"2023-09-12T16:02:04.025624+0000","last_active":"2023-09-12T16:02:50.833531+0000","last_peered":"2023-09-12T16:02:50.833531+0000","last_clean":"2023-09-12T16:02:50.833531+0000","last_became_active":"2023-09-12T16:02:04.025330+0000","last_became_peered":"2023-09-12T16:02:04.025330+0000","last_unstale":"2023-09-12T16:02:50.833531+0000","last_undegraded":"2023-09-12T16:02:50.833531+0000","last_fullsized":"2023-09-12T16:02:50.833531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:00:57.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833174+0000","last_change":"2023-09-12T16:02:05.028414+0000","last_active":"2023-09-12T16:02:50.833174+0000","last_peered":"2023-09-12T16:02:50.833174+0000","last_clean":"2023-09-12T16:02:50.833174+0000","last_became_active":"2023-09-12T16:02:05.028081+0000","last_became_peered":"2023-09-12T16:02:05.028081+0000","last_unstale":"2023-09-12T16:02:50.833174+0000","last_undegraded":"2023-09-12T16:02:50.833174+0000","last_fullsized":"2023-09-12T16:02:50.833174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:57:58.245105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834801+0000","last_change":"2023-09-12T16:02:09.283222+0000","last_active":"2023-09-12T16:02:50.834801+0000","last_peered":"2023-09-12T16:02:50.834801+0000","last_clean":"2023-09-12T16:02:50.834801+0000","last_became_active":"2023-09-12T16:02:09.282814+0000","last_became_peered":"2023-09-12T16:02:09.282814+0000","last_unstale":"2023-09-12T16:02:50.834801+0000","last_undegraded":"2023-09-12T16:02:50.834801+0000","last_fullsized":"2023-09-12T16:02:50.834801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:18.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836561+0000","last_change":"2023-09-12T16:02:04.026694+0000","last_active":"2023-09-12T16:02:50.836561+0000","last_peered":"2023-09-12T16:02:50.836561+0000","last_clean":"2023-09-12T16:02:50.836561+0000","last_became_active":"2023-09-12T16:02:04.026368+0000","last_became_peered":"2023-09-12T16:02:04.026368+0000","last_unstale":"2023-09-12T16:02:50.836561+0000","last_undegraded":"2023-09-12T16:02:50.836561+0000","last_fullsized":"2023-09-12T16:02:50.836561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:14:10.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829216+0000","last_change":"2023-09-12T16:02:05.039037+0000","last_active":"2023-09-12T16:02:50.829216+0000","last_peered":"2023-09-12T16:02:50.829216+0000","last_clean":"2023-09-12T16:02:50.829216+0000","last_became_active":"2023-09-12T16:02:05.038708+0000","last_became_peered":"2023-09-12T16:02:05.038708+0000","last_unstale":"2023-09-12T16:02:50.829216+0000","last_undegraded":"2023-09-12T16:02:50.829216+0000","last_fullsized":"2023-09-12T16:02:50.829216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:13:09.805646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836160+0000","last_change":"2023-09-12T16:02:09.841757+0000","last_active":"2023-09-12T16:02:50.836160+0000","last_peered":"2023-09-12T16:02:50.836160+0000","last_clean":"2023-09-12T16:02:50.836160+0000","last_became_active":"2023-09-12T16:02:09.841173+0000","last_became_peered":"2023-09-12T16:02:09.841173+0000","last_unstale":"2023-09-12T16:02:50.836160+0000","last_undegraded":"2023-09-12T16:02:50.836160+0000","last_fullsized":"2023-09-12T16:02:50.836160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:57:06.626828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836147+0000","last_change":"2023-09-12T16:02:04.031009+0000","last_active":"2023-09-12T16:02:50.836147+0000","last_peered":"2023-09-12T16:02:50.836147+0000","last_clean":"2023-09-12T16:02:50.836147+0000","last_became_active":"2023-09-12T16:02:04.030714+0000","last_became_peered":"2023-09-12T16:02:04.030714+0000","last_unstale":"2023-09-12T16:02:50.836147+0000","last_undegraded":"2023-09-12T16:02:50.836147+0000","last_fullsized":"2023-09-12T16:02:50.836147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:12.578686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830789+0000","last_change":"2023-09-12T16:02:05.030778+0000","last_active":"2023-09-12T16:02:50.830789+0000","last_peered":"2023-09-12T16:02:50.830789+0000","last_clean":"2023-09-12T16:02:50.830789+0000","last_became_active":"2023-09-12T16:02:05.030454+0000","last_became_peered":"2023-09-12T16:02:05.030454+0000","last_unstale":"2023-09-12T16:02:50.830789+0000","last_undegraded":"2023-09-12T16:02:50.830789+0000","last_fullsized":"2023-09-12T16:02:50.830789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:52.951246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834030+0000","last_change":"2023-09-12T16:02:09.289668+0000","last_active":"2023-09-12T16:02:50.834030+0000","last_peered":"2023-09-12T16:02:50.834030+0000","last_clean":"2023-09-12T16:02:50.834030+0000","last_became_active":"2023-09-12T16:02:09.289301+0000","last_became_peered":"2023-09-12T16:02:09.289301+0000","last_unstale":"2023-09-12T16:02:50.834030+0000","last_undegraded":"2023-09-12T16:02:50.834030+0000","last_fullsized":"2023-09-12T16:02:50.834030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:20:04.528146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832522+0000","last_change":"2023-09-12T16:02:04.036175+0000","last_active":"2023-09-12T16:02:50.832522+0000","last_peered":"2023-09-12T16:02:50.832522+0000","last_clean":"2023-09-12T16:02:50.832522+0000","last_became_active":"2023-09-12T16:02:04.035879+0000","last_became_peered":"2023-09-12T16:02:04.035879+0000","last_unstale":"2023-09-12T16:02:50.832522+0000","last_undegraded":"2023-09-12T16:02:50.832522+0000","last_fullsized":"2023-09-12T16:02:50.832522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:29:20.798716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025379+0000","last_change":"2023-09-12T16:02:05.034195+0000","last_active":"2023-09-12T16:02:51.025379+0000","last_peered":"2023-09-12T16:02:51.025379+0000","last_clean":"2023-09-12T16:02:51.025379+0000","last_became_active":"2023-09-12T16:02:05.033534+0000","last_became_peered":"2023-09-12T16:02:05.033534+0000","last_unstale":"2023-09-12T16:02:51.025379+0000","last_undegraded":"2023-09-12T16:02:51.025379+0000","last_fullsized":"2023-09-12T16:02:51.025379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:34:04.180218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834899+0000","last_change":"2023-09-12T16:02:09.287543+0000","last_active":"2023-09-12T16:02:50.834899+0000","last_peered":"2023-09-12T16:02:50.834899+0000","last_clean":"2023-09-12T16:02:50.834899+0000","last_became_active":"2023-09-12T16:02:09.287055+0000","last_became_peered":"2023-09-12T16:02:09.287055+0000","last_unstale":"2023-09-12T16:02:50.834899+0000","last_undegraded":"2023-09-12T16:02:50.834899+0000","last_fullsized":"2023-09-12T16:02:50.834899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:58.214381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834994+0000","last_change":"2023-09-12T16:02:04.030206+0000","last_active":"2023-09-12T16:02:50.834994+0000","last_peered":"2023-09-12T16:02:50.834994+0000","last_clean":"2023-09-12T16:02:50.834994+0000","last_became_active":"2023-09-12T16:02:04.029765+0000","last_became_peered":"2023-09-12T16:02:04.029765+0000","last_unstale":"2023-09-12T16:02:50.834994+0000","last_undegraded":"2023-09-12T16:02:50.834994+0000","last_fullsized":"2023-09-12T16:02:50.834994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:56:07.551681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832663+0000","last_change":"2023-09-12T16:02:05.028300+0000","last_active":"2023-09-12T16:02:50.832663+0000","last_peered":"2023-09-12T16:02:50.832663+0000","last_clean":"2023-09-12T16:02:50.832663+0000","last_became_active":"2023-09-12T16:02:05.027970+0000","last_became_peered":"2023-09-12T16:02:05.027970+0000","last_unstale":"2023-09-12T16:02:50.832663+0000","last_undegraded":"2023-09-12T16:02:50.832663+0000","last_fullsized":"2023-09-12T16:02:50.832663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:39:44.571593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835970+0000","last_change":"2023-09-12T16:02:09.841661+0000","last_active":"2023-09-12T16:02:50.835970+0000","last_peered":"2023-09-12T16:02:50.835970+0000","last_clean":"2023-09-12T16:02:50.835970+0000","last_became_active":"2023-09-12T16:02:09.841039+0000","last_became_peered":"2023-09-12T16:02:09.841039+0000","last_unstale":"2023-09-12T16:02:50.835970+0000","last_undegraded":"2023-09-12T16:02:50.835970+0000","last_fullsized":"2023-09-12T16:02:50.835970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:34.529098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830867+0000","last_change":"2023-09-12T16:02:04.031927+0000","last_active":"2023-09-12T16:02:50.830867+0000","last_peered":"2023-09-12T16:02:50.830867+0000","last_clean":"2023-09-12T16:02:50.830867+0000","last_became_active":"2023-09-12T16:02:04.031614+0000","last_became_peered":"2023-09-12T16:02:04.031614+0000","last_unstale":"2023-09-12T16:02:50.830867+0000","last_undegraded":"2023-09-12T16:02:50.830867+0000","last_fullsized":"2023-09-12T16:02:50.830867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:00:23.423933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829483+0000","last_change":"2023-09-12T16:02:05.032647+0000","last_active":"2023-09-12T16:02:50.829483+0000","last_peered":"2023-09-12T16:02:50.829483+0000","last_clean":"2023-09-12T16:02:50.829483+0000","last_became_active":"2023-09-12T16:02:05.031701+0000","last_became_peered":"2023-09-12T16:02:05.031701+0000","last_unstale":"2023-09-12T16:02:50.829483+0000","last_undegraded":"2023-09-12T16:02:50.829483+0000","last_fullsized":"2023-09-12T16:02:50.829483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:02:19.596913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837136+0000","last_change":"2023-09-12T16:02:09.291043+0000","last_active":"2023-09-12T16:02:50.837136+0000","last_peered":"2023-09-12T16:02:50.837136+0000","last_clean":"2023-09-12T16:02:50.837136+0000","last_became_active":"2023-09-12T16:02:09.290666+0000","last_became_peered":"2023-09-12T16:02:09.290666+0000","last_unstale":"2023-09-12T16:02:50.837136+0000","last_undegraded":"2023-09-12T16:02:50.837136+0000","last_fullsized":"2023-09-12T16:02:50.837136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:10:47.354479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835888+0000","last_change":"2023-09-12T16:02:04.030501+0000","last_active":"2023-09-12T16:02:50.835888+0000","last_peered":"2023-09-12T16:02:50.835888+0000","last_clean":"2023-09-12T16:02:50.835888+0000","last_became_active":"2023-09-12T16:02:04.030196+0000","last_became_peered":"2023-09-12T16:02:04.030196+0000","last_unstale":"2023-09-12T16:02:50.835888+0000","last_undegraded":"2023-09-12T16:02:50.835888+0000","last_fullsized":"2023-09-12T16:02:50.835888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:58:34.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830617+0000","last_change":"2023-09-12T16:02:05.030466+0000","last_active":"2023-09-12T16:02:50.830617+0000","last_peered":"2023-09-12T16:02:50.830617+0000","last_clean":"2023-09-12T16:02:50.830617+0000","last_became_active":"2023-09-12T16:02:05.029988+0000","last_became_peered":"2023-09-12T16:02:05.029988+0000","last_unstale":"2023-09-12T16:02:50.830617+0000","last_undegraded":"2023-09-12T16:02:50.830617+0000","last_fullsized":"2023-09-12T16:02:50.830617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:08:56.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846403+0000","last_change":"2023-09-12T16:02:51.846403+0000","last_active":"2023-09-12T16:02:51.846403+0000","last_peered":"2023-09-12T16:02:51.846403+0000","last_clean":"2023-09-12T16:02:51.846403+0000","last_became_active":"2023-09-12T16:02:51.846047+0000","last_became_peered":"2023-09-12T16:02:51.846047+0000","last_unstale":"2023-09-12T16:02:51.846403+0000","last_undegraded":"2023-09-12T16:02:51.846403+0000","last_fullsized":"2023-09-12T16:02:51.846403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:58:11.458339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832945+0000","last_change":"2023-09-12T16:02:04.027380+0000","last_active":"2023-09-12T16:02:50.832945+0000","last_peered":"2023-09-12T16:02:50.832945+0000","last_clean":"2023-09-12T16:02:50.832945+0000","last_became_active":"2023-09-12T16:02:04.027098+0000","last_became_peered":"2023-09-12T16:02:04.027098+0000","last_unstale":"2023-09-12T16:02:50.832945+0000","last_undegraded":"2023-09-12T16:02:50.832945+0000","last_fullsized":"2023-09-12T16:02:50.832945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:34:12.260747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829993+0000","last_change":"2023-09-12T16:02:05.029249+0000","last_active":"2023-09-12T16:02:50.829993+0000","last_peered":"2023-09-12T16:02:50.829993+0000","last_clean":"2023-09-12T16:02:50.829993+0000","last_became_active":"2023-09-12T16:02:05.028955+0000","last_became_peered":"2023-09-12T16:02:05.028955+0000","last_unstale":"2023-09-12T16:02:50.829993+0000","last_undegraded":"2023-09-12T16:02:50.829993+0000","last_fullsized":"2023-09-12T16:02:50.829993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:02:10.382712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831923+0000","last_change":"2023-09-12T16:02:09.288724+0000","last_active":"2023-09-12T16:02:50.831923+0000","last_peered":"2023-09-12T16:02:50.831923+0000","last_clean":"2023-09-12T16:02:50.831923+0000","last_became_active":"2023-09-12T16:02:09.288206+0000","last_became_peered":"2023-09-12T16:02:09.288206+0000","last_unstale":"2023-09-12T16:02:50.831923+0000","last_undegraded":"2023-09-12T16:02:50.831923+0000","last_fullsized":"2023-09-12T16:02:50.831923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:45:10.154195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835225+0000","last_change":"2023-09-12T16:02:04.031219+0000","last_active":"2023-09-12T16:02:50.835225+0000","last_peered":"2023-09-12T16:02:50.835225+0000","last_clean":"2023-09-12T16:02:50.835225+0000","last_became_active":"2023-09-12T16:02:04.030867+0000","last_became_peered":"2023-09-12T16:02:04.030867+0000","last_unstale":"2023-09-12T16:02:50.835225+0000","last_undegraded":"2023-09-12T16:02:50.835225+0000","last_fullsized":"2023-09-12T16:02:50.835225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:16.885094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831081+0000","last_change":"2023-09-12T16:02:05.028004+0000","last_active":"2023-09-12T16:02:50.831081+0000","last_peered":"2023-09-12T16:02:50.831081+0000","last_clean":"2023-09-12T16:02:50.831081+0000","last_became_active":"2023-09-12T16:02:05.027668+0000","last_became_peered":"2023-09-12T16:02:05.027668+0000","last_unstale":"2023-09-12T16:02:50.831081+0000","last_undegraded":"2023-09-12T16:02:50.831081+0000","last_fullsized":"2023-09-12T16:02:50.831081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:41:23.019243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025479+0000","last_change":"2023-09-12T16:02:05.034380+0000","last_active":"2023-09-12T16:02:51.025479+0000","last_peered":"2023-09-12T16:02:51.025479+0000","last_clean":"2023-09-12T16:02:51.025479+0000","last_became_active":"2023-09-12T16:02:05.034022+0000","last_became_peered":"2023-09-12T16:02:05.034022+0000","last_unstale":"2023-09-12T16:02:51.025479+0000","last_undegraded":"2023-09-12T16:02:51.025479+0000","last_fullsized":"2023-09-12T16:02:51.025479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:55:56.404742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835155+0000","last_change":"2023-09-12T16:02:05.031723+0000","last_active":"2023-09-12T16:02:50.835155+0000","last_peered":"2023-09-12T16:02:50.835155+0000","last_clean":"2023-09-12T16:02:50.835155+0000","last_became_active":"2023-09-12T16:02:05.030954+0000","last_became_peered":"2023-09-12T16:02:05.030954+0000","last_unstale":"2023-09-12T16:02:50.835155+0000","last_undegraded":"2023-09-12T16:02:50.835155+0000","last_fullsized":"2023-09-12T16:02:50.835155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.319954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834938+0000","last_change":"2023-09-12T16:02:05.034851+0000","last_active":"2023-09-12T16:02:50.834938+0000","last_peered":"2023-09-12T16:02:50.834938+0000","last_clean":"2023-09-12T16:02:50.834938+0000","last_became_active":"2023-09-12T16:02:05.030733+0000","last_became_peered":"2023-09-12T16:02:05.030733+0000","last_unstale":"2023-09-12T16:02:50.834938+0000","last_undegraded":"2023-09-12T16:02:50.834938+0000","last_fullsized":"2023-09-12T16:02:50.834938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:33.605216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830111+0000","last_change":"2023-09-12T16:02:05.037331+0000","last_active":"2023-09-12T16:02:50.830111+0000","last_peered":"2023-09-12T16:02:50.830111+0000","last_clean":"2023-09-12T16:02:50.830111+0000","last_became_active":"2023-09-12T16:02:05.037015+0000","last_became_peered":"2023-09-12T16:02:05.037015+0000","last_unstale":"2023-09-12T16:02:50.830111+0000","last_undegraded":"2023-09-12T16:02:50.830111+0000","last_fullsized":"2023-09-12T16:02:50.830111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:29:14.035891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835319+0000","last_change":"2023-09-12T16:02:05.039803+0000","last_active":"2023-09-12T16:02:50.835319+0000","last_peered":"2023-09-12T16:02:50.835319+0000","last_clean":"2023-09-12T16:02:50.835319+0000","last_became_active":"2023-09-12T16:02:05.039390+0000","last_became_peered":"2023-09-12T16:02:05.039390+0000","last_unstale":"2023-09-12T16:02:50.835319+0000","last_undegraded":"2023-09-12T16:02:50.835319+0000","last_fullsized":"2023-09-12T16:02:50.835319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:22.422090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025700+0000","last_change":"2023-09-12T16:02:05.033153+0000","last_active":"2023-09-12T16:02:51.025700+0000","last_peered":"2023-09-12T16:02:51.025700+0000","last_clean":"2023-09-12T16:02:51.025700+0000","last_became_active":"2023-09-12T16:02:05.032800+0000","last_became_peered":"2023-09-12T16:02:05.032800+0000","last_unstale":"2023-09-12T16:02:51.025700+0000","last_undegraded":"2023-09-12T16:02:51.025700+0000","last_fullsized":"2023-09-12T16:02:51.025700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:31:43.090031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836428+0000","last_change":"2023-09-12T16:02:05.031519+0000","last_active":"2023-09-12T16:02:50.836428+0000","last_peered":"2023-09-12T16:02:50.836428+0000","last_clean":"2023-09-12T16:02:50.836428+0000","last_became_active":"2023-09-12T16:02:05.030227+0000","last_became_peered":"2023-09-12T16:02:05.030227+0000","last_unstale":"2023-09-12T16:02:50.836428+0000","last_undegraded":"2023-09-12T16:02:50.836428+0000","last_fullsized":"2023-09-12T16:02:50.836428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:00.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833413+0000","last_change":"2023-09-12T16:02:05.031763+0000","last_active":"2023-09-12T16:02:50.833413+0000","last_peered":"2023-09-12T16:02:50.833413+0000","last_clean":"2023-09-12T16:02:50.833413+0000","last_became_active":"2023-09-12T16:02:05.031441+0000","last_became_peered":"2023-09-12T16:02:05.031441+0000","last_unstale":"2023-09-12T16:02:50.833413+0000","last_undegraded":"2023-09-12T16:02:50.833413+0000","last_fullsized":"2023-09-12T16:02:50.833413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:55.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832249+0000","last_change":"2023-09-12T16:02:05.033786+0000","last_active":"2023-09-12T16:02:50.832249+0000","last_peered":"2023-09-12T16:02:50.832249+0000","last_clean":"2023-09-12T16:02:50.832249+0000","last_became_active":"2023-09-12T16:02:05.033497+0000","last_became_peered":"2023-09-12T16:02:05.033497+0000","last_unstale":"2023-09-12T16:02:50.832249+0000","last_undegraded":"2023-09-12T16:02:50.832249+0000","last_fullsized":"2023-09-12T16:02:50.832249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:01:58.912488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832560+0000","last_change":"2023-09-12T16:02:05.031144+0000","last_active":"2023-09-12T16:02:50.832560+0000","last_peered":"2023-09-12T16:02:50.832560+0000","last_clean":"2023-09-12T16:02:50.832560+0000","last_became_active":"2023-09-12T16:02:05.030827+0000","last_became_peered":"2023-09-12T16:02:05.030827+0000","last_unstale":"2023-09-12T16:02:50.832560+0000","last_undegraded":"2023-09-12T16:02:50.832560+0000","last_fullsized":"2023-09-12T16:02:50.832560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:40:48.014344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837378+0000","last_change":"2023-09-12T16:02:05.021273+0000","last_active":"2023-09-12T16:02:50.837378+0000","last_peered":"2023-09-12T16:02:50.837378+0000","last_clean":"2023-09-12T16:02:50.837378+0000","last_became_active":"2023-09-12T16:02:05.020953+0000","last_became_peered":"2023-09-12T16:02:05.020953+0000","last_unstale":"2023-09-12T16:02:50.837378+0000","last_undegraded":"2023-09-12T16:02:50.837378+0000","last_fullsized":"2023-09-12T16:02:50.837378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:33:25.462412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835081+0000","last_change":"2023-09-12T16:02:05.039019+0000","last_active":"2023-09-12T16:02:50.835081+0000","last_peered":"2023-09-12T16:02:50.835081+0000","last_clean":"2023-09-12T16:02:50.835081+0000","last_became_active":"2023-09-12T16:02:05.038621+0000","last_became_peered":"2023-09-12T16:02:05.038621+0000","last_unstale":"2023-09-12T16:02:50.835081+0000","last_undegraded":"2023-09-12T16:02:50.835081+0000","last_fullsized":"2023-09-12T16:02:50.835081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:08:08.934375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025761+0000","last_change":"2023-09-12T16:02:05.039737+0000","last_active":"2023-09-12T16:02:51.025761+0000","last_peered":"2023-09-12T16:02:51.025761+0000","last_clean":"2023-09-12T16:02:51.025761+0000","last_became_active":"2023-09-12T16:02:05.039304+0000","last_became_peered":"2023-09-12T16:02:05.039304+0000","last_unstale":"2023-09-12T16:02:51.025761+0000","last_undegraded":"2023-09-12T16:02:51.025761+0000","last_fullsized":"2023-09-12T16:02:51.025761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:50:13.141915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244717,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65275,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":232,"num_read_kb":182,"num_write":154,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":59815,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":195825,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":489432,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70199999999999996}]},{"osd":1,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.755}]},{"osd":2,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.44},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78100000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.72599999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":9,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68799999999999994}]},{"osd":10,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.93400000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776154,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1137975,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.68799999999999994}]},{"osd":3,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78700000000000003}]},{"osd":5,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.84899999999999998}]},{"osd":6,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.751}]},{"osd":8,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.93400000000000005}]},{"osd":9,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.91300000000000003}]},{"osd":11,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.83499999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939680,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":493764,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.48599999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.42799999999999999,"15min":0.436},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.82499999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.35099999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60599999999999998}]},{"osd":2,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70499999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.46500000000000002,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.625},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.72099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81799999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88700000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79000000000000004}]},{"osd":8,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.70099999999999996,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.80800000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53500000000000003}]},{"osd":10,"last update":"Tue Sep 12 16:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.751},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66200000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":8,"up_from":56,"seq":240518168611,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":488675,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.497},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67500000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52800000000000002}]},{"osd":1,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.44400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78000000000000003}]},{"osd":2,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.45300000000000001,"15min":0.44},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.45700000000000002,"15min":0.44500000000000001},"min":{"1min":0.34300000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66500000000000004}]},{"osd":4,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.45300000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.51600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.91400000000000003}]},{"osd":5,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.871}]},{"osd":6,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.46800000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97199999999999998,"5min":1.075,"15min":1.075},"last":0.89000000000000001}]},{"osd":7,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72999999999999998,"15min":0.72499999999999998},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0069999999999999,"5min":1.038,"15min":1.038},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.499,"5min":0.435,"15min":0.435},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.82999999999999996}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.93600000000000005}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":1.0209999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]}]},{"osd":7,"up_from":49,"seq":210453397545,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1085555,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.40200000000000002,"15min":0.38600000000000001},"min":{"1min":0.34200000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63600000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.432},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.433,"15min":0.40699999999999997},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45600000000000002}]},{"osd":2,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59099999999999997,"15min":0.55800000000000005},"min":{"1min":0.439,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1639999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0640000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.91200000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.35099999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72599999999999998,"5min":1.069,"15min":1.069},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65100000000000002,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93999999999999995,"5min":1.909,"15min":1.909},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65000000000000002,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.139,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96299999999999997,"5min":1.119,"15min":1.119},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84699999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67900000000000005}]},{"osd":8,"last update":"Tue Sep 12 16:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":1.2030000000000001,"5min":1.879,"15min":1.879},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.49299999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.083,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.81200000000000006}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96499999999999997,"5min":1.329,"15min":1.329},"last":0.80300000000000005}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.0309999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.59799999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561069,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":484844,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.39200000000000002,"15min":0.39000000000000001},"min":{"1min":0.25600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56999999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34399999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70099999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.69799999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71899999999999997,"5min":1.079,"15min":1.079},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":1.129,"15min":1.129},"last":0.59299999999999997}]},{"osd":3,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.75,"5min":1.083,"15min":1.083},"last":0.37},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.68300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.443}]},{"osd":4,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72699999999999998,"5min":1.161,"15min":1.161},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81999999999999995,"5min":1.022,"15min":1.022},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.75}]},{"osd":8,"last update":"Tue Sep 12 16:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69399999999999995,"15min":0.70799999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.84299999999999997,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70199999999999996}]},{"osd":9,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71799999999999997,"15min":0.72899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.46300000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90800000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.61599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":545948,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.38100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48399999999999999}]},{"osd":2,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.51400000000000001},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56299999999999994,"15min":0.52600000000000002},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57099999999999995,"15min":0.51900000000000002},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":1.002,"15min":1.002},"last":0.624},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.498,"15min":0.45600000000000002},"min":{"1min":0.40899999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.57499999999999996,"15min":0.51900000000000002},"min":{"1min":0.52400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.76300000000000001}]},{"osd":5,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.38100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.79200000000000004}]},{"osd":8,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.74199999999999999,"15min":0.72699999999999998},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.51100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64400000000000002}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79100000000000004,"15min":0.77900000000000003},"min":{"1min":0.58399999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81699999999999995}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":486232,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60599999999999998},"min":{"1min":0.39900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79800000000000004,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60399999999999998,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79100000000000004,"5min":2.113,"15min":2.113},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Sep 12 16:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.874},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.754,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.39800000000000002}]},{"osd":3,"last update":"Tue Sep 12 16:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.39000000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.443,"15min":0.433},"min":{"1min":0.309,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58399999999999996}]},{"osd":5,"last update":"Tue Sep 12 16:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47499999999999998}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.74399999999999999,"5min":0.879,"15min":0.879},"last":0.437},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":1.004,"15min":1.004},"last":0.53000000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.54600000000000004}]},{"osd":8,"last update":"Tue Sep 12 16:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69299999999999995,"15min":0.70299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85499999999999998,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51100000000000001}]},{"osd":9,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.628},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.49399999999999999}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68100000000000005}]},{"osd":11,"last update":"Tue Sep 12 16:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61599999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485552,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80400000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48399999999999999,"15min":0.47399999999999998},"min":{"1min":0.29299999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82899999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63100000000000001}]},{"osd":1,"last update":"Tue Sep 12 16:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.498},"min":{"1min":0.438,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.56000000000000005,"15min":0.50600000000000001},"min":{"1min":0.41199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.81299999999999994}]},{"osd":3,"last update":"Tue Sep 12 16:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.41199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.56000000000000005,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56200000000000006}]},{"osd":4,"last update":"Tue Sep 12 16:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76200000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.73999999999999999}]},{"osd":5,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60499999999999998}]},{"osd":6,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58599999999999997}]},{"osd":7,"last update":"Tue Sep 12 16:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76700000000000002}]},{"osd":8,"last update":"Tue Sep 12 16:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.63600000000000001,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.629,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.52800000000000002}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.65400000000000003}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.754},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485038,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.36699999999999999,"15min":0.34499999999999997},"min":{"1min":0.29499999999999998,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.29699999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.51700000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.309}]},{"osd":1,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.34000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":3.552,"15min":3.552},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76800000000000002,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.755,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.379}]},{"osd":4,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53900000000000003,"15min":0.51700000000000002},"min":{"1min":0.45600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84499999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Sep 12 16:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.41499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83299999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.79100000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.47499999999999998},"min":{"1min":0.36299999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69199999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40799999999999997}]},{"osd":7,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.56100000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.503},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.5,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57499999999999996}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66100000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.083,"15min":1.083},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.75,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.53700000000000003}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":550496,"data_compressed":30118,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.46999999999999997},"min":{"1min":0.38500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.45800000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88800000000000001,"5min":1.458,"15min":1.458},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.36299999999999999},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.41799999999999998,"15min":0.374},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60799999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58199999999999996,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.316,"15min":0.316},"max":{"1min":1.0249999999999999,"5min":1.383,"15min":1.383},"last":0.627},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52900000000000003,"15min":0.49399999999999999},"min":{"1min":0.38600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56699999999999995}]},{"osd":5,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":1.54,"15min":1.54},"last":0.748},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.56899999999999995,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92700000000000005,"5min":1.502,"15min":1.502},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65300000000000002,"15min":0.64000000000000001},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.639,"15min":1.639},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40100000000000002}]},{"osd":8,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.58099999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.79300000000000004},"min":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.998,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.77500000000000002}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97499999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.69399999999999995}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1080753,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70899999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.497},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.499,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66400000000000003,"5min":0.749,"15min":0.749},"last":0.41599999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.751}]},{"osd":2,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.53200000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40000000000000002,"15min":0.39400000000000002},"min":{"1min":0.309,"5min":0.245,"15min":0.245},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39300000000000002}]},{"osd":3,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53200000000000003,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60699999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.36799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.627,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.621}]},{"osd":4,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.40300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81899999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.39200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72999999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.879,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.57299999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.72599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.626,"15min":0.626},"min":{"1min":0.50600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83299999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.82599999999999996}]},{"osd":8,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85499999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83399999999999996,"5min":0.87,"15min":0.87},"last":0.81200000000000006}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.76500000000000001}]},{"osd":10,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.85999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":61919,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T16:03:07.008 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph pg dump --format=json 2023-09-12T16:03:07.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:06 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:03:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:06 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:03:07.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:06 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:03:07.620 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:08.168 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:07 smithi130 ceph-mon[98670]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:08.168 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:07 smithi130 ceph-mon[98670]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 4 op/s; 4.9 KiB/s, 0 objects/s recovering 2023-09-12T16:03:08.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:07 smithi159 ceph-mon[102522]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:08.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:07 smithi159 ceph-mon[102522]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 4 op/s; 4.9 KiB/s, 0 objects/s recovering 2023-09-12T16:03:08.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:07 smithi203 ceph-mon[103431]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:08.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:07 smithi203 ceph-mon[103431]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 4 op/s; 4.9 KiB/s, 0 objects/s recovering 2023-09-12T16:03:08.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.820+0000 7f62f1122700 1 -- 172.21.15.130:0/465287712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ffee0 msgr2=0x7f62ec10c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:08.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.820+0000 7f62f1122700 1 --2- 172.21.15.130:0/465287712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ffee0 0x7f62ec10c3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009990 tx=0x7f62e4009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:08.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.820+0000 7f62f1122700 1 -- 172.21.15.130:0/465287712 shutdown_connections 2023-09-12T16:03:08.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.820+0000 7f62f1122700 1 --2- 172.21.15.130:0/465287712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ffee0 0x7f62ec10c3f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:08.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.820+0000 7f62f1122700 1 --2- 172.21.15.130:0/465287712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec104530 0x7f62ec104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:08.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.820+0000 7f62f1122700 1 --2- 172.21.15.130:0/465287712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62ec0ff520 0x7f62ec0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:08.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.820+0000 7f62f1122700 1 -- 172.21.15.130:0/465287712 >> 172.21.15.130:0/465287712 conn(0x7f62ec076600 msgr2=0x7f62ec076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:08.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.820+0000 7f62f1122700 1 -- 172.21.15.130:0/465287712 shutdown_connections 2023-09-12T16:03:08.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.820+0000 7f62f1122700 1 -- 172.21.15.130:0/465287712 wait complete. 2023-09-12T16:03:08.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62f1122700 1 Processor -- start 2023-09-12T16:03:08.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62f1122700 1 -- start start 2023-09-12T16:03:08.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62f1122700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ff520 0x7f62ec101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:08.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62f1122700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec0ffee0 0x7f62ec1016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:08.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62f1122700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62ec104530 0x7f62ec101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:08.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62f1122700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec1a4af0 con 0x7f62ec0ff520 2023-09-12T16:03:08.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62f1122700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec1a4c60 con 0x7f62ec104530 2023-09-12T16:03:08.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62f1122700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec1a4e20 con 0x7f62ec0ffee0 2023-09-12T16:03:08.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62ead9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ff520 0x7f62ec101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:08.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62ead9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ff520 0x7f62ec101180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54284/0 (socket says 172.21.15.130:54284) 2023-09-12T16:03:08.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62ead9d700 1 -- 172.21.15.130:0/59998949 learned_addr learned my addr 172.21.15.130:0/59998949 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:08.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.821+0000 7f62eb59e700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62ec104530 0x7f62ec101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:08.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62ea59c700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec0ffee0 0x7f62ec1016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:08.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62ead9d700 1 -- 172.21.15.130:0/59998949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec0ffee0 msgr2=0x7f62ec1016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:08.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62ead9d700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec0ffee0 0x7f62ec1016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:08.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62ead9d700 1 -- 172.21.15.130:0/59998949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62ec104530 msgr2=0x7f62ec101c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:08.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62ead9d700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62ec104530 0x7f62ec101c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:08.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62ead9d700 1 -- 172.21.15.130:0/59998949 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40095f0 con 0x7f62ec0ff520 2023-09-12T16:03:08.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62ead9d700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ff520 0x7f62ec101180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00efc0 tx=0x7f62dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:08.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62e3fff700 1 -- 172.21.15.130:0/59998949 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc018070 con 0x7f62ec0ff520 2023-09-12T16:03:08.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62e3fff700 1 -- 172.21.15.130:0/59998949 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62dc00f040 con 0x7f62ec0ff520 2023-09-12T16:03:08.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec1a50a0 con 0x7f62ec0ff520 2023-09-12T16:03:08.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62e3fff700 1 -- 172.21.15.130:0/59998949 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc004310 con 0x7f62ec0ff520 2023-09-12T16:03:08.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.822+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec1a55f0 con 0x7f62ec0ff520 2023-09-12T16:03:08.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.823+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec04e300 con 0x7f62ec0ff520 2023-09-12T16:03:08.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.824+0000 7f62e3fff700 1 -- 172.21.15.130:0/59998949 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f62dc00ca70 con 0x7f62ec0ff520 2023-09-12T16:03:08.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.825+0000 7f62e3fff700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62d80a0750 0x7f62d80a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:08.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.825+0000 7f62ea59c700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62d80a0750 0x7f62d80a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:08.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.825+0000 7f62e3fff700 1 -- 172.21.15.130:0/59998949 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f62dc0cbea0 con 0x7f62ec0ff520 2023-09-12T16:03:08.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.825+0000 7f62ea59c700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62d80a0750 0x7f62d80a2c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005950 tx=0x7f62d40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:08.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:08.829+0000 7f62e3fff700 1 -- 172.21.15.130:0/59998949 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f62dc095590 con 0x7f62ec0ff520 2023-09-12T16:03:09.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.026+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62ec10f6c0 con 0x7f62d80a0750 2023-09-12T16:03:09.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.050+0000 7f62e3fff700 1 -- 172.21.15.130:0/59998949 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424914 (secure 0 0 0) 0x7f62ec10f6c0 con 0x7f62d80a0750 2023-09-12T16:03:09.055 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:09.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62d80a0750 msgr2=0x7f62d80a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:09.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62d80a0750 0x7f62d80a2c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005950 tx=0x7f62d40058e0 comp rx=0 tx=0).stop 2023-09-12T16:03:09.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ff520 msgr2=0x7f62ec101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:09.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ff520 0x7f62ec101180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00efc0 tx=0x7f62dc00c3a0 comp rx=0 tx=0).stop 2023-09-12T16:03:09.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 shutdown_connections 2023-09-12T16:03:09.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62d80a0750 0x7f62d80a2c10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:09.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62ec104530 0x7f62ec101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:09.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62ec0ff520 0x7f62ec101180 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:09.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 --2- 172.21.15.130:0/59998949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec0ffee0 0x7f62ec1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:09.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 >> 172.21.15.130:0/59998949 conn(0x7f62ec076600 msgr2=0x7f62ec068120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:09.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 shutdown_connections 2023-09-12T16:03:09.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:09.053+0000 7f62f1122700 1 -- 172.21.15.130:0/59998949 wait complete. 2023-09-12T16:03:09.060 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2023-09-12T16:03:09.395 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2023-09-12T16:03:08.772645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655889,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":383,"num_read_kb":301,"num_write":337,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1240551,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335968,"kb_used_data":11696,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605856,"statfs":{"total":1151940427776,"available":1151596396544,"internally_reserved":0,"allocated":11976704,"data_stored":7814264,"data_compressed":363759,"data_compressed_allocated":491520,"data_compressed_original":2514944,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003671"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025907+0000","last_change":"2023-09-12T16:02:05.039617+0000","last_active":"2023-09-12T16:02:51.025907+0000","last_peered":"2023-09-12T16:02:51.025907+0000","last_clean":"2023-09-12T16:02:51.025907+0000","last_became_active":"2023-09-12T16:02:05.039264+0000","last_became_peered":"2023-09-12T16:02:05.039264+0000","last_unstale":"2023-09-12T16:02:51.025907+0000","last_undegraded":"2023-09-12T16:02:51.025907+0000","last_fullsized":"2023-09-12T16:02:51.025907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:37:01.381406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829291+0000","last_change":"2023-09-12T16:02:05.033151+0000","last_active":"2023-09-12T16:02:50.829291+0000","last_peered":"2023-09-12T16:02:50.829291+0000","last_clean":"2023-09-12T16:02:50.829291+0000","last_became_active":"2023-09-12T16:02:05.032866+0000","last_became_peered":"2023-09-12T16:02:05.032866+0000","last_unstale":"2023-09-12T16:02:50.829291+0000","last_undegraded":"2023-09-12T16:02:50.829291+0000","last_fullsized":"2023-09-12T16:02:50.829291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:35.921428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.032478+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.032124+0000","last_became_peered":"2023-09-12T16:02:05.032124+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:58:54.112577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835786+0000","last_change":"2023-09-12T16:02:05.033088+0000","last_active":"2023-09-12T16:02:50.835786+0000","last_peered":"2023-09-12T16:02:50.835786+0000","last_clean":"2023-09-12T16:02:50.835786+0000","last_became_active":"2023-09-12T16:02:05.032332+0000","last_became_peered":"2023-09-12T16:02:05.032332+0000","last_unstale":"2023-09-12T16:02:50.835786+0000","last_undegraded":"2023-09-12T16:02:50.835786+0000","last_fullsized":"2023-09-12T16:02:50.835786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:25:38.751961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:05.036612+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:05.036215+0000","last_became_peered":"2023-09-12T16:02:05.036215+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:15:04.561751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832126+0000","last_change":"2023-09-12T16:02:05.043816+0000","last_active":"2023-09-12T16:02:50.832126+0000","last_peered":"2023-09-12T16:02:50.832126+0000","last_clean":"2023-09-12T16:02:50.832126+0000","last_became_active":"2023-09-12T16:02:05.043599+0000","last_became_peered":"2023-09-12T16:02:05.043599+0000","last_unstale":"2023-09-12T16:02:50.832126+0000","last_undegraded":"2023-09-12T16:02:50.832126+0000","last_fullsized":"2023-09-12T16:02:50.832126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:32:00.286196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829345+0000","last_change":"2023-09-12T16:02:05.031476+0000","last_active":"2023-09-12T16:02:50.829345+0000","last_peered":"2023-09-12T16:02:50.829345+0000","last_clean":"2023-09-12T16:02:50.829345+0000","last_became_active":"2023-09-12T16:02:05.031150+0000","last_became_peered":"2023-09-12T16:02:05.031150+0000","last_unstale":"2023-09-12T16:02:50.829345+0000","last_undegraded":"2023-09-12T16:02:50.829345+0000","last_fullsized":"2023-09-12T16:02:50.829345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:22:27.405505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831825+0000","last_change":"2023-09-12T16:02:05.031981+0000","last_active":"2023-09-12T16:02:50.831825+0000","last_peered":"2023-09-12T16:02:50.831825+0000","last_clean":"2023-09-12T16:02:50.831825+0000","last_became_active":"2023-09-12T16:02:05.031692+0000","last_became_peered":"2023-09-12T16:02:05.031692+0000","last_unstale":"2023-09-12T16:02:50.831825+0000","last_undegraded":"2023-09-12T16:02:50.831825+0000","last_fullsized":"2023-09-12T16:02:50.831825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:51:00.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838117+0000","last_change":"2023-09-12T16:02:05.029898+0000","last_active":"2023-09-12T16:02:50.838117+0000","last_peered":"2023-09-12T16:02:50.838117+0000","last_clean":"2023-09-12T16:02:50.838117+0000","last_became_active":"2023-09-12T16:02:05.029731+0000","last_became_peered":"2023-09-12T16:02:05.029731+0000","last_unstale":"2023-09-12T16:02:50.838117+0000","last_undegraded":"2023-09-12T16:02:50.838117+0000","last_fullsized":"2023-09-12T16:02:50.838117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:16:31.194561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833228+0000","last_change":"2023-09-12T16:02:05.044846+0000","last_active":"2023-09-12T16:02:50.833228+0000","last_peered":"2023-09-12T16:02:50.833228+0000","last_clean":"2023-09-12T16:02:50.833228+0000","last_became_active":"2023-09-12T16:02:05.044430+0000","last_became_peered":"2023-09-12T16:02:05.044430+0000","last_unstale":"2023-09-12T16:02:50.833228+0000","last_undegraded":"2023-09-12T16:02:50.833228+0000","last_fullsized":"2023-09-12T16:02:50.833228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:15.999080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830392+0000","last_change":"2023-09-12T16:02:05.030285+0000","last_active":"2023-09-12T16:02:50.830392+0000","last_peered":"2023-09-12T16:02:50.830392+0000","last_clean":"2023-09-12T16:02:50.830392+0000","last_became_active":"2023-09-12T16:02:05.029585+0000","last_became_peered":"2023-09-12T16:02:05.029585+0000","last_unstale":"2023-09-12T16:02:50.830392+0000","last_undegraded":"2023-09-12T16:02:50.830392+0000","last_fullsized":"2023-09-12T16:02:50.830392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:52:27.471412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835453+0000","last_change":"2023-09-12T16:02:05.037258+0000","last_active":"2023-09-12T16:02:50.835453+0000","last_peered":"2023-09-12T16:02:50.835453+0000","last_clean":"2023-09-12T16:02:50.835453+0000","last_became_active":"2023-09-12T16:02:05.036941+0000","last_became_peered":"2023-09-12T16:02:05.036941+0000","last_unstale":"2023-09-12T16:02:50.835453+0000","last_undegraded":"2023-09-12T16:02:50.835453+0000","last_fullsized":"2023-09-12T16:02:50.835453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:04:17.998927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835219+0000","last_change":"2023-09-12T16:02:05.041986+0000","last_active":"2023-09-12T16:02:50.835219+0000","last_peered":"2023-09-12T16:02:50.835219+0000","last_clean":"2023-09-12T16:02:50.835219+0000","last_became_active":"2023-09-12T16:02:05.041584+0000","last_became_peered":"2023-09-12T16:02:05.041584+0000","last_unstale":"2023-09-12T16:02:50.835219+0000","last_undegraded":"2023-09-12T16:02:50.835219+0000","last_fullsized":"2023-09-12T16:02:50.835219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:39.065525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828785+0000","last_change":"2023-09-12T16:02:05.030405+0000","last_active":"2023-09-12T16:02:50.828785+0000","last_peered":"2023-09-12T16:02:50.828785+0000","last_clean":"2023-09-12T16:02:50.828785+0000","last_became_active":"2023-09-12T16:02:05.029488+0000","last_became_peered":"2023-09-12T16:02:05.029488+0000","last_unstale":"2023-09-12T16:02:50.828785+0000","last_undegraded":"2023-09-12T16:02:50.828785+0000","last_fullsized":"2023-09-12T16:02:50.828785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.646069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829401+0000","last_change":"2023-09-12T16:02:05.032576+0000","last_active":"2023-09-12T16:02:50.829401+0000","last_peered":"2023-09-12T16:02:50.829401+0000","last_clean":"2023-09-12T16:02:50.829401+0000","last_became_active":"2023-09-12T16:02:05.032277+0000","last_became_peered":"2023-09-12T16:02:05.032277+0000","last_unstale":"2023-09-12T16:02:50.829401+0000","last_undegraded":"2023-09-12T16:02:50.829401+0000","last_fullsized":"2023-09-12T16:02:50.829401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:47:32.734830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836039+0000","last_change":"2023-09-12T16:02:05.023468+0000","last_active":"2023-09-12T16:02:50.836039+0000","last_peered":"2023-09-12T16:02:50.836039+0000","last_clean":"2023-09-12T16:02:50.836039+0000","last_became_active":"2023-09-12T16:02:05.023132+0000","last_became_peered":"2023-09-12T16:02:05.023132+0000","last_unstale":"2023-09-12T16:02:50.836039+0000","last_undegraded":"2023-09-12T16:02:50.836039+0000","last_fullsized":"2023-09-12T16:02:50.836039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:21:49.378779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025810+0000","last_change":"2023-09-12T16:02:05.039910+0000","last_active":"2023-09-12T16:02:51.025810+0000","last_peered":"2023-09-12T16:02:51.025810+0000","last_clean":"2023-09-12T16:02:51.025810+0000","last_became_active":"2023-09-12T16:02:05.039605+0000","last_became_peered":"2023-09-12T16:02:05.039605+0000","last_unstale":"2023-09-12T16:02:51.025810+0000","last_undegraded":"2023-09-12T16:02:51.025810+0000","last_fullsized":"2023-09-12T16:02:51.025810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:19:58.823471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833500+0000","last_change":"2023-09-12T16:02:05.031667+0000","last_active":"2023-09-12T16:02:50.833500+0000","last_peered":"2023-09-12T16:02:50.833500+0000","last_clean":"2023-09-12T16:02:50.833500+0000","last_became_active":"2023-09-12T16:02:05.030886+0000","last_became_peered":"2023-09-12T16:02:05.030886+0000","last_unstale":"2023-09-12T16:02:50.833500+0000","last_undegraded":"2023-09-12T16:02:50.833500+0000","last_fullsized":"2023-09-12T16:02:50.833500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:24:02.312902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836131+0000","last_change":"2023-09-12T16:02:05.037607+0000","last_active":"2023-09-12T16:02:50.836131+0000","last_peered":"2023-09-12T16:02:50.836131+0000","last_clean":"2023-09-12T16:02:50.836131+0000","last_became_active":"2023-09-12T16:02:05.037095+0000","last_became_peered":"2023-09-12T16:02:05.037095+0000","last_unstale":"2023-09-12T16:02:50.836131+0000","last_undegraded":"2023-09-12T16:02:50.836131+0000","last_fullsized":"2023-09-12T16:02:50.836131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:17:04.886737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834805+0000","last_change":"2023-09-12T16:02:09.292093+0000","last_active":"2023-09-12T16:02:50.834805+0000","last_peered":"2023-09-12T16:02:50.834805+0000","last_clean":"2023-09-12T16:02:50.834805+0000","last_became_active":"2023-09-12T16:02:09.291602+0000","last_became_peered":"2023-09-12T16:02:09.291602+0000","last_unstale":"2023-09-12T16:02:50.834805+0000","last_undegraded":"2023-09-12T16:02:50.834805+0000","last_fullsized":"2023-09-12T16:02:50.834805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:36:23.089498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.857296+0000","last_change":"2023-09-12T16:02:51.857296+0000","last_active":"2023-09-12T16:02:51.857296+0000","last_peered":"2023-09-12T16:02:51.857296+0000","last_clean":"2023-09-12T16:02:51.857296+0000","last_became_active":"2023-09-12T16:02:51.843961+0000","last_became_peered":"2023-09-12T16:02:51.843961+0000","last_unstale":"2023-09-12T16:02:51.857296+0000","last_undegraded":"2023-09-12T16:02:51.857296+0000","last_fullsized":"2023-09-12T16:02:51.857296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:55:57.938037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832962+0000","last_change":"2023-09-12T16:02:05.032191+0000","last_active":"2023-09-12T16:02:50.832962+0000","last_peered":"2023-09-12T16:02:50.832962+0000","last_clean":"2023-09-12T16:02:50.832962+0000","last_became_active":"2023-09-12T16:02:05.031888+0000","last_became_peered":"2023-09-12T16:02:05.031888+0000","last_unstale":"2023-09-12T16:02:50.832962+0000","last_undegraded":"2023-09-12T16:02:50.832962+0000","last_fullsized":"2023-09-12T16:02:50.832962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:50:15.775078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834369+0000","last_change":"2023-09-12T16:02:09.290650+0000","last_active":"2023-09-12T16:02:50.834369+0000","last_peered":"2023-09-12T16:02:50.834369+0000","last_clean":"2023-09-12T16:02:50.834369+0000","last_became_active":"2023-09-12T16:02:09.290205+0000","last_became_peered":"2023-09-12T16:02:09.290205+0000","last_unstale":"2023-09-12T16:02:50.834369+0000","last_undegraded":"2023-09-12T16:02:50.834369+0000","last_fullsized":"2023-09-12T16:02:50.834369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:44:04.930063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833124+0000","last_change":"2023-09-12T16:02:04.032506+0000","last_active":"2023-09-12T16:02:50.833124+0000","last_peered":"2023-09-12T16:02:50.833124+0000","last_clean":"2023-09-12T16:02:50.833124+0000","last_became_active":"2023-09-12T16:02:04.032047+0000","last_became_peered":"2023-09-12T16:02:04.032047+0000","last_unstale":"2023-09-12T16:02:50.833124+0000","last_undegraded":"2023-09-12T16:02:50.833124+0000","last_fullsized":"2023-09-12T16:02:50.833124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:30:22.664465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.025692+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.025370+0000","last_became_peered":"2023-09-12T16:02:05.025370+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.305375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.068789+0000","last_change":"2023-09-12T16:02:52.068789+0000","last_active":"2023-09-12T16:02:52.068789+0000","last_peered":"2023-09-12T16:02:52.068789+0000","last_clean":"2023-09-12T16:02:52.068789+0000","last_became_active":"2023-09-12T16:02:52.068276+0000","last_became_peered":"2023-09-12T16:02:52.068276+0000","last_unstale":"2023-09-12T16:02:52.068789+0000","last_undegraded":"2023-09-12T16:02:52.068789+0000","last_fullsized":"2023-09-12T16:02:52.068789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:19:19.009173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831481+0000","last_change":"2023-09-12T16:02:04.032079+0000","last_active":"2023-09-12T16:02:50.831481+0000","last_peered":"2023-09-12T16:02:50.831481+0000","last_clean":"2023-09-12T16:02:50.831481+0000","last_became_active":"2023-09-12T16:02:04.031835+0000","last_became_peered":"2023-09-12T16:02:04.031835+0000","last_unstale":"2023-09-12T16:02:50.831481+0000","last_undegraded":"2023-09-12T16:02:50.831481+0000","last_fullsized":"2023-09-12T16:02:50.831481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:48:33.935012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837888+0000","last_change":"2023-09-12T16:02:05.037671+0000","last_active":"2023-09-12T16:02:50.837888+0000","last_peered":"2023-09-12T16:02:50.837888+0000","last_clean":"2023-09-12T16:02:50.837888+0000","last_became_active":"2023-09-12T16:02:05.037307+0000","last_became_peered":"2023-09-12T16:02:05.037307+0000","last_unstale":"2023-09-12T16:02:50.837888+0000","last_undegraded":"2023-09-12T16:02:50.837888+0000","last_fullsized":"2023-09-12T16:02:50.837888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:47:23.836612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834100+0000","last_change":"2023-09-12T16:02:09.289165+0000","last_active":"2023-09-12T16:02:50.834100+0000","last_peered":"2023-09-12T16:02:50.834100+0000","last_clean":"2023-09-12T16:02:50.834100+0000","last_became_active":"2023-09-12T16:02:09.288903+0000","last_became_peered":"2023-09-12T16:02:09.288903+0000","last_unstale":"2023-09-12T16:02:50.834100+0000","last_undegraded":"2023-09-12T16:02:50.834100+0000","last_fullsized":"2023-09-12T16:02:50.834100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:27:09.160822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831262+0000","last_change":"2023-09-12T16:02:04.027279+0000","last_active":"2023-09-12T16:02:50.831262+0000","last_peered":"2023-09-12T16:02:50.831262+0000","last_clean":"2023-09-12T16:02:50.831262+0000","last_became_active":"2023-09-12T16:02:04.026975+0000","last_became_peered":"2023-09-12T16:02:04.026975+0000","last_unstale":"2023-09-12T16:02:50.831262+0000","last_undegraded":"2023-09-12T16:02:50.831262+0000","last_fullsized":"2023-09-12T16:02:50.831262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:45.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832885+0000","last_change":"2023-09-12T16:02:05.027582+0000","last_active":"2023-09-12T16:02:50.832885+0000","last_peered":"2023-09-12T16:02:50.832885+0000","last_clean":"2023-09-12T16:02:50.832885+0000","last_became_active":"2023-09-12T16:02:05.027236+0000","last_became_peered":"2023-09-12T16:02:05.027236+0000","last_unstale":"2023-09-12T16:02:50.832885+0000","last_undegraded":"2023-09-12T16:02:50.832885+0000","last_fullsized":"2023-09-12T16:02:50.832885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:06:24.812731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836055+0000","last_change":"2023-09-12T16:02:09.284381+0000","last_active":"2023-09-12T16:02:50.836055+0000","last_peered":"2023-09-12T16:02:50.836055+0000","last_clean":"2023-09-12T16:02:50.836055+0000","last_became_active":"2023-09-12T16:02:09.283947+0000","last_became_peered":"2023-09-12T16:02:09.283947+0000","last_unstale":"2023-09-12T16:02:50.836055+0000","last_undegraded":"2023-09-12T16:02:50.836055+0000","last_fullsized":"2023-09-12T16:02:50.836055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:05:24.803781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:05.720121+0000","last_change":"2023-09-12T16:02:04.027693+0000","last_active":"2023-09-12T16:03:05.720121+0000","last_peered":"2023-09-12T16:03:05.720121+0000","last_clean":"2023-09-12T16:03:05.720121+0000","last_became_active":"2023-09-12T16:02:04.027315+0000","last_became_peered":"2023-09-12T16:02:04.027315+0000","last_unstale":"2023-09-12T16:03:05.720121+0000","last_undegraded":"2023-09-12T16:03:05.720121+0000","last_fullsized":"2023-09-12T16:03:05.720121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:32:21.756802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831054+0000","last_change":"2023-09-12T16:02:05.031673+0000","last_active":"2023-09-12T16:02:50.831054+0000","last_peered":"2023-09-12T16:02:50.831054+0000","last_clean":"2023-09-12T16:02:50.831054+0000","last_became_active":"2023-09-12T16:02:05.031364+0000","last_became_peered":"2023-09-12T16:02:05.031364+0000","last_unstale":"2023-09-12T16:02:50.831054+0000","last_undegraded":"2023-09-12T16:02:50.831054+0000","last_fullsized":"2023-09-12T16:02:50.831054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:48:05.222530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835641+0000","last_change":"2023-09-12T16:02:09.288536+0000","last_active":"2023-09-12T16:02:50.835641+0000","last_peered":"2023-09-12T16:02:50.835641+0000","last_clean":"2023-09-12T16:02:50.835641+0000","last_became_active":"2023-09-12T16:02:09.288062+0000","last_became_peered":"2023-09-12T16:02:09.288062+0000","last_unstale":"2023-09-12T16:02:50.835641+0000","last_undegraded":"2023-09-12T16:02:50.835641+0000","last_fullsized":"2023-09-12T16:02:50.835641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:39:28.935563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832024+0000","last_change":"2023-09-12T16:02:04.026298+0000","last_active":"2023-09-12T16:02:50.832024+0000","last_peered":"2023-09-12T16:02:50.832024+0000","last_clean":"2023-09-12T16:02:50.832024+0000","last_became_active":"2023-09-12T16:02:04.025975+0000","last_became_peered":"2023-09-12T16:02:04.025975+0000","last_unstale":"2023-09-12T16:02:50.832024+0000","last_undegraded":"2023-09-12T16:02:50.832024+0000","last_fullsized":"2023-09-12T16:02:50.832024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:12:04.338497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832502+0000","last_change":"2023-09-12T16:02:05.028337+0000","last_active":"2023-09-12T16:02:50.832502+0000","last_peered":"2023-09-12T16:02:50.832502+0000","last_clean":"2023-09-12T16:02:50.832502+0000","last_became_active":"2023-09-12T16:02:05.028036+0000","last_became_peered":"2023-09-12T16:02:05.028036+0000","last_unstale":"2023-09-12T16:02:50.832502+0000","last_undegraded":"2023-09-12T16:02:50.832502+0000","last_fullsized":"2023-09-12T16:02:50.832502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:39:09.387191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025809+0000","last_change":"2023-09-12T16:02:09.290950+0000","last_active":"2023-09-12T16:02:51.025809+0000","last_peered":"2023-09-12T16:02:51.025809+0000","last_clean":"2023-09-12T16:02:51.025809+0000","last_became_active":"2023-09-12T16:02:09.290477+0000","last_became_peered":"2023-09-12T16:02:09.290477+0000","last_unstale":"2023-09-12T16:02:51.025809+0000","last_undegraded":"2023-09-12T16:02:51.025809+0000","last_fullsized":"2023-09-12T16:02:51.025809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:53:45.907575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834845+0000","last_change":"2023-09-12T16:02:04.026025+0000","last_active":"2023-09-12T16:02:50.834845+0000","last_peered":"2023-09-12T16:02:50.834845+0000","last_clean":"2023-09-12T16:02:50.834845+0000","last_became_active":"2023-09-12T16:02:04.025722+0000","last_became_peered":"2023-09-12T16:02:04.025722+0000","last_unstale":"2023-09-12T16:02:50.834845+0000","last_undegraded":"2023-09-12T16:02:50.834845+0000","last_fullsized":"2023-09-12T16:02:50.834845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:29:58.357960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831920+0000","last_change":"2023-09-12T16:02:05.028461+0000","last_active":"2023-09-12T16:02:50.831920+0000","last_peered":"2023-09-12T16:02:50.831920+0000","last_clean":"2023-09-12T16:02:50.831920+0000","last_became_active":"2023-09-12T16:02:05.028176+0000","last_became_peered":"2023-09-12T16:02:05.028176+0000","last_unstale":"2023-09-12T16:02:50.831920+0000","last_undegraded":"2023-09-12T16:02:50.831920+0000","last_fullsized":"2023-09-12T16:02:50.831920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:16:48.652513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829138+0000","last_change":"2023-09-12T16:02:09.295245+0000","last_active":"2023-09-12T16:02:50.829138+0000","last_peered":"2023-09-12T16:02:50.829138+0000","last_clean":"2023-09-12T16:02:50.829138+0000","last_became_active":"2023-09-12T16:02:09.294872+0000","last_became_peered":"2023-09-12T16:02:09.294872+0000","last_unstale":"2023-09-12T16:02:50.829138+0000","last_undegraded":"2023-09-12T16:02:50.829138+0000","last_fullsized":"2023-09-12T16:02:50.829138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:43:49.462317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832140+0000","last_change":"2023-09-12T16:02:04.030815+0000","last_active":"2023-09-12T16:02:50.832140+0000","last_peered":"2023-09-12T16:02:50.832140+0000","last_clean":"2023-09-12T16:02:50.832140+0000","last_became_active":"2023-09-12T16:02:04.030435+0000","last_became_peered":"2023-09-12T16:02:04.030435+0000","last_unstale":"2023-09-12T16:02:50.832140+0000","last_undegraded":"2023-09-12T16:02:50.832140+0000","last_fullsized":"2023-09-12T16:02:50.832140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:39:59.661143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832565+0000","last_change":"2023-09-12T16:02:05.031155+0000","last_active":"2023-09-12T16:02:50.832565+0000","last_peered":"2023-09-12T16:02:50.832565+0000","last_clean":"2023-09-12T16:02:50.832565+0000","last_became_active":"2023-09-12T16:02:05.030803+0000","last_became_peered":"2023-09-12T16:02:05.030803+0000","last_unstale":"2023-09-12T16:02:50.832565+0000","last_undegraded":"2023-09-12T16:02:50.832565+0000","last_fullsized":"2023-09-12T16:02:50.832565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:55:21.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832473+0000","last_change":"2023-09-12T16:02:09.286445+0000","last_active":"2023-09-12T16:02:50.832473+0000","last_peered":"2023-09-12T16:02:50.832473+0000","last_clean":"2023-09-12T16:02:50.832473+0000","last_became_active":"2023-09-12T16:02:09.286049+0000","last_became_peered":"2023-09-12T16:02:09.286049+0000","last_unstale":"2023-09-12T16:02:50.832473+0000","last_undegraded":"2023-09-12T16:02:50.832473+0000","last_fullsized":"2023-09-12T16:02:50.832473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:43:13.943984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":64,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:04.717745+0000","last_change":"2023-09-12T16:02:04.035341+0000","last_active":"2023-09-12T16:03:04.717745+0000","last_peered":"2023-09-12T16:03:04.717745+0000","last_clean":"2023-09-12T16:03:04.717745+0000","last_became_active":"2023-09-12T16:02:04.035092+0000","last_became_peered":"2023-09-12T16:02:04.035092+0000","last_unstale":"2023-09-12T16:03:04.717745+0000","last_undegraded":"2023-09-12T16:03:04.717745+0000","last_fullsized":"2023-09-12T16:03:04.717745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:33:00.784401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838363+0000","last_change":"2023-09-12T16:02:05.025229+0000","last_active":"2023-09-12T16:02:50.838363+0000","last_peered":"2023-09-12T16:02:50.838363+0000","last_clean":"2023-09-12T16:02:50.838363+0000","last_became_active":"2023-09-12T16:02:05.024963+0000","last_became_peered":"2023-09-12T16:02:05.024963+0000","last_unstale":"2023-09-12T16:02:50.838363+0000","last_undegraded":"2023-09-12T16:02:50.838363+0000","last_fullsized":"2023-09-12T16:02:50.838363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:46:04.318082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832595+0000","last_change":"2023-09-12T16:02:09.295011+0000","last_active":"2023-09-12T16:02:50.832595+0000","last_peered":"2023-09-12T16:02:50.832595+0000","last_clean":"2023-09-12T16:02:50.832595+0000","last_became_active":"2023-09-12T16:02:09.294617+0000","last_became_peered":"2023-09-12T16:02:09.294617+0000","last_unstale":"2023-09-12T16:02:50.832595+0000","last_undegraded":"2023-09-12T16:02:50.832595+0000","last_fullsized":"2023-09-12T16:02:50.832595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:57:53.902946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830613+0000","last_change":"2023-09-12T16:02:04.028086+0000","last_active":"2023-09-12T16:02:50.830613+0000","last_peered":"2023-09-12T16:02:50.830613+0000","last_clean":"2023-09-12T16:02:50.830613+0000","last_became_active":"2023-09-12T16:02:04.027798+0000","last_became_peered":"2023-09-12T16:02:04.027798+0000","last_unstale":"2023-09-12T16:02:50.830613+0000","last_undegraded":"2023-09-12T16:02:50.830613+0000","last_fullsized":"2023-09-12T16:02:50.830613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:11:16.921824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829002+0000","last_change":"2023-09-12T16:02:05.030691+0000","last_active":"2023-09-12T16:02:50.829002+0000","last_peered":"2023-09-12T16:02:50.829002+0000","last_clean":"2023-09-12T16:02:50.829002+0000","last_became_active":"2023-09-12T16:02:05.030379+0000","last_became_peered":"2023-09-12T16:02:05.030379+0000","last_unstale":"2023-09-12T16:02:50.829002+0000","last_undegraded":"2023-09-12T16:02:50.829002+0000","last_fullsized":"2023-09-12T16:02:50.829002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:15:05.221078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835660+0000","last_change":"2023-09-12T16:02:09.290703+0000","last_active":"2023-09-12T16:02:50.835660+0000","last_peered":"2023-09-12T16:02:50.835660+0000","last_clean":"2023-09-12T16:02:50.835660+0000","last_became_active":"2023-09-12T16:02:09.290227+0000","last_became_peered":"2023-09-12T16:02:09.290227+0000","last_unstale":"2023-09-12T16:02:50.835660+0000","last_undegraded":"2023-09-12T16:02:50.835660+0000","last_fullsized":"2023-09-12T16:02:50.835660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:07:06.182295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832824+0000","last_change":"2023-09-12T16:02:04.028630+0000","last_active":"2023-09-12T16:02:50.832824+0000","last_peered":"2023-09-12T16:02:50.832824+0000","last_clean":"2023-09-12T16:02:50.832824+0000","last_became_active":"2023-09-12T16:02:04.028319+0000","last_became_peered":"2023-09-12T16:02:04.028319+0000","last_unstale":"2023-09-12T16:02:50.832824+0000","last_undegraded":"2023-09-12T16:02:50.832824+0000","last_fullsized":"2023-09-12T16:02:50.832824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:13:43.349425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838071+0000","last_change":"2023-09-12T16:02:05.032392+0000","last_active":"2023-09-12T16:02:50.838071+0000","last_peered":"2023-09-12T16:02:50.838071+0000","last_clean":"2023-09-12T16:02:50.838071+0000","last_became_active":"2023-09-12T16:02:05.032099+0000","last_became_peered":"2023-09-12T16:02:05.032099+0000","last_unstale":"2023-09-12T16:02:50.838071+0000","last_undegraded":"2023-09-12T16:02:50.838071+0000","last_fullsized":"2023-09-12T16:02:50.838071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:46:03.108459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832745+0000","last_change":"2023-09-12T16:02:09.289573+0000","last_active":"2023-09-12T16:02:50.832745+0000","last_peered":"2023-09-12T16:02:50.832745+0000","last_clean":"2023-09-12T16:02:50.832745+0000","last_became_active":"2023-09-12T16:02:09.289179+0000","last_became_peered":"2023-09-12T16:02:09.289179+0000","last_unstale":"2023-09-12T16:02:50.832745+0000","last_undegraded":"2023-09-12T16:02:50.832745+0000","last_fullsized":"2023-09-12T16:02:50.832745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:52:35.905346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":159,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:04.865834+0000","last_change":"2023-09-12T16:02:51.851180+0000","last_active":"2023-09-12T16:03:04.865834+0000","last_peered":"2023-09-12T16:03:04.865834+0000","last_clean":"2023-09-12T16:03:04.865834+0000","last_became_active":"2023-09-12T16:02:51.841452+0000","last_became_peered":"2023-09-12T16:02:51.841452+0000","last_unstale":"2023-09-12T16:03:04.865834+0000","last_undegraded":"2023-09-12T16:03:04.865834+0000","last_fullsized":"2023-09-12T16:03:04.865834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:20:20.903637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60885,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":73,"num_read_kb":92,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":59233,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831192+0000","last_change":"2023-09-12T16:02:05.031409+0000","last_active":"2023-09-12T16:02:50.831192+0000","last_peered":"2023-09-12T16:02:50.831192+0000","last_clean":"2023-09-12T16:02:50.831192+0000","last_became_active":"2023-09-12T16:02:05.031110+0000","last_became_peered":"2023-09-12T16:02:05.031110+0000","last_unstale":"2023-09-12T16:02:50.831192+0000","last_undegraded":"2023-09-12T16:02:50.831192+0000","last_fullsized":"2023-09-12T16:02:50.831192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:10:54.482078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833098+0000","last_change":"2023-09-12T16:02:09.291493+0000","last_active":"2023-09-12T16:02:50.833098+0000","last_peered":"2023-09-12T16:02:50.833098+0000","last_clean":"2023-09-12T16:02:50.833098+0000","last_became_active":"2023-09-12T16:02:09.291044+0000","last_became_peered":"2023-09-12T16:02:09.291044+0000","last_unstale":"2023-09-12T16:02:50.833098+0000","last_undegraded":"2023-09-12T16:02:50.833098+0000","last_fullsized":"2023-09-12T16:02:50.833098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:34:30.269720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835682+0000","last_change":"2023-09-12T16:02:04.029283+0000","last_active":"2023-09-12T16:02:50.835682+0000","last_peered":"2023-09-12T16:02:50.835682+0000","last_clean":"2023-09-12T16:02:50.835682+0000","last_became_active":"2023-09-12T16:02:04.028959+0000","last_became_peered":"2023-09-12T16:02:04.028959+0000","last_unstale":"2023-09-12T16:02:50.835682+0000","last_undegraded":"2023-09-12T16:02:50.835682+0000","last_fullsized":"2023-09-12T16:02:50.835682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:28:52.750278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834981+0000","last_change":"2023-09-12T16:02:05.034140+0000","last_active":"2023-09-12T16:02:50.834981+0000","last_peered":"2023-09-12T16:02:50.834981+0000","last_clean":"2023-09-12T16:02:50.834981+0000","last_became_active":"2023-09-12T16:02:05.033840+0000","last_became_peered":"2023-09-12T16:02:05.033840+0000","last_unstale":"2023-09-12T16:02:50.834981+0000","last_undegraded":"2023-09-12T16:02:50.834981+0000","last_fullsized":"2023-09-12T16:02:50.834981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:59:55.522608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025577+0000","last_change":"2023-09-12T16:02:09.289998+0000","last_active":"2023-09-12T16:02:51.025577+0000","last_peered":"2023-09-12T16:02:51.025577+0000","last_clean":"2023-09-12T16:02:51.025577+0000","last_became_active":"2023-09-12T16:02:09.289561+0000","last_became_peered":"2023-09-12T16:02:09.289561+0000","last_unstale":"2023-09-12T16:02:51.025577+0000","last_undegraded":"2023-09-12T16:02:51.025577+0000","last_fullsized":"2023-09-12T16:02:51.025577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:09:47.327213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832456+0000","last_change":"2023-09-12T16:02:04.029883+0000","last_active":"2023-09-12T16:02:50.832456+0000","last_peered":"2023-09-12T16:02:50.832456+0000","last_clean":"2023-09-12T16:02:50.832456+0000","last_became_active":"2023-09-12T16:02:04.029547+0000","last_became_peered":"2023-09-12T16:02:04.029547+0000","last_unstale":"2023-09-12T16:02:50.832456+0000","last_undegraded":"2023-09-12T16:02:50.832456+0000","last_fullsized":"2023-09-12T16:02:50.832456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:47:46.478325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846042+0000","last_change":"2023-09-12T16:02:51.846042+0000","last_active":"2023-09-12T16:02:51.846042+0000","last_peered":"2023-09-12T16:02:51.846042+0000","last_clean":"2023-09-12T16:02:51.846042+0000","last_became_active":"2023-09-12T16:02:51.845572+0000","last_became_peered":"2023-09-12T16:02:51.845572+0000","last_unstale":"2023-09-12T16:02:51.846042+0000","last_undegraded":"2023-09-12T16:02:51.846042+0000","last_fullsized":"2023-09-12T16:02:51.846042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:55.335485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069758+0000","last_change":"2023-09-12T16:02:52.069157+0000","last_active":"2023-09-12T16:02:52.069758+0000","last_peered":"2023-09-12T16:02:52.069758+0000","last_clean":"2023-09-12T16:02:52.069758+0000","last_became_active":"2023-09-12T16:02:52.068600+0000","last_became_peered":"2023-09-12T16:02:52.068600+0000","last_unstale":"2023-09-12T16:02:52.069758+0000","last_undegraded":"2023-09-12T16:02:52.069758+0000","last_fullsized":"2023-09-12T16:02:52.069758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:28:44.160612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:05.719196+0000","last_change":"2023-09-12T16:02:04.025723+0000","last_active":"2023-09-12T16:03:05.719196+0000","last_peered":"2023-09-12T16:03:05.719196+0000","last_clean":"2023-09-12T16:03:05.719196+0000","last_became_active":"2023-09-12T16:02:04.025432+0000","last_became_peered":"2023-09-12T16:02:04.025432+0000","last_unstale":"2023-09-12T16:03:05.719196+0000","last_undegraded":"2023-09-12T16:03:05.719196+0000","last_fullsized":"2023-09-12T16:03:05.719196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:56.279910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.839291+0000","last_change":"2023-09-12T16:02:51.839291+0000","last_active":"2023-09-12T16:02:51.839291+0000","last_peered":"2023-09-12T16:02:51.839291+0000","last_clean":"2023-09-12T16:02:51.839291+0000","last_became_active":"2023-09-12T16:02:51.838968+0000","last_became_peered":"2023-09-12T16:02:51.838968+0000","last_unstale":"2023-09-12T16:02:51.839291+0000","last_undegraded":"2023-09-12T16:02:51.839291+0000","last_fullsized":"2023-09-12T16:02:51.839291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:21:12.542019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835124+0000","last_change":"2023-09-12T16:02:09.286447+0000","last_active":"2023-09-12T16:02:50.835124+0000","last_peered":"2023-09-12T16:02:50.835124+0000","last_clean":"2023-09-12T16:02:50.835124+0000","last_became_active":"2023-09-12T16:02:09.285805+0000","last_became_peered":"2023-09-12T16:02:09.285805+0000","last_unstale":"2023-09-12T16:02:50.835124+0000","last_undegraded":"2023-09-12T16:02:50.835124+0000","last_fullsized":"2023-09-12T16:02:50.835124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:35:05.518872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830710+0000","last_change":"2023-09-12T16:02:04.032527+0000","last_active":"2023-09-12T16:02:50.830710+0000","last_peered":"2023-09-12T16:02:50.830710+0000","last_clean":"2023-09-12T16:02:50.830710+0000","last_became_active":"2023-09-12T16:02:04.032264+0000","last_became_peered":"2023-09-12T16:02:04.032264+0000","last_unstale":"2023-09-12T16:02:50.830710+0000","last_undegraded":"2023-09-12T16:02:50.830710+0000","last_fullsized":"2023-09-12T16:02:50.830710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:53:52.433549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833597+0000","last_change":"2023-09-12T16:02:05.029281+0000","last_active":"2023-09-12T16:02:50.833597+0000","last_peered":"2023-09-12T16:02:50.833597+0000","last_clean":"2023-09-12T16:02:50.833597+0000","last_became_active":"2023-09-12T16:02:05.028919+0000","last_became_peered":"2023-09-12T16:02:05.028919+0000","last_unstale":"2023-09-12T16:02:50.833597+0000","last_undegraded":"2023-09-12T16:02:50.833597+0000","last_fullsized":"2023-09-12T16:02:50.833597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:26.667060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069744+0000","last_change":"2023-09-12T16:02:52.069336+0000","last_active":"2023-09-12T16:02:52.069744+0000","last_peered":"2023-09-12T16:02:52.069744+0000","last_clean":"2023-09-12T16:02:52.069744+0000","last_became_active":"2023-09-12T16:02:52.068805+0000","last_became_peered":"2023-09-12T16:02:52.068805+0000","last_unstale":"2023-09-12T16:02:52.069744+0000","last_undegraded":"2023-09-12T16:02:52.069744+0000","last_fullsized":"2023-09-12T16:02:52.069744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:09.095952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831444+0000","last_change":"2023-09-12T16:02:04.026373+0000","last_active":"2023-09-12T16:02:50.831444+0000","last_peered":"2023-09-12T16:02:50.831444+0000","last_clean":"2023-09-12T16:02:50.831444+0000","last_became_active":"2023-09-12T16:02:04.025907+0000","last_became_peered":"2023-09-12T16:02:04.025907+0000","last_unstale":"2023-09-12T16:02:50.831444+0000","last_undegraded":"2023-09-12T16:02:50.831444+0000","last_fullsized":"2023-09-12T16:02:50.831444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:33:15.152128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025705+0000","last_change":"2023-09-12T16:02:05.040143+0000","last_active":"2023-09-12T16:02:51.025705+0000","last_peered":"2023-09-12T16:02:51.025705+0000","last_clean":"2023-09-12T16:02:51.025705+0000","last_became_active":"2023-09-12T16:02:05.039749+0000","last_became_peered":"2023-09-12T16:02:05.039749+0000","last_unstale":"2023-09-12T16:02:51.025705+0000","last_undegraded":"2023-09-12T16:02:51.025705+0000","last_fullsized":"2023-09-12T16:02:51.025705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:49.561252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:09.841670+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:09.841203+0000","last_became_peered":"2023-09-12T16:02:09.841203+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:55:46.128996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833178+0000","last_change":"2023-09-12T16:02:04.027953+0000","last_active":"2023-09-12T16:02:50.833178+0000","last_peered":"2023-09-12T16:02:50.833178+0000","last_clean":"2023-09-12T16:02:50.833178+0000","last_became_active":"2023-09-12T16:02:04.027624+0000","last_became_peered":"2023-09-12T16:02:04.027624+0000","last_unstale":"2023-09-12T16:02:50.833178+0000","last_undegraded":"2023-09-12T16:02:50.833178+0000","last_fullsized":"2023-09-12T16:02:50.833178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:33:58.356358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832388+0000","last_change":"2023-09-12T16:02:05.036593+0000","last_active":"2023-09-12T16:02:50.832388+0000","last_peered":"2023-09-12T16:02:50.832388+0000","last_clean":"2023-09-12T16:02:50.832388+0000","last_became_active":"2023-09-12T16:02:05.036204+0000","last_became_peered":"2023-09-12T16:02:05.036204+0000","last_unstale":"2023-09-12T16:02:50.832388+0000","last_undegraded":"2023-09-12T16:02:50.832388+0000","last_fullsized":"2023-09-12T16:02:50.832388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:12:31.068837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835475+0000","last_change":"2023-09-12T16:02:09.292405+0000","last_active":"2023-09-12T16:02:50.835475+0000","last_peered":"2023-09-12T16:02:50.835475+0000","last_clean":"2023-09-12T16:02:50.835475+0000","last_became_active":"2023-09-12T16:02:09.291877+0000","last_became_peered":"2023-09-12T16:02:09.291877+0000","last_unstale":"2023-09-12T16:02:50.835475+0000","last_undegraded":"2023-09-12T16:02:50.835475+0000","last_fullsized":"2023-09-12T16:02:50.835475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:53:04.847310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829465+0000","last_change":"2023-09-12T16:02:04.034738+0000","last_active":"2023-09-12T16:02:50.829465+0000","last_peered":"2023-09-12T16:02:50.829465+0000","last_clean":"2023-09-12T16:02:50.829465+0000","last_became_active":"2023-09-12T16:02:04.034408+0000","last_became_peered":"2023-09-12T16:02:04.034408+0000","last_unstale":"2023-09-12T16:02:50.829465+0000","last_undegraded":"2023-09-12T16:02:50.829465+0000","last_fullsized":"2023-09-12T16:02:50.829465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:27.845770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832464+0000","last_change":"2023-09-12T16:02:05.044936+0000","last_active":"2023-09-12T16:02:50.832464+0000","last_peered":"2023-09-12T16:02:50.832464+0000","last_clean":"2023-09-12T16:02:50.832464+0000","last_became_active":"2023-09-12T16:02:05.044617+0000","last_became_peered":"2023-09-12T16:02:05.044617+0000","last_unstale":"2023-09-12T16:02:50.832464+0000","last_undegraded":"2023-09-12T16:02:50.832464+0000","last_fullsized":"2023-09-12T16:02:50.832464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:41:32.639331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834992+0000","last_change":"2023-09-12T16:02:09.841919+0000","last_active":"2023-09-12T16:02:50.834992+0000","last_peered":"2023-09-12T16:02:50.834992+0000","last_clean":"2023-09-12T16:02:50.834992+0000","last_became_active":"2023-09-12T16:02:09.841410+0000","last_became_peered":"2023-09-12T16:02:09.841410+0000","last_unstale":"2023-09-12T16:02:50.834992+0000","last_undegraded":"2023-09-12T16:02:50.834992+0000","last_fullsized":"2023-09-12T16:02:50.834992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:29:57.511693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.840116+0000","last_change":"2023-09-12T16:02:51.840116+0000","last_active":"2023-09-12T16:02:51.840116+0000","last_peered":"2023-09-12T16:02:51.840116+0000","last_clean":"2023-09-12T16:02:51.840116+0000","last_became_active":"2023-09-12T16:02:51.839698+0000","last_became_peered":"2023-09-12T16:02:51.839698+0000","last_unstale":"2023-09-12T16:02:51.840116+0000","last_undegraded":"2023-09-12T16:02:51.840116+0000","last_fullsized":"2023-09-12T16:02:51.840116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:29.475855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":63,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:02.714131+0000","last_change":"2023-09-12T16:02:04.032892+0000","last_active":"2023-09-12T16:03:02.714131+0000","last_peered":"2023-09-12T16:03:02.714131+0000","last_clean":"2023-09-12T16:03:02.714131+0000","last_became_active":"2023-09-12T16:02:04.032637+0000","last_became_peered":"2023-09-12T16:02:04.032637+0000","last_unstale":"2023-09-12T16:03:02.714131+0000","last_undegraded":"2023-09-12T16:03:02.714131+0000","last_fullsized":"2023-09-12T16:03:02.714131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:22:55.805000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830231+0000","last_change":"2023-09-12T16:02:09.285127+0000","last_active":"2023-09-12T16:02:50.830231+0000","last_peered":"2023-09-12T16:02:50.830231+0000","last_clean":"2023-09-12T16:02:50.830231+0000","last_became_active":"2023-09-12T16:02:09.284790+0000","last_became_peered":"2023-09-12T16:02:09.284790+0000","last_unstale":"2023-09-12T16:02:50.830231+0000","last_undegraded":"2023-09-12T16:02:50.830231+0000","last_fullsized":"2023-09-12T16:02:50.830231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:50:50.578069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832025+0000","last_change":"2023-09-12T16:02:05.027877+0000","last_active":"2023-09-12T16:02:50.832025+0000","last_peered":"2023-09-12T16:02:50.832025+0000","last_clean":"2023-09-12T16:02:50.832025+0000","last_became_active":"2023-09-12T16:02:05.027582+0000","last_became_peered":"2023-09-12T16:02:05.027582+0000","last_unstale":"2023-09-12T16:02:50.832025+0000","last_undegraded":"2023-09-12T16:02:50.832025+0000","last_fullsized":"2023-09-12T16:02:50.832025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:47.835790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835647+0000","last_change":"2023-09-12T16:02:04.024800+0000","last_active":"2023-09-12T16:02:50.835647+0000","last_peered":"2023-09-12T16:02:50.835647+0000","last_clean":"2023-09-12T16:02:50.835647+0000","last_became_active":"2023-09-12T16:02:04.024436+0000","last_became_peered":"2023-09-12T16:02:04.024436+0000","last_unstale":"2023-09-12T16:02:50.835647+0000","last_undegraded":"2023-09-12T16:02:50.835647+0000","last_fullsized":"2023-09-12T16:02:50.835647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:57:35.093322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829225+0000","last_change":"2023-09-12T16:01:04.599706+0000","last_active":"2023-09-12T16:02:50.829225+0000","last_peered":"2023-09-12T16:02:50.829225+0000","last_clean":"2023-09-12T16:02:50.829225+0000","last_became_active":"2023-09-12T16:01:04.599328+0000","last_became_peered":"2023-09-12T16:01:04.599328+0000","last_unstale":"2023-09-12T16:02:50.829225+0000","last_undegraded":"2023-09-12T16:02:50.829225+0000","last_fullsized":"2023-09-12T16:02:50.829225+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_clean_scrub_stamp":"2023-09-12T15:58:15.250648+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:57:56.846488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833462+0000","last_change":"2023-09-12T16:02:09.841960+0000","last_active":"2023-09-12T16:02:50.833462+0000","last_peered":"2023-09-12T16:02:50.833462+0000","last_clean":"2023-09-12T16:02:50.833462+0000","last_became_active":"2023-09-12T16:02:09.841279+0000","last_became_peered":"2023-09-12T16:02:09.841279+0000","last_unstale":"2023-09-12T16:02:50.833462+0000","last_undegraded":"2023-09-12T16:02:50.833462+0000","last_fullsized":"2023-09-12T16:02:50.833462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:17:19.419350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.847600+0000","last_change":"2023-09-12T16:02:51.847600+0000","last_active":"2023-09-12T16:02:51.847600+0000","last_peered":"2023-09-12T16:02:51.847600+0000","last_clean":"2023-09-12T16:02:51.847600+0000","last_became_active":"2023-09-12T16:02:51.847292+0000","last_became_peered":"2023-09-12T16:02:51.847292+0000","last_unstale":"2023-09-12T16:02:51.847600+0000","last_undegraded":"2023-09-12T16:02:51.847600+0000","last_fullsized":"2023-09-12T16:02:51.847600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:25:58.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829344+0000","last_change":"2023-09-12T16:02:04.029790+0000","last_active":"2023-09-12T16:02:50.829344+0000","last_peered":"2023-09-12T16:02:50.829344+0000","last_clean":"2023-09-12T16:02:50.829344+0000","last_became_active":"2023-09-12T16:02:04.029440+0000","last_became_peered":"2023-09-12T16:02:04.029440+0000","last_unstale":"2023-09-12T16:02:50.829344+0000","last_undegraded":"2023-09-12T16:02:50.829344+0000","last_fullsized":"2023-09-12T16:02:50.829344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:50:33.690628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025552+0000","last_change":"2023-09-12T16:02:09.289757+0000","last_active":"2023-09-12T16:02:51.025552+0000","last_peered":"2023-09-12T16:02:51.025552+0000","last_clean":"2023-09-12T16:02:51.025552+0000","last_became_active":"2023-09-12T16:02:09.289348+0000","last_became_peered":"2023-09-12T16:02:09.289348+0000","last_unstale":"2023-09-12T16:02:51.025552+0000","last_undegraded":"2023-09-12T16:02:51.025552+0000","last_fullsized":"2023-09-12T16:02:51.025552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:15:14.695521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831296+0000","last_change":"2023-09-12T16:02:04.033213+0000","last_active":"2023-09-12T16:02:50.831296+0000","last_peered":"2023-09-12T16:02:50.831296+0000","last_clean":"2023-09-12T16:02:50.831296+0000","last_became_active":"2023-09-12T16:02:04.032978+0000","last_became_peered":"2023-09-12T16:02:04.032978+0000","last_unstale":"2023-09-12T16:02:50.831296+0000","last_undegraded":"2023-09-12T16:02:50.831296+0000","last_fullsized":"2023-09-12T16:02:50.831296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:25:33.718954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835250+0000","last_change":"2023-09-12T16:02:05.043252+0000","last_active":"2023-09-12T16:02:50.835250+0000","last_peered":"2023-09-12T16:02:50.835250+0000","last_clean":"2023-09-12T16:02:50.835250+0000","last_became_active":"2023-09-12T16:02:05.042975+0000","last_became_peered":"2023-09-12T16:02:05.042975+0000","last_unstale":"2023-09-12T16:02:50.835250+0000","last_undegraded":"2023-09-12T16:02:50.835250+0000","last_fullsized":"2023-09-12T16:02:50.835250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:46:31.549547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831861+0000","last_change":"2023-09-12T16:02:09.841728+0000","last_active":"2023-09-12T16:02:50.831861+0000","last_peered":"2023-09-12T16:02:50.831861+0000","last_clean":"2023-09-12T16:02:50.831861+0000","last_became_active":"2023-09-12T16:02:09.841351+0000","last_became_peered":"2023-09-12T16:02:09.841351+0000","last_unstale":"2023-09-12T16:02:50.831861+0000","last_undegraded":"2023-09-12T16:02:50.831861+0000","last_fullsized":"2023-09-12T16:02:50.831861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:00:50.637272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833531+0000","last_change":"2023-09-12T16:02:04.025624+0000","last_active":"2023-09-12T16:02:50.833531+0000","last_peered":"2023-09-12T16:02:50.833531+0000","last_clean":"2023-09-12T16:02:50.833531+0000","last_became_active":"2023-09-12T16:02:04.025330+0000","last_became_peered":"2023-09-12T16:02:04.025330+0000","last_unstale":"2023-09-12T16:02:50.833531+0000","last_undegraded":"2023-09-12T16:02:50.833531+0000","last_fullsized":"2023-09-12T16:02:50.833531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:00:57.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833174+0000","last_change":"2023-09-12T16:02:05.028414+0000","last_active":"2023-09-12T16:02:50.833174+0000","last_peered":"2023-09-12T16:02:50.833174+0000","last_clean":"2023-09-12T16:02:50.833174+0000","last_became_active":"2023-09-12T16:02:05.028081+0000","last_became_peered":"2023-09-12T16:02:05.028081+0000","last_unstale":"2023-09-12T16:02:50.833174+0000","last_undegraded":"2023-09-12T16:02:50.833174+0000","last_fullsized":"2023-09-12T16:02:50.833174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:57:58.245105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834801+0000","last_change":"2023-09-12T16:02:09.283222+0000","last_active":"2023-09-12T16:02:50.834801+0000","last_peered":"2023-09-12T16:02:50.834801+0000","last_clean":"2023-09-12T16:02:50.834801+0000","last_became_active":"2023-09-12T16:02:09.282814+0000","last_became_peered":"2023-09-12T16:02:09.282814+0000","last_unstale":"2023-09-12T16:02:50.834801+0000","last_undegraded":"2023-09-12T16:02:50.834801+0000","last_fullsized":"2023-09-12T16:02:50.834801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:18.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836561+0000","last_change":"2023-09-12T16:02:04.026694+0000","last_active":"2023-09-12T16:02:50.836561+0000","last_peered":"2023-09-12T16:02:50.836561+0000","last_clean":"2023-09-12T16:02:50.836561+0000","last_became_active":"2023-09-12T16:02:04.026368+0000","last_became_peered":"2023-09-12T16:02:04.026368+0000","last_unstale":"2023-09-12T16:02:50.836561+0000","last_undegraded":"2023-09-12T16:02:50.836561+0000","last_fullsized":"2023-09-12T16:02:50.836561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:14:10.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829216+0000","last_change":"2023-09-12T16:02:05.039037+0000","last_active":"2023-09-12T16:02:50.829216+0000","last_peered":"2023-09-12T16:02:50.829216+0000","last_clean":"2023-09-12T16:02:50.829216+0000","last_became_active":"2023-09-12T16:02:05.038708+0000","last_became_peered":"2023-09-12T16:02:05.038708+0000","last_unstale":"2023-09-12T16:02:50.829216+0000","last_undegraded":"2023-09-12T16:02:50.829216+0000","last_fullsized":"2023-09-12T16:02:50.829216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:13:09.805646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836160+0000","last_change":"2023-09-12T16:02:09.841757+0000","last_active":"2023-09-12T16:02:50.836160+0000","last_peered":"2023-09-12T16:02:50.836160+0000","last_clean":"2023-09-12T16:02:50.836160+0000","last_became_active":"2023-09-12T16:02:09.841173+0000","last_became_peered":"2023-09-12T16:02:09.841173+0000","last_unstale":"2023-09-12T16:02:50.836160+0000","last_undegraded":"2023-09-12T16:02:50.836160+0000","last_fullsized":"2023-09-12T16:02:50.836160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:57:06.626828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836147+0000","last_change":"2023-09-12T16:02:04.031009+0000","last_active":"2023-09-12T16:02:50.836147+0000","last_peered":"2023-09-12T16:02:50.836147+0000","last_clean":"2023-09-12T16:02:50.836147+0000","last_became_active":"2023-09-12T16:02:04.030714+0000","last_became_peered":"2023-09-12T16:02:04.030714+0000","last_unstale":"2023-09-12T16:02:50.836147+0000","last_undegraded":"2023-09-12T16:02:50.836147+0000","last_fullsized":"2023-09-12T16:02:50.836147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:12.578686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830789+0000","last_change":"2023-09-12T16:02:05.030778+0000","last_active":"2023-09-12T16:02:50.830789+0000","last_peered":"2023-09-12T16:02:50.830789+0000","last_clean":"2023-09-12T16:02:50.830789+0000","last_became_active":"2023-09-12T16:02:05.030454+0000","last_became_peered":"2023-09-12T16:02:05.030454+0000","last_unstale":"2023-09-12T16:02:50.830789+0000","last_undegraded":"2023-09-12T16:02:50.830789+0000","last_fullsized":"2023-09-12T16:02:50.830789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:52.951246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834030+0000","last_change":"2023-09-12T16:02:09.289668+0000","last_active":"2023-09-12T16:02:50.834030+0000","last_peered":"2023-09-12T16:02:50.834030+0000","last_clean":"2023-09-12T16:02:50.834030+0000","last_became_active":"2023-09-12T16:02:09.289301+0000","last_became_peered":"2023-09-12T16:02:09.289301+0000","last_unstale":"2023-09-12T16:02:50.834030+0000","last_undegraded":"2023-09-12T16:02:50.834030+0000","last_fullsized":"2023-09-12T16:02:50.834030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:20:04.528146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832522+0000","last_change":"2023-09-12T16:02:04.036175+0000","last_active":"2023-09-12T16:02:50.832522+0000","last_peered":"2023-09-12T16:02:50.832522+0000","last_clean":"2023-09-12T16:02:50.832522+0000","last_became_active":"2023-09-12T16:02:04.035879+0000","last_became_peered":"2023-09-12T16:02:04.035879+0000","last_unstale":"2023-09-12T16:02:50.832522+0000","last_undegraded":"2023-09-12T16:02:50.832522+0000","last_fullsized":"2023-09-12T16:02:50.832522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:29:20.798716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025379+0000","last_change":"2023-09-12T16:02:05.034195+0000","last_active":"2023-09-12T16:02:51.025379+0000","last_peered":"2023-09-12T16:02:51.025379+0000","last_clean":"2023-09-12T16:02:51.025379+0000","last_became_active":"2023-09-12T16:02:05.033534+0000","last_became_peered":"2023-09-12T16:02:05.033534+0000","last_unstale":"2023-09-12T16:02:51.025379+0000","last_undegraded":"2023-09-12T16:02:51.025379+0000","last_fullsized":"2023-09-12T16:02:51.025379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:34:04.180218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834899+0000","last_change":"2023-09-12T16:02:09.287543+0000","last_active":"2023-09-12T16:02:50.834899+0000","last_peered":"2023-09-12T16:02:50.834899+0000","last_clean":"2023-09-12T16:02:50.834899+0000","last_became_active":"2023-09-12T16:02:09.287055+0000","last_became_peered":"2023-09-12T16:02:09.287055+0000","last_unstale":"2023-09-12T16:02:50.834899+0000","last_undegraded":"2023-09-12T16:02:50.834899+0000","last_fullsized":"2023-09-12T16:02:50.834899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:58.214381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834994+0000","last_change":"2023-09-12T16:02:04.030206+0000","last_active":"2023-09-12T16:02:50.834994+0000","last_peered":"2023-09-12T16:02:50.834994+0000","last_clean":"2023-09-12T16:02:50.834994+0000","last_became_active":"2023-09-12T16:02:04.029765+0000","last_became_peered":"2023-09-12T16:02:04.029765+0000","last_unstale":"2023-09-12T16:02:50.834994+0000","last_undegraded":"2023-09-12T16:02:50.834994+0000","last_fullsized":"2023-09-12T16:02:50.834994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:56:07.551681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832663+0000","last_change":"2023-09-12T16:02:05.028300+0000","last_active":"2023-09-12T16:02:50.832663+0000","last_peered":"2023-09-12T16:02:50.832663+0000","last_clean":"2023-09-12T16:02:50.832663+0000","last_became_active":"2023-09-12T16:02:05.027970+0000","last_became_peered":"2023-09-12T16:02:05.027970+0000","last_unstale":"2023-09-12T16:02:50.832663+0000","last_undegraded":"2023-09-12T16:02:50.832663+0000","last_fullsized":"2023-09-12T16:02:50.832663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:39:44.571593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835970+0000","last_change":"2023-09-12T16:02:09.841661+0000","last_active":"2023-09-12T16:02:50.835970+0000","last_peered":"2023-09-12T16:02:50.835970+0000","last_clean":"2023-09-12T16:02:50.835970+0000","last_became_active":"2023-09-12T16:02:09.841039+0000","last_became_peered":"2023-09-12T16:02:09.841039+0000","last_unstale":"2023-09-12T16:02:50.835970+0000","last_undegraded":"2023-09-12T16:02:50.835970+0000","last_fullsized":"2023-09-12T16:02:50.835970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:34.529098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830867+0000","last_change":"2023-09-12T16:02:04.031927+0000","last_active":"2023-09-12T16:02:50.830867+0000","last_peered":"2023-09-12T16:02:50.830867+0000","last_clean":"2023-09-12T16:02:50.830867+0000","last_became_active":"2023-09-12T16:02:04.031614+0000","last_became_peered":"2023-09-12T16:02:04.031614+0000","last_unstale":"2023-09-12T16:02:50.830867+0000","last_undegraded":"2023-09-12T16:02:50.830867+0000","last_fullsized":"2023-09-12T16:02:50.830867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:00:23.423933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829483+0000","last_change":"2023-09-12T16:02:05.032647+0000","last_active":"2023-09-12T16:02:50.829483+0000","last_peered":"2023-09-12T16:02:50.829483+0000","last_clean":"2023-09-12T16:02:50.829483+0000","last_became_active":"2023-09-12T16:02:05.031701+0000","last_became_peered":"2023-09-12T16:02:05.031701+0000","last_unstale":"2023-09-12T16:02:50.829483+0000","last_undegraded":"2023-09-12T16:02:50.829483+0000","last_fullsized":"2023-09-12T16:02:50.829483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:02:19.596913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837136+0000","last_change":"2023-09-12T16:02:09.291043+0000","last_active":"2023-09-12T16:02:50.837136+0000","last_peered":"2023-09-12T16:02:50.837136+0000","last_clean":"2023-09-12T16:02:50.837136+0000","last_became_active":"2023-09-12T16:02:09.290666+0000","last_became_peered":"2023-09-12T16:02:09.290666+0000","last_unstale":"2023-09-12T16:02:50.837136+0000","last_undegraded":"2023-09-12T16:02:50.837136+0000","last_fullsized":"2023-09-12T16:02:50.837136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:10:47.354479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835888+0000","last_change":"2023-09-12T16:02:04.030501+0000","last_active":"2023-09-12T16:02:50.835888+0000","last_peered":"2023-09-12T16:02:50.835888+0000","last_clean":"2023-09-12T16:02:50.835888+0000","last_became_active":"2023-09-12T16:02:04.030196+0000","last_became_peered":"2023-09-12T16:02:04.030196+0000","last_unstale":"2023-09-12T16:02:50.835888+0000","last_undegraded":"2023-09-12T16:02:50.835888+0000","last_fullsized":"2023-09-12T16:02:50.835888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:58:34.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830617+0000","last_change":"2023-09-12T16:02:05.030466+0000","last_active":"2023-09-12T16:02:50.830617+0000","last_peered":"2023-09-12T16:02:50.830617+0000","last_clean":"2023-09-12T16:02:50.830617+0000","last_became_active":"2023-09-12T16:02:05.029988+0000","last_became_peered":"2023-09-12T16:02:05.029988+0000","last_unstale":"2023-09-12T16:02:50.830617+0000","last_undegraded":"2023-09-12T16:02:50.830617+0000","last_fullsized":"2023-09-12T16:02:50.830617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:08:56.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846403+0000","last_change":"2023-09-12T16:02:51.846403+0000","last_active":"2023-09-12T16:02:51.846403+0000","last_peered":"2023-09-12T16:02:51.846403+0000","last_clean":"2023-09-12T16:02:51.846403+0000","last_became_active":"2023-09-12T16:02:51.846047+0000","last_became_peered":"2023-09-12T16:02:51.846047+0000","last_unstale":"2023-09-12T16:02:51.846403+0000","last_undegraded":"2023-09-12T16:02:51.846403+0000","last_fullsized":"2023-09-12T16:02:51.846403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:58:11.458339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832945+0000","last_change":"2023-09-12T16:02:04.027380+0000","last_active":"2023-09-12T16:02:50.832945+0000","last_peered":"2023-09-12T16:02:50.832945+0000","last_clean":"2023-09-12T16:02:50.832945+0000","last_became_active":"2023-09-12T16:02:04.027098+0000","last_became_peered":"2023-09-12T16:02:04.027098+0000","last_unstale":"2023-09-12T16:02:50.832945+0000","last_undegraded":"2023-09-12T16:02:50.832945+0000","last_fullsized":"2023-09-12T16:02:50.832945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:34:12.260747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829993+0000","last_change":"2023-09-12T16:02:05.029249+0000","last_active":"2023-09-12T16:02:50.829993+0000","last_peered":"2023-09-12T16:02:50.829993+0000","last_clean":"2023-09-12T16:02:50.829993+0000","last_became_active":"2023-09-12T16:02:05.028955+0000","last_became_peered":"2023-09-12T16:02:05.028955+0000","last_unstale":"2023-09-12T16:02:50.829993+0000","last_undegraded":"2023-09-12T16:02:50.829993+0000","last_fullsized":"2023-09-12T16:02:50.829993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:02:10.382712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831923+0000","last_change":"2023-09-12T16:02:09.288724+0000","last_active":"2023-09-12T16:02:50.831923+0000","last_peered":"2023-09-12T16:02:50.831923+0000","last_clean":"2023-09-12T16:02:50.831923+0000","last_became_active":"2023-09-12T16:02:09.288206+0000","last_became_peered":"2023-09-12T16:02:09.288206+0000","last_unstale":"2023-09-12T16:02:50.831923+0000","last_undegraded":"2023-09-12T16:02:50.831923+0000","last_fullsized":"2023-09-12T16:02:50.831923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:45:10.154195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835225+0000","last_change":"2023-09-12T16:02:04.031219+0000","last_active":"2023-09-12T16:02:50.835225+0000","last_peered":"2023-09-12T16:02:50.835225+0000","last_clean":"2023-09-12T16:02:50.835225+0000","last_became_active":"2023-09-12T16:02:04.030867+0000","last_became_peered":"2023-09-12T16:02:04.030867+0000","last_unstale":"2023-09-12T16:02:50.835225+0000","last_undegraded":"2023-09-12T16:02:50.835225+0000","last_fullsized":"2023-09-12T16:02:50.835225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:16.885094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831081+0000","last_change":"2023-09-12T16:02:05.028004+0000","last_active":"2023-09-12T16:02:50.831081+0000","last_peered":"2023-09-12T16:02:50.831081+0000","last_clean":"2023-09-12T16:02:50.831081+0000","last_became_active":"2023-09-12T16:02:05.027668+0000","last_became_peered":"2023-09-12T16:02:05.027668+0000","last_unstale":"2023-09-12T16:02:50.831081+0000","last_undegraded":"2023-09-12T16:02:50.831081+0000","last_fullsized":"2023-09-12T16:02:50.831081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:41:23.019243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025479+0000","last_change":"2023-09-12T16:02:05.034380+0000","last_active":"2023-09-12T16:02:51.025479+0000","last_peered":"2023-09-12T16:02:51.025479+0000","last_clean":"2023-09-12T16:02:51.025479+0000","last_became_active":"2023-09-12T16:02:05.034022+0000","last_became_peered":"2023-09-12T16:02:05.034022+0000","last_unstale":"2023-09-12T16:02:51.025479+0000","last_undegraded":"2023-09-12T16:02:51.025479+0000","last_fullsized":"2023-09-12T16:02:51.025479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:55:56.404742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835155+0000","last_change":"2023-09-12T16:02:05.031723+0000","last_active":"2023-09-12T16:02:50.835155+0000","last_peered":"2023-09-12T16:02:50.835155+0000","last_clean":"2023-09-12T16:02:50.835155+0000","last_became_active":"2023-09-12T16:02:05.030954+0000","last_became_peered":"2023-09-12T16:02:05.030954+0000","last_unstale":"2023-09-12T16:02:50.835155+0000","last_undegraded":"2023-09-12T16:02:50.835155+0000","last_fullsized":"2023-09-12T16:02:50.835155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.319954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834938+0000","last_change":"2023-09-12T16:02:05.034851+0000","last_active":"2023-09-12T16:02:50.834938+0000","last_peered":"2023-09-12T16:02:50.834938+0000","last_clean":"2023-09-12T16:02:50.834938+0000","last_became_active":"2023-09-12T16:02:05.030733+0000","last_became_peered":"2023-09-12T16:02:05.030733+0000","last_unstale":"2023-09-12T16:02:50.834938+0000","last_undegraded":"2023-09-12T16:02:50.834938+0000","last_fullsized":"2023-09-12T16:02:50.834938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:33.605216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830111+0000","last_change":"2023-09-12T16:02:05.037331+0000","last_active":"2023-09-12T16:02:50.830111+0000","last_peered":"2023-09-12T16:02:50.830111+0000","last_clean":"2023-09-12T16:02:50.830111+0000","last_became_active":"2023-09-12T16:02:05.037015+0000","last_became_peered":"2023-09-12T16:02:05.037015+0000","last_unstale":"2023-09-12T16:02:50.830111+0000","last_undegraded":"2023-09-12T16:02:50.830111+0000","last_fullsized":"2023-09-12T16:02:50.830111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:29:14.035891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835319+0000","last_change":"2023-09-12T16:02:05.039803+0000","last_active":"2023-09-12T16:02:50.835319+0000","last_peered":"2023-09-12T16:02:50.835319+0000","last_clean":"2023-09-12T16:02:50.835319+0000","last_became_active":"2023-09-12T16:02:05.039390+0000","last_became_peered":"2023-09-12T16:02:05.039390+0000","last_unstale":"2023-09-12T16:02:50.835319+0000","last_undegraded":"2023-09-12T16:02:50.835319+0000","last_fullsized":"2023-09-12T16:02:50.835319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:22.422090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025700+0000","last_change":"2023-09-12T16:02:05.033153+0000","last_active":"2023-09-12T16:02:51.025700+0000","last_peered":"2023-09-12T16:02:51.025700+0000","last_clean":"2023-09-12T16:02:51.025700+0000","last_became_active":"2023-09-12T16:02:05.032800+0000","last_became_peered":"2023-09-12T16:02:05.032800+0000","last_unstale":"2023-09-12T16:02:51.025700+0000","last_undegraded":"2023-09-12T16:02:51.025700+0000","last_fullsized":"2023-09-12T16:02:51.025700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:31:43.090031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836428+0000","last_change":"2023-09-12T16:02:05.031519+0000","last_active":"2023-09-12T16:02:50.836428+0000","last_peered":"2023-09-12T16:02:50.836428+0000","last_clean":"2023-09-12T16:02:50.836428+0000","last_became_active":"2023-09-12T16:02:05.030227+0000","last_became_peered":"2023-09-12T16:02:05.030227+0000","last_unstale":"2023-09-12T16:02:50.836428+0000","last_undegraded":"2023-09-12T16:02:50.836428+0000","last_fullsized":"2023-09-12T16:02:50.836428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:00.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833413+0000","last_change":"2023-09-12T16:02:05.031763+0000","last_active":"2023-09-12T16:02:50.833413+0000","last_peered":"2023-09-12T16:02:50.833413+0000","last_clean":"2023-09-12T16:02:50.833413+0000","last_became_active":"2023-09-12T16:02:05.031441+0000","last_became_peered":"2023-09-12T16:02:05.031441+0000","last_unstale":"2023-09-12T16:02:50.833413+0000","last_undegraded":"2023-09-12T16:02:50.833413+0000","last_fullsized":"2023-09-12T16:02:50.833413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:55.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832249+0000","last_change":"2023-09-12T16:02:05.033786+0000","last_active":"2023-09-12T16:02:50.832249+0000","last_peered":"2023-09-12T16:02:50.832249+0000","last_clean":"2023-09-12T16:02:50.832249+0000","last_became_active":"2023-09-12T16:02:05.033497+0000","last_became_peered":"2023-09-12T16:02:05.033497+0000","last_unstale":"2023-09-12T16:02:50.832249+0000","last_undegraded":"2023-09-12T16:02:50.832249+0000","last_fullsized":"2023-09-12T16:02:50.832249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:01:58.912488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832560+0000","last_change":"2023-09-12T16:02:05.031144+0000","last_active":"2023-09-12T16:02:50.832560+0000","last_peered":"2023-09-12T16:02:50.832560+0000","last_clean":"2023-09-12T16:02:50.832560+0000","last_became_active":"2023-09-12T16:02:05.030827+0000","last_became_peered":"2023-09-12T16:02:05.030827+0000","last_unstale":"2023-09-12T16:02:50.832560+0000","last_undegraded":"2023-09-12T16:02:50.832560+0000","last_fullsized":"2023-09-12T16:02:50.832560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:40:48.014344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837378+0000","last_change":"2023-09-12T16:02:05.021273+0000","last_active":"2023-09-12T16:02:50.837378+0000","last_peered":"2023-09-12T16:02:50.837378+0000","last_clean":"2023-09-12T16:02:50.837378+0000","last_became_active":"2023-09-12T16:02:05.020953+0000","last_became_peered":"2023-09-12T16:02:05.020953+0000","last_unstale":"2023-09-12T16:02:50.837378+0000","last_undegraded":"2023-09-12T16:02:50.837378+0000","last_fullsized":"2023-09-12T16:02:50.837378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:33:25.462412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835081+0000","last_change":"2023-09-12T16:02:05.039019+0000","last_active":"2023-09-12T16:02:50.835081+0000","last_peered":"2023-09-12T16:02:50.835081+0000","last_clean":"2023-09-12T16:02:50.835081+0000","last_became_active":"2023-09-12T16:02:05.038621+0000","last_became_peered":"2023-09-12T16:02:05.038621+0000","last_unstale":"2023-09-12T16:02:50.835081+0000","last_undegraded":"2023-09-12T16:02:50.835081+0000","last_fullsized":"2023-09-12T16:02:50.835081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:08:08.934375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025761+0000","last_change":"2023-09-12T16:02:05.039737+0000","last_active":"2023-09-12T16:02:51.025761+0000","last_peered":"2023-09-12T16:02:51.025761+0000","last_clean":"2023-09-12T16:02:51.025761+0000","last_became_active":"2023-09-12T16:02:05.039304+0000","last_became_peered":"2023-09-12T16:02:05.039304+0000","last_unstale":"2023-09-12T16:02:51.025761+0000","last_undegraded":"2023-09-12T16:02:51.025761+0000","last_fullsized":"2023-09-12T16:02:51.025761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:50:13.141915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244717,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65275,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":257,"num_read_kb":192,"num_write":154,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":59815,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":195825,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":489432,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70199999999999996}]},{"osd":1,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.755}]},{"osd":2,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.44},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78100000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.72599999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":9,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68799999999999994}]},{"osd":10,"last update":"Tue Sep 12 16:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.93400000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776155,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1137975,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.5,"15min":0.501},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80000000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.72699999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.629,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.84099999999999997}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82899999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.58999999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.90800000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.82399999999999995}]},{"osd":6,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95299999999999996,"5min":1.042,"15min":1.042},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.56499999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.85099999999999998,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.90100000000000002}]},{"osd":8,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.63700000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.008,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93400000000000005,"5min":1.073,"15min":1.073},"last":0.88100000000000001}]},{"osd":9,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96299999999999997,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.099,"15min":1.099},"last":0.96999999999999997}]},{"osd":11,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.0069999999999999}]}]},{"osd":9,"up_from":63,"seq":270582939681,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":493764,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.48599999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.42799999999999999,"15min":0.436},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.82499999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.40600000000000003}]},{"osd":1,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46100000000000002}]},{"osd":2,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70499999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64800000000000002}]},{"osd":3,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.46500000000000002,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.318},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.504}]},{"osd":4,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.72099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.432}]},{"osd":5,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.749},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81799999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70599999999999996}]},{"osd":6,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88700000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52600000000000002}]},{"osd":8,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.70099999999999996,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.80800000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67300000000000004}]},{"osd":10,"last update":"Tue Sep 12 16:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168612,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":488675,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.496},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67500000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45100000000000001}]},{"osd":1,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.44400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.70799999999999996}]},{"osd":2,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.45300000000000001,"15min":0.44},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.45700000000000002,"15min":0.44500000000000001},"min":{"1min":0.34300000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.43099999999999999}]},{"osd":3,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57399999999999995}]},{"osd":4,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.45300000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.51600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.78700000000000003}]},{"osd":5,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.61299999999999999}]},{"osd":6,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.46800000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97199999999999998,"5min":1.075,"15min":1.075},"last":0.76800000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72999999999999998,"15min":0.72499999999999998},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0069999999999999,"5min":1.038,"15min":1.038},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.499,"5min":0.435,"15min":0.435},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.75800000000000001}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397545,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1085555,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.40200000000000002,"15min":0.38600000000000001},"min":{"1min":0.34200000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63600000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.432},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.433,"15min":0.40699999999999997},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45600000000000002}]},{"osd":2,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59099999999999997,"15min":0.55800000000000005},"min":{"1min":0.439,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1639999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0640000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.91200000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.35099999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72599999999999998,"5min":1.069,"15min":1.069},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65100000000000002,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93999999999999995,"5min":1.909,"15min":1.909},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65000000000000002,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.139,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96299999999999997,"5min":1.119,"15min":1.119},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84699999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67900000000000005}]},{"osd":8,"last update":"Tue Sep 12 16:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":1.2030000000000001,"5min":1.879,"15min":1.879},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.49299999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.083,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.81200000000000006}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96499999999999997,"5min":1.329,"15min":1.329},"last":0.80300000000000005}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.0309999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.59799999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561069,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":484844,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.39200000000000002,"15min":0.39000000000000001},"min":{"1min":0.25600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56999999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34399999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70099999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.69799999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71899999999999997,"5min":1.079,"15min":1.079},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":1.129,"15min":1.129},"last":0.59299999999999997}]},{"osd":3,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.75,"5min":1.083,"15min":1.083},"last":0.37},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.68300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.443}]},{"osd":4,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72699999999999998,"5min":1.161,"15min":1.161},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81999999999999995,"5min":1.022,"15min":1.022},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.75}]},{"osd":8,"last update":"Tue Sep 12 16:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69399999999999995,"15min":0.70799999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.84299999999999997,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70199999999999996}]},{"osd":9,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71799999999999997,"15min":0.72899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.46300000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90800000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.61599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":545948,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.38100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58299999999999996}]},{"osd":2,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.51400000000000001},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56299999999999994,"15min":0.52600000000000002},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.79700000000000004}]},{"osd":3,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57099999999999995,"15min":0.51900000000000002},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":1.002,"15min":1.002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60199999999999998}]},{"osd":4,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.498,"15min":0.45600000000000002},"min":{"1min":0.40899999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.57499999999999996,"15min":0.51900000000000002},"min":{"1min":0.52400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.85099999999999998}]},{"osd":6,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.77400000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.66900000000000004,"15min":0.623},"min":{"1min":0.63,"5min":0.317,"15min":0.317},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.68100000000000005,"15min":0.63700000000000001},"min":{"1min":0.70499999999999996,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.70499999999999996}]},{"osd":8,"last update":"Tue Sep 12 16:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.74199999999999999,"15min":0.72699999999999998},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.751,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.51100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.78700000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79100000000000004,"15min":0.77900000000000003},"min":{"1min":0.58399999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.83999999999999997}]},{"osd":10,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.67300000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.92100000000000004}]},{"osd":11,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.95599999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":486232,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58099999999999996,"15min":0.60599999999999998},"min":{"1min":0.38300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82999999999999996,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.48699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":2.113,"15min":2.113},"last":0.65200000000000002}]},{"osd":2,"last update":"Tue Sep 12 16:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.874},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.754,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.39900000000000002}]},{"osd":3,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.35499999999999998}]},{"osd":4,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.442},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Sep 12 16:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53000000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.74399999999999999,"5min":0.879,"15min":0.879},"last":0.495},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":1.004,"15min":1.004},"last":0.46899999999999997}]},{"osd":7,"last update":"Tue Sep 12 16:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48199999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69299999999999995,"15min":0.70299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85499999999999998,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.57799999999999996}]},{"osd":9,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.64300000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.629},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.028,"15min":1.028},"last":0.68799999999999994}]},{"osd":11,"last update":"Tue Sep 12 16:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69799999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485552,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80400000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48399999999999999,"15min":0.47399999999999998},"min":{"1min":0.29299999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82899999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.48299999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.498},"min":{"1min":0.438,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.56000000000000005,"15min":0.50600000000000001},"min":{"1min":0.41199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61499999999999999}]},{"osd":3,"last update":"Tue Sep 12 16:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.41199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.56000000000000005,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80200000000000005}]},{"osd":5,"last update":"Tue Sep 12 16:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.69399999999999995}]},{"osd":6,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Tue Sep 12 16:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.82799999999999996}]},{"osd":8,"last update":"Tue Sep 12 16:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.63600000000000001,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.629,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.91800000000000004}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.85899999999999999}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.60799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.747,"15min":0.72099999999999997},"min":{"1min":0.64400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.97099999999999997}]},{"osd":11,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.94699999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485038,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.36699999999999999,"15min":0.34499999999999997},"min":{"1min":0.29499999999999998,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.29699999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.51700000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.309}]},{"osd":1,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.34000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":3.552,"15min":3.552},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76800000000000002,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.755,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.379}]},{"osd":4,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53900000000000003,"15min":0.51700000000000002},"min":{"1min":0.45600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84499999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Sep 12 16:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.41499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83299999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.79100000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.47499999999999998},"min":{"1min":0.36299999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69199999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40799999999999997}]},{"osd":7,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.56100000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.503},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.5,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57499999999999996}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66100000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.083,"15min":1.083},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.75,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.53700000000000003}]},{"osd":10,"last update":"Tue Sep 12 16:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":550496,"data_compressed":30118,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.46999999999999997},"min":{"1min":0.38500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.45800000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88800000000000001,"5min":1.458,"15min":1.458},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.36299999999999999},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.41799999999999998,"15min":0.374},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56100000000000005}]},{"osd":2,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.72399999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58199999999999996,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.316,"15min":0.316},"max":{"1min":1.0249999999999999,"5min":1.383,"15min":1.383},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52900000000000003,"15min":0.49399999999999999},"min":{"1min":0.38600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":1.54,"15min":1.54},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.56899999999999995,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92700000000000005,"5min":1.502,"15min":1.502},"last":0.85299999999999998}]},{"osd":6,"last update":"Tue Sep 12 16:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.496,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93999999999999995,"5min":1.639,"15min":1.639},"last":0.755},{"interface":"front","average":{"1min":0.621,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.70999999999999996}]},{"osd":8,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.58099999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.79300000000000004},"min":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.998,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.96599999999999997}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97499999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82199999999999995}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.755,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.93000000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1,"5min":1,"15min":1},"last":0.93000000000000005}]},{"osd":11,"last update":"Tue Sep 12 16:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.88},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.997}]}]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1080753,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70899999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.499,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66400000000000003,"5min":0.749,"15min":0.749},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.75800000000000001}]},{"osd":2,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.53200000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40000000000000002,"15min":0.39400000000000002},"min":{"1min":0.309,"5min":0.245,"15min":0.245},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53200000000000003,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60699999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.498},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.36799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.627,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52500000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.40300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81899999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.39200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72999999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.622}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.879,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63400000000000001}]},{"osd":7,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.57499999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.83999999999999997}]},{"osd":8,"last update":"Tue Sep 12 16:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85499999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83399999999999996,"5min":0.87,"15min":0.87},"last":0.67100000000000004}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.77700000000000002}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.56899999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.871},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":61919,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T16:03:09.404 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-09-12T16:03:09.405 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-12T16:03:09.405 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-09-12T16:03:09.405 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph health --format=json 2023-09-12T16:03:09.970 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:10.240 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:09 smithi130 ceph-mon[98670]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2023-09-12T16:03:10.241 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.241 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.241 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:09 smithi130 ceph-mon[98670]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:10.241 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.241 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.242 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.242 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:09 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:09 smithi203 ceph-mon[103431]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2023-09-12T16:03:10.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:09 smithi203 ceph-mon[103431]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:10.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:09 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:09 smithi159 ceph-mon[102522]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2023-09-12T16:03:10.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:09 smithi159 ceph-mon[102522]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:10.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:10.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:09 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:11.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 -- 172.21.15.130:0/239815683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 msgr2=0x7fc89c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:11.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 --2- 172.21.15.130:0/239815683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 0x7fc89c106a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009990 tx=0x7fc88c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 -- 172.21.15.130:0/239815683 shutdown_connections 2023-09-12T16:03:11.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 --2- 172.21.15.130:0/239815683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc89c100fd0 0x7fc89c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 --2- 172.21.15.130:0/239815683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 0x7fc89c106a80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 --2- 172.21.15.130:0/239815683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc89c100610 0x7fc89c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 -- 172.21.15.130:0/239815683 >> 172.21.15.130:0/239815683 conn(0x7fc89c0fc100 msgr2=0x7fc89c0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:11.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 -- 172.21.15.130:0/239815683 shutdown_connections 2023-09-12T16:03:11.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 -- 172.21.15.130:0/239815683 wait complete. 2023-09-12T16:03:11.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.178+0000 7fc8a157d700 1 Processor -- start 2023-09-12T16:03:11.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc8a157d700 1 -- start start 2023-09-12T16:03:11.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc8a157d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc89c100610 0x7fc89c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:11.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc8a157d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc89c100fd0 0x7fc89c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:11.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc8a157d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 0x7fc89c0733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:11.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc8a157d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1a4b50 con 0x7fc89c1066a0 2023-09-12T16:03:11.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc8a157d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1a4cc0 con 0x7fc89c100fd0 2023-09-12T16:03:11.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc8a157d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1a4e80 con 0x7fc89c100610 2023-09-12T16:03:11.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 0x7fc89c0733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:11.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 0x7fc89c0733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54300/0 (socket says 172.21.15.130:54300) 2023-09-12T16:03:11.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89b7fe700 1 -- 172.21.15.130:0/4282186394 learned_addr learned my addr 172.21.15.130:0/4282186394 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:11.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89a7fc700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc89c100fd0 0x7fc89c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:11.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89affd700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc89c100610 0x7fc89c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:11.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89b7fe700 1 -- 172.21.15.130:0/4282186394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc89c100610 msgr2=0x7fc89c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:11.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89b7fe700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc89c100610 0x7fc89c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89b7fe700 1 -- 172.21.15.130:0/4282186394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc89c100fd0 msgr2=0x7fc89c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:11.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89b7fe700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc89c100fd0 0x7fc89c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.179+0000 7fc89b7fe700 1 -- 172.21.15.130:0/4282186394 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0095f0 con 0x7fc89c1066a0 2023-09-12T16:03:11.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.180+0000 7fc89b7fe700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 0x7fc89c0733c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc894007ee0 tx=0x7fc89400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:11.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.180+0000 7fc883fff700 1 -- 172.21.15.130:0/4282186394 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894017070 con 0x7fc89c1066a0 2023-09-12T16:03:11.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.180+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c1a5160 con 0x7fc89c1066a0 2023-09-12T16:03:11.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.180+0000 7fc883fff700 1 -- 172.21.15.130:0/4282186394 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc89400f040 con 0x7fc89c1066a0 2023-09-12T16:03:11.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.180+0000 7fc883fff700 1 -- 172.21.15.130:0/4282186394 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894005740 con 0x7fc89c1066a0 2023-09-12T16:03:11.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.180+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c1a5660 con 0x7fc89c1066a0 2023-09-12T16:03:11.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.181+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c04e300 con 0x7fc89c1066a0 2023-09-12T16:03:11.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.182+0000 7fc883fff700 1 -- 172.21.15.130:0/4282186394 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fc89400dc60 con 0x7fc89c1066a0 2023-09-12T16:03:11.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.182+0000 7fc883fff700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8840a06d0 0x7fc8840a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:11.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.182+0000 7fc883fff700 1 -- 172.21.15.130:0/4282186394 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc8940cb7e0 con 0x7fc89c1066a0 2023-09-12T16:03:11.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.182+0000 7fc89affd700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8840a06d0 0x7fc8840a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:11.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.183+0000 7fc89affd700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8840a06d0 0x7fc8840a2b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc88c006010 tx=0x7fc88c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:11.188 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.187+0000 7fc883fff700 1 -- 172.21.15.130:0/4282186394 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc894094ea0 con 0x7fc89c1066a0 2023-09-12T16:03:11.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:10 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:03:11.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:10 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:03:11.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:10 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:11.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:10 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:03:11.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:10 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:03:11.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:10 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:11.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.418+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc89c110790 con 0x7fc89c1066a0 2023-09-12T16:03:11.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.418+0000 7fc883fff700 1 -- 172.21.15.130:0/4282186394 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc8940984c0 con 0x7fc89c1066a0 2023-09-12T16:03:11.419 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:11.419 INFO:teuthology.orchestra.run.smithi130.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-12T16:03:11.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8840a06d0 msgr2=0x7fc8840a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:11.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8840a06d0 0x7fc8840a2b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc88c006010 tx=0x7fc88c00b540 comp rx=0 tx=0).stop 2023-09-12T16:03:11.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 msgr2=0x7fc89c0733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:11.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 0x7fc89c0733c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc894007ee0 tx=0x7fc89400d3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 shutdown_connections 2023-09-12T16:03:11.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc8840a06d0 0x7fc8840a2b90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc89c1066a0 0x7fc89c0733c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc89c100610 0x7fc89c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 --2- 172.21.15.130:0/4282186394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc89c100fd0 0x7fc89c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 >> 172.21.15.130:0/4282186394 conn(0x7fc89c0fc100 msgr2=0x7fc89c104340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:11.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 shutdown_connections 2023-09-12T16:03:11.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.420+0000 7fc8a157d700 1 -- 172.21.15.130:0/4282186394 wait complete. 2023-09-12T16:03:11.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:10 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:03:11.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:10 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:03:11.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:10 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:03:11.838 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-09-12T16:03:11.839 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:11.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 -- 172.21.15.130:0/478606161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 msgr2=0x7f1fc4100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/478606161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 0x7f1fc4100b40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40099c0 tx=0x7f1fb4009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 -- 172.21.15.130:0/478606161 shutdown_connections 2023-09-12T16:03:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/478606161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc4101080 0x7f1fc4107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/478606161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc40ff4f0 0x7f1fc40ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/478606161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 0x7f1fc4100b40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 -- 172.21.15.130:0/478606161 >> 172.21.15.130:0/478606161 conn(0x7f1fc40faa60 msgr2=0x7f1fc40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 -- 172.21.15.130:0/478606161 shutdown_connections 2023-09-12T16:03:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 -- 172.21.15.130:0/478606161 wait complete. 2023-09-12T16:03:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.964+0000 7f1fc90ae700 1 Processor -- start 2023-09-12T16:03:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc90ae700 1 -- start start 2023-09-12T16:03:11.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc90ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc40ff4f0 0x7f1fc4069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:11.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc90ae700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 0x7f1fc419b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:11.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc90ae700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc4101080 0x7f1fc419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:11.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc90ae700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc419de40 con 0x7f1fc41006e0 2023-09-12T16:03:11.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc90ae700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc419dfb0 con 0x7f1fc4101080 2023-09-12T16:03:11.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc90ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc419e170 con 0x7f1fc40ff4f0 2023-09-12T16:03:11.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc259c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 0x7f1fc419b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:11.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc259c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 0x7f1fc419b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54310/0 (socket says 172.21.15.130:54310) 2023-09-12T16:03:11.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc259c700 1 -- 172.21.15.130:0/4194681995 learned_addr learned my addr 172.21.15.130:0/4194681995 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:11.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc359e700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc4101080 0x7f1fc419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:11.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc2d9d700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc40ff4f0 0x7f1fc4069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:11.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc259c700 1 -- 172.21.15.130:0/4194681995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc40ff4f0 msgr2=0x7f1fc4069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:11.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc259c700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc40ff4f0 0x7f1fc4069510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc259c700 1 -- 172.21.15.130:0/4194681995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc4101080 msgr2=0x7f1fc419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:11.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc259c700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc4101080 0x7f1fc419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:11.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.965+0000 7f1fc259c700 1 -- 172.21.15.130:0/4194681995 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb40095f0 con 0x7f1fc41006e0 2023-09-12T16:03:11.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.966+0000 7f1fc259c700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 0x7f1fc419b020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009990 tx=0x7f1fb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:11.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.966+0000 7f1fabfff700 1 -- 172.21.15.130:0/4194681995 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb401d070 con 0x7f1fc41006e0 2023-09-12T16:03:11.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.966+0000 7f1fabfff700 1 -- 172.21.15.130:0/4194681995 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb4004090 con 0x7f1fc41006e0 2023-09-12T16:03:11.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.966+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc419e3f0 con 0x7f1fc41006e0 2023-09-12T16:03:11.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.966+0000 7f1fabfff700 1 -- 172.21.15.130:0/4194681995 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4004a00 con 0x7f1fc41006e0 2023-09-12T16:03:11.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.966+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc419e8a0 con 0x7f1fc41006e0 2023-09-12T16:03:11.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.967+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc410ada0 con 0x7f1fc41006e0 2023-09-12T16:03:11.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.968+0000 7f1fabfff700 1 -- 172.21.15.130:0/4194681995 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f1fb4003bd0 con 0x7f1fc41006e0 2023-09-12T16:03:11.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.969+0000 7f1fabfff700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fac0a0710 0x7f1fac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:11.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.969+0000 7f1fc2d9d700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fac0a0710 0x7f1fac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:11.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.969+0000 7f1fabfff700 1 -- 172.21.15.130:0/4194681995 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1fb40cd3d0 con 0x7f1fc41006e0 2023-09-12T16:03:11.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.969+0000 7f1fc2d9d700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fac0a0710 0x7f1fac0a2bd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8006fd0 tx=0x7f1fb8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:11.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:11.972+0000 7f1fabfff700 1 -- 172.21.15.130:0/4194681995 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1fb4096b20 con 0x7f1fc41006e0 2023-09-12T16:03:12.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.168+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fc419eac0 con 0x7f1fc41006e0 2023-09-12T16:03:12.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.170+0000 7f1fabfff700 1 -- 172.21.15.130:0/4194681995 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24473 (secure 0 0 0) 0x7f1fb409a140 con 0x7f1fc41006e0 2023-09-12T16:03:12.169 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:12.170 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":88,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:51.826597+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:12.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fac0a0710 msgr2=0x7f1fac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fac0a0710 0x7f1fac0a2bd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8006fd0 tx=0x7f1fb8008040 comp rx=0 tx=0).stop 2023-09-12T16:03:12.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 msgr2=0x7f1fc419b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 0x7f1fc419b020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009990 tx=0x7f1fb4004730 comp rx=0 tx=0).stop 2023-09-12T16:03:12.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 shutdown_connections 2023-09-12T16:03:12.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fac0a0710 0x7f1fac0a2bd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc4101080 0x7f1fc419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc40ff4f0 0x7f1fc4069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 --2- 172.21.15.130:0/4194681995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc41006e0 0x7f1fc419b020 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 >> 172.21.15.130:0/4194681995 conn(0x7f1fc40faa60 msgr2=0x7f1fc40fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:12.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 shutdown_connections 2023-09-12T16:03:12.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.172+0000 7f1fc90ae700 1 -- 172.21.15.130:0/4194681995 wait complete. 2023-09-12T16:03:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:03:12.185 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:03:12.311 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.309+0000 7f3310fcf700 1 -- 172.21.15.130:0/2044756837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c102210 msgr2=0x7f330c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.309+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2044756837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c102210 0x7f330c108ac0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f330400b0a0 tx=0x7f330400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.309+0000 7f3310fcf700 1 -- 172.21.15.130:0/2044756837 shutdown_connections 2023-09-12T16:03:12.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.309+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2044756837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c102210 0x7f330c108ac0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.309+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2044756837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f330c0fe8a0 0x7f330c0fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.309+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2044756837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f330c101870 0x7f330c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.309+0000 7f3310fcf700 1 -- 172.21.15.130:0/2044756837 >> 172.21.15.130:0/2044756837 conn(0x7f330c0fa1d0 msgr2=0x7f330c0fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:12.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.309+0000 7f3310fcf700 1 -- 172.21.15.130:0/2044756837 shutdown_connections 2023-09-12T16:03:12.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.309+0000 7f3310fcf700 1 -- 172.21.15.130:0/2044756837 wait complete. 2023-09-12T16:03:12.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3310fcf700 1 Processor -- start 2023-09-12T16:03:12.314 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3310fcf700 1 -- start start 2023-09-12T16:03:12.314 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3310fcf700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f330c0fe8a0 0x7f330c1011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.314 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3310fcf700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c101870 0x7f330c0ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3310fcf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f330c102210 0x7f330c0ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3310fcf700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c100500 con 0x7f330c101870 2023-09-12T16:03:12.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3310fcf700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c100670 con 0x7f330c0fe8a0 2023-09-12T16:03:12.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3310fcf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c1a0400 con 0x7f330c102210 2023-09-12T16:03:12.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3309d9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c101870 0x7f330c0ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.316 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3309d9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c101870 0x7f330c0ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54324/0 (socket says 172.21.15.130:54324) 2023-09-12T16:03:12.316 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f3309d9b700 1 -- 172.21.15.130:0/2215197636 learned_addr learned my addr 172.21.15.130:0/2215197636 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:12.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f330ad9d700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f330c102210 0x7f330c0ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.310+0000 7f330a59c700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f330c0fe8a0 0x7f330c1011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f3309d9b700 1 -- 172.21.15.130:0/2215197636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f330c102210 msgr2=0x7f330c0ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f3309d9b700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f330c102210 0x7f330c0ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f3309d9b700 1 -- 172.21.15.130:0/2215197636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f330c0fe8a0 msgr2=0x7f330c1011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f3309d9b700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f330c0fe8a0 0x7f330c1011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f3309d9b700 1 -- 172.21.15.130:0/2215197636 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3304009d00 con 0x7f330c101870 2023-09-12T16:03:12.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f3309d9b700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c101870 0x7f330c0ff820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00d950 tx=0x7f32fc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:12.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f33037fe700 1 -- 172.21.15.130:0/2215197636 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc009b40 con 0x7f330c101870 2023-09-12T16:03:12.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c1a06e0 con 0x7f330c101870 2023-09-12T16:03:12.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f33037fe700 1 -- 172.21.15.130:0/2215197636 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32fc009ca0 con 0x7f330c101870 2023-09-12T16:03:12.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f33037fe700 1 -- 172.21.15.130:0/2215197636 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00f620 con 0x7f330c101870 2023-09-12T16:03:12.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.311+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c1a0c40 con 0x7f330c101870 2023-09-12T16:03:12.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.312+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c1075b0 con 0x7f330c101870 2023-09-12T16:03:12.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.313+0000 7f33037fe700 1 -- 172.21.15.130:0/2215197636 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f32fc010460 con 0x7f330c101870 2023-09-12T16:03:12.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.314+0000 7f33037fe700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32f80a06d0 0x7f32f80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.314+0000 7f33037fe700 1 -- 172.21.15.130:0/2215197636 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f32fc0cc500 con 0x7f330c101870 2023-09-12T16:03:12.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.314+0000 7f330a59c700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32f80a06d0 0x7f32f80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.314+0000 7f330a59c700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32f80a06d0 0x7f32f80a2b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f330c067560 tx=0x7f32f400b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:12.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.317+0000 7f33037fe700 1 -- 172.21.15.130:0/2215197636 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f32fc095b60 con 0x7f330c101870 2023-09-12T16:03:12.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:12 smithi130 ceph-mon[98670]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:12.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:12 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4282186394' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-12T16:03:12.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:12 smithi159 ceph-mon[102522]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:12.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:12 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4282186394' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-12T16:03:12.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:12 smithi203 ceph-mon[103431]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:12.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:12 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4282186394' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-12T16:03:12.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.508+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f330c10c000 con 0x7f330c101870 2023-09-12T16:03:12.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.508+0000 7f33037fe700 1 -- 172.21.15.130:0/2215197636 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32fc020020 con 0x7f330c101870 2023-09-12T16:03:12.508 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:03:12.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32f80a06d0 msgr2=0x7f32f80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32f80a06d0 0x7f32f80a2b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f330c067560 tx=0x7f32f400b5b0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c101870 msgr2=0x7f330c0ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c101870 0x7f330c0ff820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00d950 tx=0x7f32fc00dd10 comp rx=0 tx=0).stop 2023-09-12T16:03:12.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 shutdown_connections 2023-09-12T16:03:12.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32f80a06d0 0x7f32f80a2b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f330c102210 0x7f330c0ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f330c0fe8a0 0x7f330c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 --2- 172.21.15.130:0/2215197636 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f330c101870 0x7f330c0ff820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 >> 172.21.15.130:0/2215197636 conn(0x7f330c0fa1d0 msgr2=0x7f330c0fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:12.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.510+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 shutdown_connections 2023-09-12T16:03:12.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.511+0000 7f3310fcf700 1 -- 172.21.15.130:0/2215197636 wait complete. 2023-09-12T16:03:12.522 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:03:12.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.647+0000 7f248bac3700 1 -- 172.21.15.130:0/3361194065 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2484102210 msgr2=0x7f2484108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.647+0000 7f248bac3700 1 --2- 172.21.15.130:0/3361194065 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2484102210 0x7f2484108ac0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f247c00b0a0 tx=0x7f247c00b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.647+0000 7f248bac3700 1 -- 172.21.15.130:0/3361194065 shutdown_connections 2023-09-12T16:03:12.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.647+0000 7f248bac3700 1 --2- 172.21.15.130:0/3361194065 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2484102210 0x7f2484108ac0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.647+0000 7f248bac3700 1 --2- 172.21.15.130:0/3361194065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24840fe8a0 0x7f24840fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.647+0000 7f248bac3700 1 --2- 172.21.15.130:0/3361194065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2484101870 0x7f2484101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.647+0000 7f248bac3700 1 -- 172.21.15.130:0/3361194065 >> 172.21.15.130:0/3361194065 conn(0x7f24840fa1d0 msgr2=0x7f24840fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.647+0000 7f248bac3700 1 -- 172.21.15.130:0/3361194065 shutdown_connections 2023-09-12T16:03:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.647+0000 7f248bac3700 1 -- 172.21.15.130:0/3361194065 wait complete. 2023-09-12T16:03:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248bac3700 1 Processor -- start 2023-09-12T16:03:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248bac3700 1 -- start start 2023-09-12T16:03:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248bac3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24840fe8a0 0x7f24841011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248bac3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2484101870 0x7f24840ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248bac3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2484102210 0x7f24840ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248bac3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484100500 con 0x7f24840fe8a0 2023-09-12T16:03:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248bac3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484100670 con 0x7f2484102210 2023-09-12T16:03:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248bac3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24841a0390 con 0x7f2484101870 2023-09-12T16:03:12.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248985f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24840fe8a0 0x7f24841011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248985f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24840fe8a0 0x7f24841011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54336/0 (socket says 172.21.15.130:54336) 2023-09-12T16:03:12.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248985f700 1 -- 172.21.15.130:0/3005437466 learned_addr learned my addr 172.21.15.130:0/3005437466 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:12.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248905e700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2484101870 0x7f24840ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248a060700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2484102210 0x7f24840ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248985f700 1 -- 172.21.15.130:0/3005437466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2484101870 msgr2=0x7f24840ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248985f700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2484101870 0x7f24840ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248985f700 1 -- 172.21.15.130:0/3005437466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2484102210 msgr2=0x7f24840ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248985f700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2484102210 0x7f24840ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248985f700 1 -- 172.21.15.130:0/3005437466 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c009d00 con 0x7f24840fe8a0 2023-09-12T16:03:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248905e700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2484101870 0x7f24840ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:12.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248985f700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24840fe8a0 0x7f24841011e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f248000d980 tx=0x7f248000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:12.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f247affd700 1 -- 172.21.15.130:0/3005437466 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248000de60 con 0x7f24840fe8a0 2023-09-12T16:03:12.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f247affd700 1 -- 172.21.15.130:0/3005437466 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24800044e0 con 0x7f24840fe8a0 2023-09-12T16:03:12.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.648+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24841a0670 con 0x7f24840fe8a0 2023-09-12T16:03:12.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.649+0000 7f247affd700 1 -- 172.21.15.130:0/3005437466 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480003be0 con 0x7f24840fe8a0 2023-09-12T16:03:12.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.649+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24841a0b80 con 0x7f24840fe8a0 2023-09-12T16:03:12.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.649+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484107510 con 0x7f24840fe8a0 2023-09-12T16:03:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.650+0000 7f247affd700 1 -- 172.21.15.130:0/3005437466 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f2480009bb0 con 0x7f24840fe8a0 2023-09-12T16:03:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.651+0000 7f247affd700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f24700a06d0 0x7f24700a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.651+0000 7f248905e700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f24700a06d0 0x7f24700a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.651+0000 7f247affd700 1 -- 172.21.15.130:0/3005437466 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f24800cc130 con 0x7f24840fe8a0 2023-09-12T16:03:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.651+0000 7f248905e700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f24700a06d0 0x7f24700a2b90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2474005fd0 tx=0x7f2474005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:12.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.655+0000 7f247affd700 1 -- 172.21.15.130:0/3005437466 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f24800957f0 con 0x7f24840fe8a0 2023-09-12T16:03:12.844 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.845+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f248410c0a0 con 0x7f24840fe8a0 2023-09-12T16:03:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.845+0000 7f247affd700 1 -- 172.21.15.130:0/3005437466 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f248410c0a0 con 0x7f24840fe8a0 2023-09-12T16:03:12.845 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:12.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.847+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f24700a06d0 msgr2=0x7f24700a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.847+0000 7f248bac3700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f24700a06d0 0x7f24700a2b90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2474005fd0 tx=0x7f2474005e30 comp rx=0 tx=0).stop 2023-09-12T16:03:12.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.847+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24840fe8a0 msgr2=0x7f24841011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.847+0000 7f248bac3700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24840fe8a0 0x7f24841011e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f248000d980 tx=0x7f248000dd40 comp rx=0 tx=0).stop 2023-09-12T16:03:12.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.848+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 shutdown_connections 2023-09-12T16:03:12.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.848+0000 7f248bac3700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f24700a06d0 0x7f24700a2b90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.848+0000 7f248bac3700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2484102210 0x7f24840ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.848+0000 7f248bac3700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24840fe8a0 0x7f24841011e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.848+0000 7f248bac3700 1 --2- 172.21.15.130:0/3005437466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2484101870 0x7f24840ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.848+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 >> 172.21.15.130:0/3005437466 conn(0x7f24840fa1d0 msgr2=0x7f24840fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:12.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.848+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 shutdown_connections 2023-09-12T16:03:12.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.848+0000 7f248bac3700 1 -- 172.21.15.130:0/3005437466 wait complete. 2023-09-12T16:03:12.862 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:03:12.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.986+0000 7f5108a3c700 1 -- 172.21.15.130:0/1977644227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040fe6b0 msgr2=0x7f51040feb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.986+0000 7f5108a3c700 1 --2- 172.21.15.130:0/1977644227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040fe6b0 0x7f51040feb10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009990 tx=0x7f50ec009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.986+0000 7f5108a3c700 1 -- 172.21.15.130:0/1977644227 shutdown_connections 2023-09-12T16:03:12.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.986+0000 7f5108a3c700 1 --2- 172.21.15.130:0/1977644227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040ff050 0x7f5104108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.986+0000 7f5108a3c700 1 --2- 172.21.15.130:0/1977644227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5104101c40 0x7f5104102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.986+0000 7f5108a3c700 1 --2- 172.21.15.130:0/1977644227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040fe6b0 0x7f51040feb10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.986+0000 7f5108a3c700 1 -- 172.21.15.130:0/1977644227 >> 172.21.15.130:0/1977644227 conn(0x7f51040fa1f0 msgr2=0x7f51040fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:12.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.986+0000 7f5108a3c700 1 -- 172.21.15.130:0/1977644227 shutdown_connections 2023-09-12T16:03:12.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.986+0000 7f5108a3c700 1 -- 172.21.15.130:0/1977644227 wait complete. 2023-09-12T16:03:12.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5108a3c700 1 Processor -- start 2023-09-12T16:03:12.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5108a3c700 1 -- start start 2023-09-12T16:03:12.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5108a3c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040fe6b0 0x7f5104196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5108a3c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040ff050 0x7f5104196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5108a3c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5104101c40 0x7f510419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5108a3c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510419de80 con 0x7f51040ff050 2023-09-12T16:03:12.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5108a3c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510419dfc0 con 0x7f51040fe6b0 2023-09-12T16:03:12.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5108a3c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510419e150 con 0x7f5104101c40 2023-09-12T16:03:12.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5101d9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040ff050 0x7f5104196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5101d9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040ff050 0x7f5104196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54352/0 (socket says 172.21.15.130:54352) 2023-09-12T16:03:12.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5101d9b700 1 -- 172.21.15.130:0/86425856 learned_addr learned my addr 172.21.15.130:0/86425856 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:12.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5102d9d700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5104101c40 0x7f510419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f510259c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040fe6b0 0x7f5104196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5101d9b700 1 -- 172.21.15.130:0/86425856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5104101c40 msgr2=0x7f510419d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5101d9b700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5104101c40 0x7f510419d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5101d9b700 1 -- 172.21.15.130:0/86425856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040fe6b0 msgr2=0x7f5104196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:12.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5101d9b700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040fe6b0 0x7f5104196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:12.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5101d9b700 1 -- 172.21.15.130:0/86425856 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec0095f0 con 0x7f51040ff050 2023-09-12T16:03:12.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5102d9d700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5104101c40 0x7f510419d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:12.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f510259c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040fe6b0 0x7f5104196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:12.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.987+0000 7f5101d9b700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040ff050 0x7f5104196840 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0098e0 tx=0x7f50ec00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:12.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.988+0000 7f50fb7fe700 1 -- 172.21.15.130:0/86425856 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec01d070 con 0x7f51040ff050 2023-09-12T16:03:12.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.988+0000 7f50fb7fe700 1 -- 172.21.15.130:0/86425856 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50ec00fb70 con 0x7f51040ff050 2023-09-12T16:03:12.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.988+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510419e3d0 con 0x7f51040ff050 2023-09-12T16:03:12.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.988+0000 7f50fb7fe700 1 -- 172.21.15.130:0/86425856 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec017650 con 0x7f51040ff050 2023-09-12T16:03:12.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.988+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510419e7f0 con 0x7f51040ff050 2023-09-12T16:03:12.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.989+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5104107dc0 con 0x7f51040ff050 2023-09-12T16:03:12.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.990+0000 7f50fb7fe700 1 -- 172.21.15.130:0/86425856 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f50ec0050d0 con 0x7f51040ff050 2023-09-12T16:03:12.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.990+0000 7f50fb7fe700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50f00a06d0 0x7f50f00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:12.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.990+0000 7f50fb7fe700 1 -- 172.21.15.130:0/86425856 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50ec0cd6a0 con 0x7f51040ff050 2023-09-12T16:03:12.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.990+0000 7f510259c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50f00a06d0 0x7f50f00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:12.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.991+0000 7f510259c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50f00a06d0 0x7f50f00a2b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5104068630 tx=0x7f50f400b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:12.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:12.993+0000 7f50fb7fe700 1 -- 172.21.15.130:0/86425856 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f50ec096df0 con 0x7f51040ff050 2023-09-12T16:03:13.189 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.189+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f510410c040 con 0x7f51040ff050 2023-09-12T16:03:13.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.189+0000 7f50fb7fe700 1 -- 172.21.15.130:0/86425856 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f50ec09a410 con 0x7f51040ff050 2023-09-12T16:03:13.191 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:03:13.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.191+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50f00a06d0 msgr2=0x7f50f00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.191+0000 7f5108a3c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50f00a06d0 0x7f50f00a2b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5104068630 tx=0x7f50f400b5f0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040ff050 msgr2=0x7f5104196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040ff050 0x7f5104196840 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0098e0 tx=0x7f50ec00f740 comp rx=0 tx=0).stop 2023-09-12T16:03:13.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 shutdown_connections 2023-09-12T16:03:13.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50f00a06d0 0x7f50f00a2b90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5104101c40 0x7f510419d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040fe6b0 0x7f5104196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 --2- 172.21.15.130:0/86425856 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51040ff050 0x7f5104196840 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 >> 172.21.15.130:0/86425856 conn(0x7f51040fa1f0 msgr2=0x7f5104104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:13.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 shutdown_connections 2023-09-12T16:03:13.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.192+0000 7f5108a3c700 1 -- 172.21.15.130:0/86425856 wait complete. 2023-09-12T16:03:13.203 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:13 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4194681995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:13.204 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:13 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2215197636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:13.204 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:13 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3005437466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:13.205 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:03:13.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.330+0000 7f623c653700 1 -- 172.21.15.130:0/486774207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62340ff4f0 msgr2=0x7f62340ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.330+0000 7f623c653700 1 --2- 172.21.15.130:0/486774207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62340ff4f0 0x7f62340ff910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6224009920 tx=0x7f6224009c30 comp rx=0 tx=0).stop 2023-09-12T16:03:13.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.330+0000 7f623c653700 1 -- 172.21.15.130:0/486774207 shutdown_connections 2023-09-12T16:03:13.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.330+0000 7f623c653700 1 --2- 172.21.15.130:0/486774207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6234101080 0x7f6234107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.330+0000 7f623c653700 1 --2- 172.21.15.130:0/486774207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62340ff4f0 0x7f62340ff910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.330+0000 7f623c653700 1 --2- 172.21.15.130:0/486774207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62341006e0 0x7f6234100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.330+0000 7f623c653700 1 -- 172.21.15.130:0/486774207 >> 172.21.15.130:0/486774207 conn(0x7f62340faa60 msgr2=0x7f62340fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:13.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.330+0000 7f623c653700 1 -- 172.21.15.130:0/486774207 shutdown_connections 2023-09-12T16:03:13.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.330+0000 7f623c653700 1 -- 172.21.15.130:0/486774207 wait complete. 2023-09-12T16:03:13.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623c653700 1 Processor -- start 2023-09-12T16:03:13.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623c653700 1 -- start start 2023-09-12T16:03:13.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623c653700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62340ff4f0 0x7f62340682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:13.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623c653700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62341006e0 0x7f623419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:13.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623c653700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6234101080 0x7f623419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:13.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623c653700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623419def0 con 0x7f62341006e0 2023-09-12T16:03:13.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623c653700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623419e060 con 0x7f6234101080 2023-09-12T16:03:13.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623c653700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623419e220 con 0x7f62340ff4f0 2023-09-12T16:03:13.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f6239bee700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62341006e0 0x7f623419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:13.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f6239bee700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62341006e0 0x7f623419b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54366/0 (socket says 172.21.15.130:54366) 2023-09-12T16:03:13.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f6239bee700 1 -- 172.21.15.130:0/2158771159 learned_addr learned my addr 172.21.15.130:0/2158771159 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:13.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623a3ef700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62340ff4f0 0x7f62340682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:13.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623abf0700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6234101080 0x7f623419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:13.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f6239bee700 1 -- 172.21.15.130:0/2158771159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62340ff4f0 msgr2=0x7f62340682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f6239bee700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62340ff4f0 0x7f62340682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f6239bee700 1 -- 172.21.15.130:0/2158771159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6234101080 msgr2=0x7f623419d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f6239bee700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6234101080 0x7f623419d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f6239bee700 1 -- 172.21.15.130:0/2158771159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6224009520 con 0x7f62341006e0 2023-09-12T16:03:13.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.331+0000 7f623abf0700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6234101080 0x7f623419d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:13.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.332+0000 7f6239bee700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62341006e0 0x7f623419b0d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9a0 tx=0x7f623000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:13.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.332+0000 7f622b7fe700 1 -- 172.21.15.130:0/2158771159 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6230009790 con 0x7f62341006e0 2023-09-12T16:03:13.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.332+0000 7f622b7fe700 1 -- 172.21.15.130:0/2158771159 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f623000cbd0 con 0x7f62341006e0 2023-09-12T16:03:13.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.332+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623419e500 con 0x7f62341006e0 2023-09-12T16:03:13.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.332+0000 7f622b7fe700 1 -- 172.21.15.130:0/2158771159 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6230018680 con 0x7f62341006e0 2023-09-12T16:03:13.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.332+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623419ea10 con 0x7f62341006e0 2023-09-12T16:03:13.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.333+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62341064d0 con 0x7f62341006e0 2023-09-12T16:03:13.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.334+0000 7f622b7fe700 1 -- 172.21.15.130:0/2158771159 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f623000cd40 con 0x7f62341006e0 2023-09-12T16:03:13.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.334+0000 7f622b7fe700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62200a06d0 0x7f62200a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:13.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.335+0000 7f623a3ef700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62200a06d0 0x7f62200a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:13.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.335+0000 7f622b7fe700 1 -- 172.21.15.130:0/2158771159 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6230014070 con 0x7f62341006e0 2023-09-12T16:03:13.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.335+0000 7f623a3ef700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62200a06d0 0x7f62200a2b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f622400fa40 tx=0x7f6224019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:13.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.338+0000 7f622b7fe700 1 -- 172.21.15.130:0/2158771159 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6230095f30 con 0x7f62341006e0 2023-09-12T16:03:13.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:13 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4194681995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:13.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:13 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2215197636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:13.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:13 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3005437466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:13.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:13 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4194681995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:13.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:13 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2215197636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:13.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:13 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3005437466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:13.530 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.530+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f623410af10 con 0x7f62341006e0 2023-09-12T16:03:13.531 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.530+0000 7f622b7fe700 1 -- 172.21.15.130:0/2158771159 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6230099550 con 0x7f62341006e0 2023-09-12T16:03:13.531 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:13.533 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62200a06d0 msgr2=0x7f62200a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62200a06d0 0x7f62200a2b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f622400fa40 tx=0x7f6224019040 comp rx=0 tx=0).stop 2023-09-12T16:03:13.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62341006e0 msgr2=0x7f623419b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62341006e0 0x7f623419b0d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9a0 tx=0x7f623000ed60 comp rx=0 tx=0).stop 2023-09-12T16:03:13.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 shutdown_connections 2023-09-12T16:03:13.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f62200a06d0 0x7f62200a2b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6234101080 0x7f623419d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62340ff4f0 0x7f62340682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 --2- 172.21.15.130:0/2158771159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62341006e0 0x7f623419b0d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 >> 172.21.15.130:0/2158771159 conn(0x7f62340faa60 msgr2=0x7f62340fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:13.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 shutdown_connections 2023-09-12T16:03:13.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.533+0000 7f623c653700 1 -- 172.21.15.130:0/2158771159 wait complete. 2023-09-12T16:03:13.546 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:03:13.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 -- 172.21.15.130:0/376412221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 msgr2=0x7f1e04100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 --2- 172.21.15.130:0/376412221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 0x7f1e04100b30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1e000099c0 tx=0x7f1e00009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 -- 172.21.15.130:0/376412221 shutdown_connections 2023-09-12T16:03:13.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 --2- 172.21.15.130:0/376412221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e04101070 0x7f1e04107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 --2- 172.21.15.130:0/376412221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e040ff4e0 0x7f1e040ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 --2- 172.21.15.130:0/376412221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 0x7f1e04100b30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 -- 172.21.15.130:0/376412221 >> 172.21.15.130:0/376412221 conn(0x7f1e040faa30 msgr2=0x7f1e040fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 -- 172.21.15.130:0/376412221 shutdown_connections 2023-09-12T16:03:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 -- 172.21.15.130:0/376412221 wait complete. 2023-09-12T16:03:13.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 Processor -- start 2023-09-12T16:03:13.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.667+0000 7f1e0badb700 1 -- start start 2023-09-12T16:03:13.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0badb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e040ff4e0 0x7f1e040682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0badb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 0x7f1e0419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0badb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e04101070 0x7f1e0419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0badb700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0419dda0 con 0x7f1e041006d0 2023-09-12T16:03:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0badb700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0419df10 con 0x7f1e04101070 2023-09-12T16:03:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0badb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0419e0d0 con 0x7f1e040ff4e0 2023-09-12T16:03:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09076700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 0x7f1e0419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09076700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 0x7f1e0419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54380/0 (socket says 172.21.15.130:54380) 2023-09-12T16:03:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09877700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e040ff4e0 0x7f1e040682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09076700 1 -- 172.21.15.130:0/4184111071 learned_addr learned my addr 172.21.15.130:0/4184111071 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0a078700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e04101070 0x7f1e0419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09076700 1 -- 172.21.15.130:0/4184111071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e040ff4e0 msgr2=0x7f1e040682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09076700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e040ff4e0 0x7f1e040682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09076700 1 -- 172.21.15.130:0/4184111071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e04101070 msgr2=0x7f1e0419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09076700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e04101070 0x7f1e0419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09076700 1 -- 172.21.15.130:0/4184111071 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e000095f0 con 0x7f1e041006d0 2023-09-12T16:03:13.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0a078700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e04101070 0x7f1e0419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:13.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e09076700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 0x7f1e0419af80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009990 tx=0x7f1e00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:13.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1dfaffd700 1 -- 172.21.15.130:0/4184111071 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0001d070 con 0x7f1e041006d0 2023-09-12T16:03:13.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1dfaffd700 1 -- 172.21.15.130:0/4184111071 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e00004090 con 0x7f1e041006d0 2023-09-12T16:03:13.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0419e350 con 0x7f1e041006d0 2023-09-12T16:03:13.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1dfaffd700 1 -- 172.21.15.130:0/4184111071 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00004a00 con 0x7f1e041006d0 2023-09-12T16:03:13.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.668+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0419e800 con 0x7f1e041006d0 2023-09-12T16:03:13.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.669+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e04106410 con 0x7f1e041006d0 2023-09-12T16:03:13.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.670+0000 7f1dfaffd700 1 -- 172.21.15.130:0/4184111071 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f1e00003bd0 con 0x7f1e041006d0 2023-09-12T16:03:13.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.671+0000 7f1dfaffd700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1df00a0710 0x7f1df00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:13.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.671+0000 7f1dfaffd700 1 -- 172.21.15.130:0/4184111071 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1e000ccf30 con 0x7f1e041006d0 2023-09-12T16:03:13.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.671+0000 7f1e09877700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1df00a0710 0x7f1df00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:13.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.671+0000 7f1e09877700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1df00a0710 0x7f1df00a2bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1e04100530 tx=0x7f1df400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:13.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.675+0000 7f1dfaffd700 1 -- 172.21.15.130:0/4184111071 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1e00096680 con 0x7f1e041006d0 2023-09-12T16:03:13.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.914+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e0410ad00 con 0x7f1e041006d0 2023-09-12T16:03:13.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.915+0000 7f1dfaffd700 1 -- 172.21.15.130:0/4184111071 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11342 (secure 0 0 0) 0x7f1e00099ca0 con 0x7f1e041006d0 2023-09-12T16:03:13.916 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:13.917 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:25.646573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:03:13.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1df00a0710 msgr2=0x7f1df00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1df00a0710 0x7f1df00a2bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1e04100530 tx=0x7f1df400b410 comp rx=0 tx=0).stop 2023-09-12T16:03:13.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 msgr2=0x7f1e0419af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:13.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 0x7f1e0419af80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009990 tx=0x7f1e00004730 comp rx=0 tx=0).stop 2023-09-12T16:03:13.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 shutdown_connections 2023-09-12T16:03:13.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1df00a0710 0x7f1df00a2bd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e04101070 0x7f1e0419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e040ff4e0 0x7f1e040682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 --2- 172.21.15.130:0/4184111071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e041006d0 0x7f1e0419af80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:13.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 >> 172.21.15.130:0/4184111071 conn(0x7f1e040faa30 msgr2=0x7f1e040fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:13.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 shutdown_connections 2023-09-12T16:03:13.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:13.917+0000 7f1e0badb700 1 -- 172.21.15.130:0/4184111071 wait complete. 2023-09-12T16:03:13.921 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 24 2023-09-12T16:03:13.930 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:14.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 -- 172.21.15.130:0/1984590926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8730100720 msgr2=0x7f8730100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 --2- 172.21.15.130:0/1984590926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8730100720 0x7f8730100ba0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f871c009990 tx=0x7f871c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 -- 172.21.15.130:0/1984590926 shutdown_connections 2023-09-12T16:03:14.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 --2- 172.21.15.130:0/1984590926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87301010e0 0x7f8730107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 --2- 172.21.15.130:0/1984590926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87300ffa10 0x7f87300ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 --2- 172.21.15.130:0/1984590926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8730100720 0x7f8730100ba0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 -- 172.21.15.130:0/1984590926 >> 172.21.15.130:0/1984590926 conn(0x7f87300fa1c0 msgr2=0x7f87300fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:14.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 -- 172.21.15.130:0/1984590926 shutdown_connections 2023-09-12T16:03:14.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 -- 172.21.15.130:0/1984590926 wait complete. 2023-09-12T16:03:14.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 Processor -- start 2023-09-12T16:03:14.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 -- start start 2023-09-12T16:03:14.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.051+0000 7f8736c73700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87300ffa10 0x7f8730195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8736c73700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8730100720 0x7f8730195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8736c73700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87301010e0 0x7f873019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8736c73700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873019cd80 con 0x7f87300ffa10 2023-09-12T16:03:14.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8736c73700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873019cec0 con 0x7f8730100720 2023-09-12T16:03:14.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8736c73700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873019d050 con 0x7f87301010e0 2023-09-12T16:03:14.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8734a0f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87300ffa10 0x7f8730195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8734a0f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87300ffa10 0x7f8730195200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54392/0 (socket says 172.21.15.130:54392) 2023-09-12T16:03:14.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8734a0f700 1 -- 172.21.15.130:0/1667160481 learned_addr learned my addr 172.21.15.130:0/1667160481 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:14.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8735210700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87301010e0 0x7f873019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f872ffff700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8730100720 0x7f8730195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8734a0f700 1 -- 172.21.15.130:0/1667160481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87301010e0 msgr2=0x7f873019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8734a0f700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87301010e0 0x7f873019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8734a0f700 1 -- 172.21.15.130:0/1667160481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8730100720 msgr2=0x7f8730195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8734a0f700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8730100720 0x7f8730195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8734a0f700 1 -- 172.21.15.130:0/1667160481 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c0095f0 con 0x7f87300ffa10 2023-09-12T16:03:14.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8734a0f700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87300ffa10 0x7f8730195200 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f872400b500 tx=0x7f872400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:14.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f872dffb700 1 -- 172.21.15.130:0/1667160481 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724011830 con 0x7f87300ffa10 2023-09-12T16:03:14.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f872dffb700 1 -- 172.21.15.130:0/1667160481 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8724011e50 con 0x7f87300ffa10 2023-09-12T16:03:14.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f872dffb700 1 -- 172.21.15.130:0/1667160481 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872400f2e0 con 0x7f87300ffa10 2023-09-12T16:03:14.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.052+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873019d330 con 0x7f87300ffa10 2023-09-12T16:03:14.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.053+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873019d8c0 con 0x7f87300ffa10 2023-09-12T16:03:14.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.054+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8730106410 con 0x7f87300ffa10 2023-09-12T16:03:14.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.054+0000 7f872dffb700 1 -- 172.21.15.130:0/1667160481 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f8724010580 con 0x7f87300ffa10 2023-09-12T16:03:14.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.055+0000 7f872dffb700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87200a0790 0x7f87200a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.055+0000 7f872ffff700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87200a0790 0x7f87200a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.055+0000 7f872dffb700 1 -- 172.21.15.130:0/1667160481 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f87240cb950 con 0x7f87300ffa10 2023-09-12T16:03:14.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.056+0000 7f872ffff700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87200a0790 0x7f87200a2c50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f871c0098e0 tx=0x7f871c005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:14.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.060+0000 7f872dffb700 1 -- 172.21.15.130:0/1667160481 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f87240950c0 con 0x7f87300ffa10 2023-09-12T16:03:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:14 smithi130 ceph-mon[98670]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:14 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/86425856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:14 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2158771159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:14 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4184111071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:14.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.252+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f873010afc0 con 0x7f87300ffa10 2023-09-12T16:03:14.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.253+0000 7f872dffb700 1 -- 172.21.15.130:0/1667160481 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24473 (secure 0 0 0) 0x7f87240986e0 con 0x7f87300ffa10 2023-09-12T16:03:14.254 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:14.255 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":88,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:51.826597+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:14.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.255+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87200a0790 msgr2=0x7f87200a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87200a0790 0x7f87200a2c50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f871c0098e0 tx=0x7f871c005e80 comp rx=0 tx=0).stop 2023-09-12T16:03:14.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87300ffa10 msgr2=0x7f8730195200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87300ffa10 0x7f8730195200 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f872400b500 tx=0x7f872400b810 comp rx=0 tx=0).stop 2023-09-12T16:03:14.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 shutdown_connections 2023-09-12T16:03:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87200a0790 0x7f87200a2c50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87301010e0 0x7f873019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87300ffa10 0x7f8730195200 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 --2- 172.21.15.130:0/1667160481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8730100720 0x7f8730195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 >> 172.21.15.130:0/1667160481 conn(0x7f87300fa1c0 msgr2=0x7f87300fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:14.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 shutdown_connections 2023-09-12T16:03:14.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.256+0000 7f8736c73700 1 -- 172.21.15.130:0/1667160481 wait complete. 2023-09-12T16:03:14.269 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:03:14.270 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:03:14.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 -- 172.21.15.130:0/528000907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e8101080 msgr2=0x7f18e8107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/528000907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e8101080 0x7f18e8107930 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f18e000b0d0 tx=0x7f18e000b3e0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 -- 172.21.15.130:0/528000907 shutdown_connections 2023-09-12T16:03:14.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/528000907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e8101080 0x7f18e8107930 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/528000907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18e80ff4f0 0x7f18e80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/528000907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e81006e0 0x7f18e8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 -- 172.21.15.130:0/528000907 >> 172.21.15.130:0/528000907 conn(0x7f18e80faa60 msgr2=0x7f18e80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:14.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 -- 172.21.15.130:0/528000907 shutdown_connections 2023-09-12T16:03:14.400 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 -- 172.21.15.130:0/528000907 wait complete. 2023-09-12T16:03:14.400 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.395+0000 7f18ee8fc700 1 Processor -- start 2023-09-12T16:03:14.400 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18ee8fc700 1 -- start start 2023-09-12T16:03:14.401 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18ee8fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18e80ff4f0 0x7f18e8103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.401 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18ee8fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e81006e0 0x7f18e8101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.401 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18ee8fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e8101080 0x7f18e8102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.401 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18ee8fc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e81027a0 con 0x7f18e81006e0 2023-09-12T16:03:14.402 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18ee8fc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8102910 con 0x7f18e80ff4f0 2023-09-12T16:03:14.402 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18ee8fc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e81a0400 con 0x7f18e8101080 2023-09-12T16:03:14.402 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e77fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e81006e0 0x7f18e8101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e77fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e81006e0 0x7f18e8101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54408/0 (socket says 172.21.15.130:54408) 2023-09-12T16:03:14.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e77fe700 1 -- 172.21.15.130:0/3548838873 learned_addr learned my addr 172.21.15.130:0/3548838873 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:14.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e7fff700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18e80ff4f0 0x7f18e8103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18ece99700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e8101080 0x7f18e8102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e77fe700 1 -- 172.21.15.130:0/3548838873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e8101080 msgr2=0x7f18e8102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e77fe700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e8101080 0x7f18e8102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e77fe700 1 -- 172.21.15.130:0/3548838873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18e80ff4f0 msgr2=0x7f18e8103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e77fe700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18e80ff4f0 0x7f18e8103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e77fe700 1 -- 172.21.15.130:0/3548838873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0009d00 con 0x7f18e81006e0 2023-09-12T16:03:14.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e77fe700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e81006e0 0x7f18e8101ac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e9a0 tx=0x7f18dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:14.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.396+0000 7f18e57fa700 1 -- 172.21.15.130:0/3548838873 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc009790 con 0x7f18e81006e0 2023-09-12T16:03:14.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.397+0000 7f18e57fa700 1 -- 172.21.15.130:0/3548838873 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18dc00cbd0 con 0x7f18e81006e0 2023-09-12T16:03:14.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.397+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e81a06e0 con 0x7f18e81006e0 2023-09-12T16:03:14.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.397+0000 7f18e57fa700 1 -- 172.21.15.130:0/3548838873 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc018680 con 0x7f18e81006e0 2023-09-12T16:03:14.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.397+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e81a0bf0 con 0x7f18e81006e0 2023-09-12T16:03:14.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.397+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e810ad10 con 0x7f18e81006e0 2023-09-12T16:03:14.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.399+0000 7f18e57fa700 1 -- 172.21.15.130:0/3548838873 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f18dc00cd40 con 0x7f18e81006e0 2023-09-12T16:03:14.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.399+0000 7f18e57fa700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f18d00a06d0 0x7f18d00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.399+0000 7f18e57fa700 1 -- 172.21.15.130:0/3548838873 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f18dc014070 con 0x7f18e81006e0 2023-09-12T16:03:14.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.399+0000 7f18e7fff700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f18d00a06d0 0x7f18d00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.400+0000 7f18e7fff700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f18d00a06d0 0x7f18d00a2b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f18e8100540 tx=0x7f18d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:14.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.403+0000 7f18e57fa700 1 -- 172.21.15.130:0/3548838873 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f18dc095c20 con 0x7f18e81006e0 2023-09-12T16:03:14.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:14 smithi159 ceph-mon[102522]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:14.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:14 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/86425856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:14.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:14 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2158771159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:14.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:14 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4184111071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:14.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:14 smithi203 ceph-mon[103431]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:14.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:14 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/86425856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:14.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:14 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2158771159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:14.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:14 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4184111071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:14.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.592+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18e81a0e10 con 0x7f18e81006e0 2023-09-12T16:03:14.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.593+0000 7f18e57fa700 1 -- 172.21.15.130:0/3548838873 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18dc099240 con 0x7f18e81006e0 2023-09-12T16:03:14.594 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:03:14.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f18d00a06d0 msgr2=0x7f18d00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f18d00a06d0 0x7f18d00a2b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f18e8100540 tx=0x7f18d800b410 comp rx=0 tx=0).stop 2023-09-12T16:03:14.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e81006e0 msgr2=0x7f18e8101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e81006e0 0x7f18e8101ac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e9a0 tx=0x7f18dc00ecb0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 shutdown_connections 2023-09-12T16:03:14.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f18d00a06d0 0x7f18d00a2b90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e8101080 0x7f18e8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18e80ff4f0 0x7f18e8103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 --2- 172.21.15.130:0/3548838873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e81006e0 0x7f18e8101ac0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 >> 172.21.15.130:0/3548838873 conn(0x7f18e80faa60 msgr2=0x7f18e8068810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:14.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 shutdown_connections 2023-09-12T16:03:14.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.595+0000 7f18ee8fc700 1 -- 172.21.15.130:0/3548838873 wait complete. 2023-09-12T16:03:14.608 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:03:14.735 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.732+0000 7f51bc05b700 1 -- 172.21.15.130:0/1773449059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b4100620 msgr2=0x7f51b4100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.732+0000 7f51bc05b700 1 --2- 172.21.15.130:0/1773449059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b4100620 0x7f51b4100a40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f51b00099c0 tx=0x7f51b0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.732+0000 7f51bc05b700 1 -- 172.21.15.130:0/1773449059 shutdown_connections 2023-09-12T16:03:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.732+0000 7f51bc05b700 1 --2- 172.21.15.130:0/1773449059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b41021b0 0x7f51b4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.732+0000 7f51bc05b700 1 --2- 172.21.15.130:0/1773449059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b4100620 0x7f51b4100a40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.732+0000 7f51bc05b700 1 --2- 172.21.15.130:0/1773449059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b4101810 0x7f51b4101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.732+0000 7f51bc05b700 1 -- 172.21.15.130:0/1773449059 >> 172.21.15.130:0/1773449059 conn(0x7f51b40fbb70 msgr2=0x7f51b40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:14.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.733+0000 7f51bc05b700 1 -- 172.21.15.130:0/1773449059 shutdown_connections 2023-09-12T16:03:14.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.733+0000 7f51bc05b700 1 -- 172.21.15.130:0/1773449059 wait complete. 2023-09-12T16:03:14.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.733+0000 7f51bc05b700 1 Processor -- start 2023-09-12T16:03:14.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.733+0000 7f51bc05b700 1 -- start start 2023-09-12T16:03:14.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51bc05b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b4100620 0x7f51b4196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51bc05b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b4101810 0x7f51b4196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51bc05b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b41021b0 0x7f51b419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51bc05b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b419deb0 con 0x7f51b41021b0 2023-09-12T16:03:14.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51bc05b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b419dff0 con 0x7f51b4101810 2023-09-12T16:03:14.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51bc05b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b419e180 con 0x7f51b4100620 2023-09-12T16:03:14.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51ba5f8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b41021b0 0x7f51b419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51ba5f8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b41021b0 0x7f51b419d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54424/0 (socket says 172.21.15.130:54424) 2023-09-12T16:03:14.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51ba5f8700 1 -- 172.21.15.130:0/190141549 learned_addr learned my addr 172.21.15.130:0/190141549 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:14.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51b95f6700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b4101810 0x7f51b4196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.740 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51b9df7700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b4100620 0x7f51b4196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.740 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51ba5f8700 1 -- 172.21.15.130:0/190141549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b4100620 msgr2=0x7f51b4196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.740 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51ba5f8700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b4100620 0x7f51b4196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.740 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51ba5f8700 1 -- 172.21.15.130:0/190141549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b4101810 msgr2=0x7f51b4196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51ba5f8700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b4101810 0x7f51b4196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51ba5f8700 1 -- 172.21.15.130:0/190141549 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b00095f0 con 0x7f51b41021b0 2023-09-12T16:03:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51ba5f8700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b41021b0 0x7f51b419d8e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00ef30 tx=0x7f51ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51aaffd700 1 -- 172.21.15.130:0/190141549 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac018070 con 0x7f51b41021b0 2023-09-12T16:03:14.741 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b419e400 con 0x7f51b41021b0 2023-09-12T16:03:14.742 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51aaffd700 1 -- 172.21.15.130:0/190141549 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51ac00f040 con 0x7f51b41021b0 2023-09-12T16:03:14.742 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.734+0000 7f51aaffd700 1 -- 172.21.15.130:0/190141549 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00cd00 con 0x7f51b41021b0 2023-09-12T16:03:14.742 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.735+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b419e910 con 0x7f51b41021b0 2023-09-12T16:03:14.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.736+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b410be90 con 0x7f51b41021b0 2023-09-12T16:03:14.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.736+0000 7f51aaffd700 1 -- 172.21.15.130:0/190141549 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f51ac003a00 con 0x7f51b41021b0 2023-09-12T16:03:14.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.737+0000 7f51aaffd700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51a00a06d0 0x7f51a00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:14.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.737+0000 7f51b9df7700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51a00a06d0 0x7f51a00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:14.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.737+0000 7f51aaffd700 1 -- 172.21.15.130:0/190141549 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f51ac0cb840 con 0x7f51b41021b0 2023-09-12T16:03:14.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.738+0000 7f51b9df7700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51a00a06d0 0x7f51a00a2b90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009990 tx=0x7f51b0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:14.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.741+0000 7f51aaffd700 1 -- 172.21.15.130:0/190141549 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f51ac094f30 con 0x7f51b41021b0 2023-09-12T16:03:14.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.933+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51b4107560 con 0x7f51b41021b0 2023-09-12T16:03:14.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.934+0000 7f51aaffd700 1 -- 172.21.15.130:0/190141549 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51ac098550 con 0x7f51b41021b0 2023-09-12T16:03:14.935 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:14.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51a00a06d0 msgr2=0x7f51a00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51a00a06d0 0x7f51a00a2b90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009990 tx=0x7f51b0005fb0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b41021b0 msgr2=0x7f51b419d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:14.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b41021b0 0x7f51b419d8e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00ef30 tx=0x7f51ac00c3a0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 shutdown_connections 2023-09-12T16:03:14.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51a00a06d0 0x7f51a00a2b90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51b41021b0 0x7f51b419d8e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b4100620 0x7f51b4196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 --2- 172.21.15.130:0/190141549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b4101810 0x7f51b4196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:14.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 >> 172.21.15.130:0/190141549 conn(0x7f51b40fbb70 msgr2=0x7f51b40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:14.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 shutdown_connections 2023-09-12T16:03:14.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:14.936+0000 7f51bc05b700 1 -- 172.21.15.130:0/190141549 wait complete. 2023-09-12T16:03:14.949 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:03:15.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 -- 172.21.15.130:0/1522336068 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 msgr2=0x7f07d8100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1522336068 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 0x7f07d8100b40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0099c0 tx=0x7f07cc009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 -- 172.21.15.130:0/1522336068 shutdown_connections 2023-09-12T16:03:15.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1522336068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07d8101080 0x7f07d8107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1522336068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07d80ff4f0 0x7f07d80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1522336068 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 0x7f07d8100b40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 -- 172.21.15.130:0/1522336068 >> 172.21.15.130:0/1522336068 conn(0x7f07d80faa60 msgr2=0x7f07d80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:15.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 -- 172.21.15.130:0/1522336068 shutdown_connections 2023-09-12T16:03:15.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 -- 172.21.15.130:0/1522336068 wait complete. 2023-09-12T16:03:15.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.077+0000 7f07ddacc700 1 Processor -- start 2023-09-12T16:03:15.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07ddacc700 1 -- start start 2023-09-12T16:03:15.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07ddacc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07d80ff4f0 0x7f07d80682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07ddacc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 0x7f07d819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07ddacc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07d8101080 0x7f07d819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07ddacc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d819dda0 con 0x7f07d81006e0 2023-09-12T16:03:15.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07ddacc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d819df10 con 0x7f07d8101080 2023-09-12T16:03:15.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07ddacc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d819e0d0 con 0x7f07d80ff4f0 2023-09-12T16:03:15.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d6ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 0x7f07d819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d6ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 0x7f07d819afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54436/0 (socket says 172.21.15.130:54436) 2023-09-12T16:03:15.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d6ffd700 1 -- 172.21.15.130:0/1752733590 learned_addr learned my addr 172.21.15.130:0/1752733590 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:15.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d77fe700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07d80ff4f0 0x7f07d80682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d6ffd700 1 -- 172.21.15.130:0/1752733590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07d80ff4f0 msgr2=0x7f07d80682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d6ffd700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07d80ff4f0 0x7f07d80682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d7fff700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07d8101080 0x7f07d819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d6ffd700 1 -- 172.21.15.130:0/1752733590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07d8101080 msgr2=0x7f07d819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d6ffd700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07d8101080 0x7f07d819d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d6ffd700 1 -- 172.21.15.130:0/1752733590 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0095f0 con 0x7f07d81006e0 2023-09-12T16:03:15.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d77fe700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07d80ff4f0 0x7f07d80682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:15.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d6ffd700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 0x7f07d819afc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0036f0 tx=0x7f07cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:15.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d4ff9700 1 -- 172.21.15.130:0/1752733590 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc01d070 con 0x7f07d81006e0 2023-09-12T16:03:15.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d819e350 con 0x7f07d81006e0 2023-09-12T16:03:15.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d4ff9700 1 -- 172.21.15.130:0/1752733590 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07cc004110 con 0x7f07d81006e0 2023-09-12T16:03:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.078+0000 7f07d4ff9700 1 -- 172.21.15.130:0/1752733590 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc022ea0 con 0x7f07d81006e0 2023-09-12T16:03:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.079+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d819e7d0 con 0x7f07d81006e0 2023-09-12T16:03:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.079+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d810ad10 con 0x7f07d81006e0 2023-09-12T16:03:15.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.080+0000 7f07d4ff9700 1 -- 172.21.15.130:0/1752733590 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f07cc022600 con 0x7f07d81006e0 2023-09-12T16:03:15.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.081+0000 7f07d4ff9700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07c00a06d0 0x7f07c00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.081+0000 7f07d77fe700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07c00a06d0 0x7f07c00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.081+0000 7f07d4ff9700 1 -- 172.21.15.130:0/1752733590 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f07cc0cd3d0 con 0x7f07d81006e0 2023-09-12T16:03:15.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.081+0000 7f07d77fe700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07c00a06d0 0x7f07c00a2b90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f07c8005fd0 tx=0x7f07c8005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:15.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.085+0000 7f07d4ff9700 1 -- 172.21.15.130:0/1752733590 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f07cc05ce10 con 0x7f07d81006e0 2023-09-12T16:03:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:15 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1667160481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:15 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3548838873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:15 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/190141549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:15.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.275+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f07d81063f0 con 0x7f07d81006e0 2023-09-12T16:03:15.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.275+0000 7f07d4ff9700 1 -- 172.21.15.130:0/1752733590 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f07cc027020 con 0x7f07d81006e0 2023-09-12T16:03:15.276 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:03:15.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.277+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07c00a06d0 msgr2=0x7f07c00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.277+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07c00a06d0 0x7f07c00a2b90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f07c8005fd0 tx=0x7f07c8005e40 comp rx=0 tx=0).stop 2023-09-12T16:03:15.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.277+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 msgr2=0x7f07d819afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.277+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 0x7f07d819afc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0036f0 tx=0x7f07cc0057f0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.278+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 shutdown_connections 2023-09-12T16:03:15.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.278+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f07c00a06d0 0x7f07c00a2b90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.278+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07d8101080 0x7f07d819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.278+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07d80ff4f0 0x7f07d80682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.278+0000 7f07ddacc700 1 --2- 172.21.15.130:0/1752733590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07d81006e0 0x7f07d819afc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.278+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 >> 172.21.15.130:0/1752733590 conn(0x7f07d80faa60 msgr2=0x7f07d80fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:15.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.278+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 shutdown_connections 2023-09-12T16:03:15.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.278+0000 7f07ddacc700 1 -- 172.21.15.130:0/1752733590 wait complete. 2023-09-12T16:03:15.291 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:03:15.417 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.415+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3740958978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c18101080 msgr2=0x7f7c18107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.415+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3740958978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c18101080 0x7f7c18107930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000b0d0 tx=0x7f7c1000b3e0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.415+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3740958978 shutdown_connections 2023-09-12T16:03:15.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.415+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3740958978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c18101080 0x7f7c18107930 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.415+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3740958978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c180ff4f0 0x7f7c180ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.415+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3740958978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c181006e0 0x7f7c18100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.415+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3740958978 >> 172.21.15.130:0/3740958978 conn(0x7f7c180faa60 msgr2=0x7f7c180fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:15.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.415+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3740958978 shutdown_connections 2023-09-12T16:03:15.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.415+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3740958978 wait complete. 2023-09-12T16:03:15.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1cd43700 1 Processor -- start 2023-09-12T16:03:15.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1cd43700 1 -- start start 2023-09-12T16:03:15.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1cd43700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c180ff4f0 0x7f7c181985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1cd43700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c181006e0 0x7f7c18198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1cd43700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c18101080 0x7f7c1819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1cd43700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c181a0130 con 0x7f7c180ff4f0 2023-09-12T16:03:15.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1cd43700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c181a0270 con 0x7f7c181006e0 2023-09-12T16:03:15.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1cd43700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c181a0400 con 0x7f7c18101080 2023-09-12T16:03:15.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c180ff4f0 0x7f7c181985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c180ff4f0 0x7f7c181985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54448/0 (socket says 172.21.15.130:54448) 2023-09-12T16:03:15.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1659c700 1 -- 172.21.15.130:0/3914494089 learned_addr learned my addr 172.21.15.130:0/3914494089 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:15.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c15d9b700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c181006e0 0x7f7c18198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c16d9d700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c18101080 0x7f7c1819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1659c700 1 -- 172.21.15.130:0/3914494089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c18101080 msgr2=0x7f7c1819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1659c700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c18101080 0x7f7c1819fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1659c700 1 -- 172.21.15.130:0/3914494089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c181006e0 msgr2=0x7f7c18198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1659c700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c181006e0 0x7f7c18198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.416+0000 7f7c1659c700 1 -- 172.21.15.130:0/3914494089 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10009d00 con 0x7f7c180ff4f0 2023-09-12T16:03:15.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.417+0000 7f7c1659c700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c180ff4f0 0x7f7c181985b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e970 tx=0x7f7c0800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:15.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.417+0000 7f7c077fe700 1 -- 172.21.15.130:0/3914494089 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08009790 con 0x7f7c180ff4f0 2023-09-12T16:03:15.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.417+0000 7f7c077fe700 1 -- 172.21.15.130:0/3914494089 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c0800c7e0 con 0x7f7c180ff4f0 2023-09-12T16:03:15.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.417+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c181a06e0 con 0x7f7c180ff4f0 2023-09-12T16:03:15.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.417+0000 7f7c077fe700 1 -- 172.21.15.130:0/3914494089 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08003c70 con 0x7f7c180ff4f0 2023-09-12T16:03:15.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.417+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c181a0bf0 con 0x7f7c180ff4f0 2023-09-12T16:03:15.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.418+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1810ad00 con 0x7f7c180ff4f0 2023-09-12T16:03:15.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.419+0000 7f7c077fe700 1 -- 172.21.15.130:0/3914494089 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f7c0800cc50 con 0x7f7c180ff4f0 2023-09-12T16:03:15.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.420+0000 7f7c077fe700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c000a06d0 0x7f7c000a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.420+0000 7f7c15d9b700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c000a06d0 0x7f7c000a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.420+0000 7f7c077fe700 1 -- 172.21.15.130:0/3914494089 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7c08014070 con 0x7f7c180ff4f0 2023-09-12T16:03:15.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.420+0000 7f7c15d9b700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c000a06d0 0x7f7c000a2b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c005950 tx=0x7f7c0c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:15.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.423+0000 7f7c077fe700 1 -- 172.21.15.130:0/3914494089 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7c08095a40 con 0x7f7c180ff4f0 2023-09-12T16:03:15.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:15 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1667160481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:15.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:15 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3548838873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:15.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:15 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/190141549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:15.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:15 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1667160481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:15.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:15 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3548838873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:15.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:15 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/190141549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:15.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.614+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7c181a0e70 con 0x7f7c180ff4f0 2023-09-12T16:03:15.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.614+0000 7f7c077fe700 1 -- 172.21.15.130:0/3914494089 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7c08003dd0 con 0x7f7c180ff4f0 2023-09-12T16:03:15.615 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:15.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c000a06d0 msgr2=0x7f7c000a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c000a06d0 0x7f7c000a2b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c005950 tx=0x7f7c0c00b410 comp rx=0 tx=0).stop 2023-09-12T16:03:15.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c180ff4f0 msgr2=0x7f7c181985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c180ff4f0 0x7f7c181985b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e970 tx=0x7f7c0800ec80 comp rx=0 tx=0).stop 2023-09-12T16:03:15.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 shutdown_connections 2023-09-12T16:03:15.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7c000a06d0 0x7f7c000a2b90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c18101080 0x7f7c1819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c180ff4f0 0x7f7c181985b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 --2- 172.21.15.130:0/3914494089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c181006e0 0x7f7c18198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 >> 172.21.15.130:0/3914494089 conn(0x7f7c180faa60 msgr2=0x7f7c18067760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:15.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 shutdown_connections 2023-09-12T16:03:15.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.616+0000 7f7c1cd43700 1 -- 172.21.15.130:0/3914494089 wait complete. 2023-09-12T16:03:15.629 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:03:15.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 -- 172.21.15.130:0/1627470935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 msgr2=0x7f3100100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 --2- 172.21.15.130:0/1627470935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 0x7f3100100b30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f30f00099c0 tx=0x7f30f0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 -- 172.21.15.130:0/1627470935 shutdown_connections 2023-09-12T16:03:15.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 --2- 172.21.15.130:0/1627470935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3100101070 0x7f3100107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 --2- 172.21.15.130:0/1627470935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31000ff4e0 0x7f31000ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 --2- 172.21.15.130:0/1627470935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 0x7f3100100b30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 -- 172.21.15.130:0/1627470935 >> 172.21.15.130:0/1627470935 conn(0x7f31000faa30 msgr2=0x7f31000fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:15.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 -- 172.21.15.130:0/1627470935 shutdown_connections 2023-09-12T16:03:15.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 -- 172.21.15.130:0/1627470935 wait complete. 2023-09-12T16:03:15.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.751+0000 7f31060b9700 1 Processor -- start 2023-09-12T16:03:15.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f31060b9700 1 -- start start 2023-09-12T16:03:15.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f31060b9700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31000ff4e0 0x7f31000682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f31060b9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 0x7f310019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f31060b9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3100101070 0x7f310019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f31060b9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310019dd10 con 0x7f31001006d0 2023-09-12T16:03:15.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f31060b9700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310019de80 con 0x7f31000ff4e0 2023-09-12T16:03:15.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f31060b9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310019e040 con 0x7f3100101070 2023-09-12T16:03:15.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30feffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 0x7f310019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30feffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 0x7f310019af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54464/0 (socket says 172.21.15.130:54464) 2023-09-12T16:03:15.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30feffd700 1 -- 172.21.15.130:0/962333930 learned_addr learned my addr 172.21.15.130:0/962333930 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:15.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30fffff700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3100101070 0x7f310019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30ff7fe700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31000ff4e0 0x7f31000682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30feffd700 1 -- 172.21.15.130:0/962333930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3100101070 msgr2=0x7f310019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30feffd700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3100101070 0x7f310019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30feffd700 1 -- 172.21.15.130:0/962333930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31000ff4e0 msgr2=0x7f31000682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30feffd700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31000ff4e0 0x7f31000682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:15.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30feffd700 1 -- 172.21.15.130:0/962333930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00095f0 con 0x7f31001006d0 2023-09-12T16:03:15.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.752+0000 7f30feffd700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 0x7f310019af30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009fd0 tx=0x7f30f000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:15.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.753+0000 7f30fcff9700 1 -- 172.21.15.130:0/962333930 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f001d070 con 0x7f31001006d0 2023-09-12T16:03:15.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.753+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310019e2c0 con 0x7f31001006d0 2023-09-12T16:03:15.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.753+0000 7f30fcff9700 1 -- 172.21.15.130:0/962333930 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30f00054f0 con 0x7f31001006d0 2023-09-12T16:03:15.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.753+0000 7f30fcff9700 1 -- 172.21.15.130:0/962333930 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0004c70 con 0x7f31001006d0 2023-09-12T16:03:15.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.753+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310019e740 con 0x7f31001006d0 2023-09-12T16:03:15.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.753+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310010ad00 con 0x7f31001006d0 2023-09-12T16:03:15.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.755+0000 7f30fcff9700 1 -- 172.21.15.130:0/962333930 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f30f000fc30 con 0x7f31001006d0 2023-09-12T16:03:15.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.756+0000 7f30fcff9700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f30e80a0710 0x7f30e80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:15.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.756+0000 7f30ff7fe700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f30e80a0710 0x7f30e80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:15.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.756+0000 7f30fcff9700 1 -- 172.21.15.130:0/962333930 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f30f00cd3f0 con 0x7f31001006d0 2023-09-12T16:03:15.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.756+0000 7f30ff7fe700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f30e80a0710 0x7f30e80a2bd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3100100530 tx=0x7f30f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:15.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.759+0000 7f30fcff9700 1 -- 172.21.15.130:0/962333930 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f30f0096ae0 con 0x7f31001006d0 2023-09-12T16:03:15.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.994+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3100106330 con 0x7f31001006d0 2023-09-12T16:03:15.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.995+0000 7f30fcff9700 1 -- 172.21.15.130:0/962333930 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11342 (secure 0 0 0) 0x7f30f009a100 con 0x7f31001006d0 2023-09-12T16:03:15.996 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:15.997 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:25.646573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:03:15.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f30e80a0710 msgr2=0x7f30e80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f30e80a0710 0x7f30e80a2bd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3100100530 tx=0x7f30f4009260 comp rx=0 tx=0).stop 2023-09-12T16:03:15.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 msgr2=0x7f310019af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:15.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 0x7f310019af30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009fd0 tx=0x7f30f000f740 comp rx=0 tx=0).stop 2023-09-12T16:03:16.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 shutdown_connections 2023-09-12T16:03:16.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f30e80a0710 0x7f30e80a2bd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3100101070 0x7f310019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31000ff4e0 0x7f31000682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 --2- 172.21.15.130:0/962333930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31001006d0 0x7f310019af30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 >> 172.21.15.130:0/962333930 conn(0x7f31000faa30 msgr2=0x7f31000fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:16.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 shutdown_connections 2023-09-12T16:03:16.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:15.997+0000 7f31060b9700 1 -- 172.21.15.130:0/962333930 wait complete. 2023-09-12T16:03:16.002 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 24 2023-09-12T16:03:16.011 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:16.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 -- 172.21.15.130:0/318569681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300fec70 msgr2=0x7f0d300ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:16.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/318569681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300fec70 0x7f0d300ff090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0099c0 tx=0x7f0d2c009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 -- 172.21.15.130:0/318569681 shutdown_connections 2023-09-12T16:03:16.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/318569681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d30100800 0x7f0d301070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/318569681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300fec70 0x7f0d300ff090 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/318569681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d300ffe60 0x7f0d301002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 -- 172.21.15.130:0/318569681 >> 172.21.15.130:0/318569681 conn(0x7f0d300fa1c0 msgr2=0x7f0d300fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:16.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 -- 172.21.15.130:0/318569681 shutdown_connections 2023-09-12T16:03:16.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 -- 172.21.15.130:0/318569681 wait complete. 2023-09-12T16:03:16.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 Processor -- start 2023-09-12T16:03:16.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.135+0000 7f0d37b0c700 1 -- start start 2023-09-12T16:03:16.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d37b0c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d300fec70 0x7f0d30195160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:16.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d37b0c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300ffe60 0x7f0d301956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:16.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d37b0c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d30100800 0x7f0d3019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:16.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d37b0c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3019cce0 con 0x7f0d300ffe60 2023-09-12T16:03:16.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d37b0c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3019ce20 con 0x7f0d300fec70 2023-09-12T16:03:16.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d37b0c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3019cfb0 con 0x7f0d30100800 2023-09-12T16:03:16.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d350a7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300ffe60 0x7f0d301956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:16.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d350a7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300ffe60 0x7f0d301956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54478/0 (socket says 172.21.15.130:54478) 2023-09-12T16:03:16.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d350a7700 1 -- 172.21.15.130:0/2589456897 learned_addr learned my addr 172.21.15.130:0/2589456897 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:16.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d358a8700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d300fec70 0x7f0d30195160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:16.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d360a9700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d30100800 0x7f0d3019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:16.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d350a7700 1 -- 172.21.15.130:0/2589456897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d30100800 msgr2=0x7f0d3019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:16.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d350a7700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d30100800 0x7f0d3019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d350a7700 1 -- 172.21.15.130:0/2589456897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d300fec70 msgr2=0x7f0d30195160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:16.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d350a7700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d300fec70 0x7f0d30195160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:16.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d350a7700 1 -- 172.21.15.130:0/2589456897 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c0095f0 con 0x7f0d300ffe60 2023-09-12T16:03:16.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d350a7700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300ffe60 0x7f0d301956a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000d950 tx=0x7f0d2000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:16.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d26ffd700 1 -- 172.21.15.130:0/2589456897 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2000f890 con 0x7f0d300ffe60 2023-09-12T16:03:16.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d26ffd700 1 -- 172.21.15.130:0/2589456897 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d20010460 con 0x7f0d300ffe60 2023-09-12T16:03:16.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3019d290 con 0x7f0d300ffe60 2023-09-12T16:03:16.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d26ffd700 1 -- 172.21.15.130:0/2589456897 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2000e610 con 0x7f0d300ffe60 2023-09-12T16:03:16.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.136+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3019d7a0 con 0x7f0d300ffe60 2023-09-12T16:03:16.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.137+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d30105bf0 con 0x7f0d300ffe60 2023-09-12T16:03:16.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.138+0000 7f0d26ffd700 1 -- 172.21.15.130:0/2589456897 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f0d200108d0 con 0x7f0d300ffe60 2023-09-12T16:03:16.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.139+0000 7f0d26ffd700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d1c0a0710 0x7f0d1c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:16.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.139+0000 7f0d358a8700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d1c0a0710 0x7f0d1c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:16.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.139+0000 7f0d26ffd700 1 -- 172.21.15.130:0/2589456897 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0d200cbdf0 con 0x7f0d300ffe60 2023-09-12T16:03:16.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.139+0000 7f0d358a8700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d1c0a0710 0x7f0d1c0a2bd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009f50 tx=0x7f0d2c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:16.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.143+0000 7f0d26ffd700 1 -- 172.21.15.130:0/2589456897 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0d200954b0 con 0x7f0d300ffe60 2023-09-12T16:03:16.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.333+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d3010af20 con 0x7f0d300ffe60 2023-09-12T16:03:17.089 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:16 smithi203 ceph-mon[103431]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:17.089 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:16 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1752733590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:17.089 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:16 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3914494089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:17.089 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:16 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/962333930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:17.090 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:16 smithi159 ceph-mon[102522]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:17.090 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:16 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1752733590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:17.090 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:16 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3914494089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:17.090 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:16 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/962333930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:17.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.335+0000 7f0d26ffd700 1 -- 172.21.15.130:0/2589456897 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24473 (secure 0 0 0) 0x7f0d20016090 con 0x7f0d300ffe60 2023-09-12T16:03:17.091 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:17.091 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":88,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:51.826597+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:17.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d1c0a0710 msgr2=0x7f0d1c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d1c0a0710 0x7f0d1c0a2bd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009f50 tx=0x7f0d2c00b540 comp rx=0 tx=0).stop 2023-09-12T16:03:17.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300ffe60 msgr2=0x7f0d301956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300ffe60 0x7f0d301956a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000d950 tx=0x7f0d2000dc60 comp rx=0 tx=0).stop 2023-09-12T16:03:17.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 shutdown_connections 2023-09-12T16:03:17.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0d1c0a0710 0x7f0d1c0a2bd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d30100800 0x7f0d3019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d300fec70 0x7f0d30195160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 --2- 172.21.15.130:0/2589456897 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d300ffe60 0x7f0d301956a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 >> 172.21.15.130:0/2589456897 conn(0x7f0d300fa1c0 msgr2=0x7f0d300fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.337+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 shutdown_connections 2023-09-12T16:03:17.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:16.338+0000 7f0d37b0c700 1 -- 172.21.15.130:0/2589456897 wait complete. 2023-09-12T16:03:17.095 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:16 smithi130 ceph-mon[98670]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:17.095 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:16 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1752733590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:17.095 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:16 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3914494089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:17.095 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:16 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/962333930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:17.097 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:03:17.230 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.226+0000 7ff7d624e700 1 -- 172.21.15.130:0/533549972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 msgr2=0x7ff7d00ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.230 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.226+0000 7ff7d624e700 1 --2- 172.21.15.130:0/533549972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 0x7ff7d00ff090 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80099c0 tx=0x7ff7b8009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.226+0000 7ff7d624e700 1 -- 172.21.15.130:0/533549972 shutdown_connections 2023-09-12T16:03:17.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.226+0000 7ff7d624e700 1 --2- 172.21.15.130:0/533549972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7d0100800 0x7ff7d01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.226+0000 7ff7d624e700 1 --2- 172.21.15.130:0/533549972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 0x7ff7d00ff090 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.226+0000 7ff7d624e700 1 --2- 172.21.15.130:0/533549972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7d00ffe60 0x7ff7d01002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.226+0000 7ff7d624e700 1 -- 172.21.15.130:0/533549972 >> 172.21.15.130:0/533549972 conn(0x7ff7d00fa1c0 msgr2=0x7ff7d00fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.226+0000 7ff7d624e700 1 -- 172.21.15.130:0/533549972 shutdown_connections 2023-09-12T16:03:17.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.226+0000 7ff7d624e700 1 -- 172.21.15.130:0/533549972 wait complete. 2023-09-12T16:03:17.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7d624e700 1 Processor -- start 2023-09-12T16:03:17.232 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7d624e700 1 -- start start 2023-09-12T16:03:17.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7d624e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 0x7ff7d01087e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7d624e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7d00ffe60 0x7ff7d019b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7d624e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7d0100800 0x7ff7d019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7d624e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d019de70 con 0x7ff7d00fec70 2023-09-12T16:03:17.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7d624e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d019dfe0 con 0x7ff7d0100800 2023-09-12T16:03:17.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7d624e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d019e1a0 con 0x7ff7d00ffe60 2023-09-12T16:03:17.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7cf7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 0x7ff7d01087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7cf7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 0x7ff7d01087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54490/0 (socket says 172.21.15.130:54490) 2023-09-12T16:03:17.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7cf7fe700 1 -- 172.21.15.130:0/4271345652 learned_addr learned my addr 172.21.15.130:0/4271345652 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.227+0000 7ff7ceffd700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7d00ffe60 0x7ff7d019b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7cffff700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7d0100800 0x7ff7d019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7cf7fe700 1 -- 172.21.15.130:0/4271345652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7d00ffe60 msgr2=0x7ff7d019b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7cf7fe700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7d00ffe60 0x7ff7d019b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7cf7fe700 1 -- 172.21.15.130:0/4271345652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7d0100800 msgr2=0x7ff7d019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7cf7fe700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7d0100800 0x7ff7d019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7cf7fe700 1 -- 172.21.15.130:0/4271345652 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80095f0 con 0x7ff7d00fec70 2023-09-12T16:03:17.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7cffff700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7d0100800 0x7ff7d019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:17.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7cf7fe700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 0x7ff7d01087e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009990 tx=0x7ff7b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7ccff9700 1 -- 172.21.15.130:0/4271345652 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b801d070 con 0x7ff7d00fec70 2023-09-12T16:03:17.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d019e420 con 0x7ff7d00fec70 2023-09-12T16:03:17.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7ccff9700 1 -- 172.21.15.130:0/4271345652 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b80054f0 con 0x7ff7d00fec70 2023-09-12T16:03:17.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7ccff9700 1 -- 172.21.15.130:0/4271345652 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8004bd0 con 0x7ff7d00fec70 2023-09-12T16:03:17.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.228+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d019e850 con 0x7ff7d00fec70 2023-09-12T16:03:17.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.229+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d010ad70 con 0x7ff7d00fec70 2023-09-12T16:03:17.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.230+0000 7ff7ccff9700 1 -- 172.21.15.130:0/4271345652 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7ff7b800fb80 con 0x7ff7d00fec70 2023-09-12T16:03:17.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.230+0000 7ff7ccff9700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff7bc0a0710 0x7ff7bc0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.231+0000 7ff7ceffd700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff7bc0a0710 0x7ff7bc0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.231+0000 7ff7ccff9700 1 -- 172.21.15.130:0/4271345652 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff7b80cd310 con 0x7ff7d00fec70 2023-09-12T16:03:17.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.231+0000 7ff7ceffd700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff7bc0a0710 0x7ff7bc0a2bd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0108550 tx=0x7ff7c00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.234+0000 7ff7ccff9700 1 -- 172.21.15.130:0/4271345652 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff7b8096a00 con 0x7ff7d00fec70 2023-09-12T16:03:17.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.469+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7d0105b70 con 0x7ff7d00fec70 2023-09-12T16:03:17.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.470+0000 7ff7ccff9700 1 -- 172.21.15.130:0/4271345652 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11342 (secure 0 0 0) 0x7ff7b809a020 con 0x7ff7d00fec70 2023-09-12T16:03:17.471 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:17.471 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:25.646573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:03:17.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff7bc0a0710 msgr2=0x7ff7bc0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff7bc0a0710 0x7ff7bc0a2bd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0108550 tx=0x7ff7c00073d0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 msgr2=0x7ff7d01087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 0x7ff7d01087e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009990 tx=0x7ff7b800f690 comp rx=0 tx=0).stop 2023-09-12T16:03:17.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 shutdown_connections 2023-09-12T16:03:17.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff7bc0a0710 0x7ff7bc0a2bd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7d0100800 0x7ff7d019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7d00fec70 0x7ff7d01087e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 --2- 172.21.15.130:0/4271345652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7d00ffe60 0x7ff7d019b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 >> 172.21.15.130:0/4271345652 conn(0x7ff7d00fa1c0 msgr2=0x7ff7d00fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 shutdown_connections 2023-09-12T16:03:17.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.472+0000 7ff7d624e700 1 -- 172.21.15.130:0/4271345652 wait complete. 2023-09-12T16:03:17.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:17 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2589456897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:17.477 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 24 2023-09-12T16:03:17.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:17 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2589456897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:17.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-12T16:02:04.422106+0000', 'modified': '2023-09-12T16:02:25.646573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24593, 'mds_1': 24592, 'mds_2': 24574}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24533': {'gid': 24533, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/3667735298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 3667735298}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3667735298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24557': {'gid': 24557, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6837/3413713845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 3413713845}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 3413713845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24562': {'gid': 24562, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/1801270748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 1801270748}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 1801270748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24574': {'gid': 24574, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.203:6837/3950241499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 3950241499}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 3950241499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24592': {'gid': 24592, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.203:6839/2671073294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 2671073294}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 2671073294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6839/3094783118', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 3094783118}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 3094783118}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-12T16:03:17.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-09-12T16:03:17.486 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-12T16:03:17.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:17 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2589456897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:17.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 -- 172.21.15.130:0/616308089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100100 msgr2=0x7f5bb81079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 --2- 172.21.15.130:0/616308089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100100 0x7f5bb81079a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000b0a0 tx=0x7f5bb000b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 -- 172.21.15.130:0/616308089 shutdown_connections 2023-09-12T16:03:17.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 --2- 172.21.15.130:0/616308089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100100 0x7f5bb81079a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 --2- 172.21.15.130:0/616308089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bb8100af0 0x7f5bb8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 --2- 172.21.15.130:0/616308089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb80ff740 0x7f5bb80ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 -- 172.21.15.130:0/616308089 >> 172.21.15.130:0/616308089 conn(0x7f5bb80fa1d0 msgr2=0x7f5bb80fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 -- 172.21.15.130:0/616308089 shutdown_connections 2023-09-12T16:03:17.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 -- 172.21.15.130:0/616308089 wait complete. 2023-09-12T16:03:17.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.607+0000 7f5bc03db700 1 Processor -- start 2023-09-12T16:03:17.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bc03db700 1 -- start start 2023-09-12T16:03:17.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bc03db700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb80ff740 0x7f5bb81034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bc03db700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bb8100100 0x7f5bb8101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bc03db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100af0 0x7f5bb8102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bc03db700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8102810 con 0x7f5bb8100af0 2023-09-12T16:03:17.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bc03db700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8102980 con 0x7f5bb80ff740 2023-09-12T16:03:17.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bc03db700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb81a0460 con 0x7f5bb8100100 2023-09-12T16:03:17.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe978700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100af0 0x7f5bb8102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe978700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100af0 0x7f5bb8102070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54506/0 (socket says 172.21.15.130:54506) 2023-09-12T16:03:17.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe978700 1 -- 172.21.15.130:0/2920153238 learned_addr learned my addr 172.21.15.130:0/2920153238 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe177700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb80ff740 0x7f5bb81034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbd976700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bb8100100 0x7f5bb8101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe978700 1 -- 172.21.15.130:0/2920153238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bb8100100 msgr2=0x7f5bb8101b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe978700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bb8100100 0x7f5bb8101b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe978700 1 -- 172.21.15.130:0/2920153238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb80ff740 msgr2=0x7f5bb81034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe978700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb80ff740 0x7f5bb81034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe978700 1 -- 172.21.15.130:0/2920153238 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb0009d00 con 0x7f5bb8100af0 2023-09-12T16:03:17.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bbe978700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100af0 0x7f5bb8102070 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000b780 tx=0x7f5bb000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bab7fe700 1 -- 172.21.15.130:0/2920153238 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000e040 con 0x7f5bb8100af0 2023-09-12T16:03:17.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bab7fe700 1 -- 172.21.15.130:0/2920153238 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb00094a0 con 0x7f5bb8100af0 2023-09-12T16:03:17.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bc03db700 1 -- 172.21.15.130:0/2920153238 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb81a06e0 con 0x7f5bb8100af0 2023-09-12T16:03:17.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.608+0000 7f5bab7fe700 1 -- 172.21.15.130:0/2920153238 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0005390 con 0x7f5bb8100af0 2023-09-12T16:03:17.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.609+0000 7f5bc03db700 1 -- 172.21.15.130:0/2920153238 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb81a0bb0 con 0x7f5bb8100af0 2023-09-12T16:03:17.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.611+0000 7f5bab7fe700 1 -- 172.21.15.130:0/2920153238 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f5bb001d470 con 0x7f5bb8100af0 2023-09-12T16:03:17.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.611+0000 7f5bab7fe700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ba40a0750 0x7f5ba40a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.611+0000 7f5bbe177700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ba40a0750 0x7f5ba40a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.611+0000 7f5bab7fe700 1 -- 172.21.15.130:0/2920153238 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5bb00cd590 con 0x7f5bb8100af0 2023-09-12T16:03:17.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.612+0000 7f5bbe177700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ba40a0750 0x7f5ba40a2c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8067540 tx=0x7f5bb4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.612+0000 7f5bcc5a1b80 1 Processor -- start 2023-09-12T16:03:17.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.612+0000 7f5bcc5a1b80 1 -- start start 2023-09-12T16:03:17.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1dbb0 0x562be6e1dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 0x562be6e1c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 0x562be6e1d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6d756e0 con 0x562be6e1dbb0 2023-09-12T16:03:17.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6d73000 con 0x562be6e1cdb0 2023-09-12T16:03:17.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6d6e8f0 con 0x562be6e1e550 2023-09-12T16:03:17.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe177700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1dbb0 0x562be6e1dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe177700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1dbb0 0x562be6e1dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54510/0 (socket says 172.21.15.130:54510) 2023-09-12T16:03:17.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe177700 1 -- 172.21.15.130:0/278466946 learned_addr learned my addr 172.21.15.130:0/278466946 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbd976700 1 --2- 172.21.15.130:0/278466946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 0x562be6e1c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe978700 1 --2- 172.21.15.130:0/278466946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 0x562be6e1d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe177700 1 -- 172.21.15.130:0/278466946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 msgr2=0x562be6e1c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe177700 1 --2- 172.21.15.130:0/278466946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 0x562be6e1c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe177700 1 -- 172.21.15.130:0/278466946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 msgr2=0x562be6e1d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe177700 1 --2- 172.21.15.130:0/278466946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 0x562be6e1d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe177700 1 -- 172.21.15.130:0/278466946 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be6b9ad90 con 0x562be6e1dbb0 2023-09-12T16:03:17.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.613+0000 7f5bbe177700 1 --2- 172.21.15.130:0/278466946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1dbb0 0x562be6e1dfd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4005950 tx=0x7f5bb4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7987639969462a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.614+0000 7f5ba8d89700 1 -- 172.21.15.130:0/278466946 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4004220 con 0x562be6e1dbb0 2023-09-12T16:03:17.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.614+0000 7f5ba8d89700 1 -- 172.21.15.130:0/278466946 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb4021070 con 0x562be6e1dbb0 2023-09-12T16:03:17.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.614+0000 7f5ba8d89700 1 -- 172.21.15.130:0/278466946 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4003c60 con 0x562be6e1dbb0 2023-09-12T16:03:17.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.615+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/278466946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1dbb0 msgr2=0x562be6e1dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.615+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/278466946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1dbb0 0x562be6e1dfd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4005950 tx=0x7f5bb4014000 comp rx=0 tx=0).stop 2023-09-12T16:03:17.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.615+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/278466946 shutdown_connections 2023-09-12T16:03:17.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.615+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/278466946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 0x562be6e1d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.615+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/278466946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1dbb0 0x562be6e1dfd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.615+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/278466946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 0x562be6e1c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.615+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/278466946 >> 172.21.15.130:0/278466946 conn(0x562be6d91040 msgr2=0x562be6e176d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.615+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/278466946 shutdown_connections 2023-09-12T16:03:17.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.615+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/278466946 wait complete. 2023-09-12T16:03:17.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bcc5a1b80 1 -- start start 2023-09-12T16:03:17.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bcc5a1b80 1 Processor -- start 2023-09-12T16:03:17.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 0x562be6e26af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1e550 0x562be6e2db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6d0cb20 con 0x562be6e1cdb0 2023-09-12T16:03:17.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6d0c820 con 0x562be6e1e550 2023-09-12T16:03:17.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6bf08d0 con 0x562be6e1dbb0 2023-09-12T16:03:17.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bcc5a1b80 10 client.0 _ll_register_callbacks cb 0x562be6c56a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:17.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe177700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe177700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e265b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54516/0 (socket says 172.21.15.130:54516) 2023-09-12T16:03:17.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe177700 1 -- 172.21.15.130:0/4169954283 learned_addr learned my addr 172.21.15.130:0/4169954283 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbd976700 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 0x562be6e26af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe978700 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1e550 0x562be6e2db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe177700 1 -- 172.21.15.130:0/4169954283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 msgr2=0x562be6e26af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe177700 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 0x562be6e26af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe177700 1 -- 172.21.15.130:0/4169954283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1e550 msgr2=0x562be6e2db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe177700 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1e550 0x562be6e2db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe177700 1 -- 172.21.15.130:0/4169954283 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb4017070 con 0x562be6e1cdb0 2023-09-12T16:03:17.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe978700 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1e550 0x562be6e2db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:17.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5bbe177700 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e265b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009fd0 tx=0x7f5bb4005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b70edda585d471d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.616+0000 7f5ba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:17.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/4169954283 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400edd0 con 0x562be6e1cdb0 2023-09-12T16:03:17.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/4169954283 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5bb4003960 con 0x562be6e1cdb0 2023-09-12T16:03:17.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/4169954283 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562be6cc64a0 con 0x562be6e1cdb0 2023-09-12T16:03:17.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/4169954283 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb4003b20 con 0x562be6e1cdb0 2023-09-12T16:03:17.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5bcc5a1b80 10 client.15372 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:17.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/4169954283 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562be6b83c10 con 0x562be6e1cdb0 2023-09-12T16:03:17.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/4169954283 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb4038020 con 0x562be6e1cdb0 2023-09-12T16:03:17.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/4169954283 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5bb40269f0 con 0x562be6e1cdb0 2023-09-12T16:03:17.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5bcc5a1b80 10 client.15372 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:17.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5bcc5a1b80 10 client.15372 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-12T16:03:17.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5bcc5a1b80 20 client.15372 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:17.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x562be6e31b40 0x562be6e30b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5bcc5a1b80 4 client.15372 mds_command: new command op to 24574 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:03:17.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.617+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/4169954283 --> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562be6d83850 con 0x562be6e31b40 2023-09-12T16:03:17.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.618+0000 7f5bbd976700 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x562be6e31b40 0x562be6e30b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.618+0000 7f5bbd976700 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x562be6e31b40 0x562be6e30b70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b22b46012557833c server_cookie=d9918ae85e2a905e in_seq=0 out_seq=0 2023-09-12T16:03:17.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.618+0000 7f5ba1ffb700 10 client.15372 ms_handle_connect on v2:172.21.15.203:6836/3950241499 2023-09-12T16:03:17.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/4169954283 <== mds.2 v2:172.21.15.203:6836/3950241499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562be6d83850 con 0x562be6e31b40 2023-09-12T16:03:17.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5ba1ffb700 10 client.15372 handle_command_reply: tid=0 2023-09-12T16:03:17.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 client.15372 shutdown 2023-09-12T16:03:17.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/4169954283 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 msgr2=0x562be6e265b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e265b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009fd0 tx=0x7f5bb4005700 comp rx=0 tx=0).stop 2023-09-12T16:03:17.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/4169954283 shutdown_connections 2023-09-12T16:03:17.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x562be6e31b40 0x562be6e30b70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1e550 0x562be6e2db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5ba1ffb700 0 client.15372 ms_handle_reset on v2:172.21.15.203:6836/3950241499 2023-09-12T16:03:17.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e265b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/4169954283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 0x562be6e26af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/4169954283 >> 172.21.15.130:0/4169954283 conn(0x562be6d91040 msgr2=0x562be6e16d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/4169954283 shutdown_connections 2023-09-12T16:03:17.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/4169954283 wait complete. 2023-09-12T16:03:17.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.619+0000 7f5bcc5a1b80 20 client.15372 trim_cache size 0 max 16384 2023-09-12T16:03:17.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bcc5a1b80 1 Processor -- start 2023-09-12T16:03:17.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bcc5a1b80 1 -- start start 2023-09-12T16:03:17.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e1c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1dbb0 0x562be6e2daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 0x562be6e16d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6d37f70 con 0x562be6e1cdb0 2023-09-12T16:03:17.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6d2aed0 con 0x562be6e1dbb0 2023-09-12T16:03:17.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6e24eb0 con 0x562be6e1e550 2023-09-12T16:03:17.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bbe177700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e1c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bbe177700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e1c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54524/0 (socket says 172.21.15.130:54524) 2023-09-12T16:03:17.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bbe177700 1 -- 172.21.15.130:0/1025444708 learned_addr learned my addr 172.21.15.130:0/1025444708 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.636+0000 7f5bbe978700 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 0x562be6e16d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bbe177700 1 -- 172.21.15.130:0/1025444708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 msgr2=0x562be6e16d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bbe177700 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 0x562be6e16d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bbd976700 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1dbb0 0x562be6e2daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bbe177700 1 -- 172.21.15.130:0/1025444708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1dbb0 msgr2=0x562be6e2daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bbe177700 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1dbb0 0x562be6e2daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bbe177700 1 -- 172.21.15.130:0/1025444708 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be6bc1200 con 0x562be6e1cdb0 2023-09-12T16:03:17.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bbd976700 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1dbb0 0x562be6e2daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:17.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bbe177700 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e1c6c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4011b60 tx=0x7f5bb40093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a009e67ed3d20d6a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5ba958a700 1 -- 172.21.15.130:0/1025444708 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40305c0 con 0x562be6e1cdb0 2023-09-12T16:03:17.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5ba958a700 1 -- 172.21.15.130:0/1025444708 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb4038030 con 0x562be6e1cdb0 2023-09-12T16:03:17.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5ba958a700 1 -- 172.21.15.130:0/1025444708 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4026480 con 0x562be6e1cdb0 2023-09-12T16:03:17.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/1025444708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 msgr2=0x562be6e1c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.637+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e1c6c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4011b60 tx=0x7f5bb40093f0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/1025444708 shutdown_connections 2023-09-12T16:03:17.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1e550 0x562be6e16d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1cdb0 0x562be6e1c6c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/1025444708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1dbb0 0x562be6e2daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/1025444708 >> 172.21.15.130:0/1025444708 conn(0x562be6d91040 msgr2=0x562be6e1d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/1025444708 shutdown_connections 2023-09-12T16:03:17.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/1025444708 wait complete. 2023-09-12T16:03:17.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 -- start start 2023-09-12T16:03:17.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 Processor -- start 2023-09-12T16:03:17.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 0x562be6e2aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 0x562be6e2afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1e550 0x562be6e85b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6e860d0 con 0x562be6e1e550 2023-09-12T16:03:17.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6e86210 con 0x562be6e1cdb0 2023-09-12T16:03:17.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6e86350 con 0x562be6e1dbb0 2023-09-12T16:03:17.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bcc5a1b80 10 client.0 _ll_register_callbacks cb 0x7f5bb0004e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:17.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bbe978700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1e550 0x562be6e85b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bbe978700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1e550 0x562be6e85b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54534/0 (socket says 172.21.15.130:54534) 2023-09-12T16:03:17.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bbe978700 1 -- 172.21.15.130:0/437023995 learned_addr learned my addr 172.21.15.130:0/437023995 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bbe177700 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 0x562be6e2aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.638+0000 7f5bbd976700 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 0x562be6e2afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bbe978700 1 -- 172.21.15.130:0/437023995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 msgr2=0x562be6e2afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bbe978700 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 0x562be6e2afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bbe978700 1 -- 172.21.15.130:0/437023995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 msgr2=0x562be6e2aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bbe978700 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 0x562be6e2aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bbe978700 1 -- 172.21.15.130:0/437023995 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562be6b83c10 con 0x562be6e1e550 2023-09-12T16:03:17.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bbe978700 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1e550 0x562be6e85b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00cac60 tx=0x7f5bb009a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5228472c5c11b33c server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5ba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:17.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/437023995 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb009a9a0 con 0x562be6e1e550 2023-09-12T16:03:17.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/437023995 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5bb009e7d0 con 0x562be6e1e550 2023-09-12T16:03:17.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/437023995 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bb4021070 con 0x562be6e1e550 2023-09-12T16:03:17.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/437023995 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb00a2980 con 0x562be6e1e550 2023-09-12T16:03:17.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bcc5a1b80 10 client.15378 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:17.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/437023995 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562be6b99260 con 0x562be6e1e550 2023-09-12T16:03:17.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/437023995 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb4021070 con 0x562be6e1e550 2023-09-12T16:03:17.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/437023995 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5bb0099e90 con 0x562be6e1e550 2023-09-12T16:03:17.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bcc5a1b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:17.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bcc5a1b80 10 client.15378 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-12T16:03:17.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bcc5a1b80 20 client.15378 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:17.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x562be6e8b350 0x562be6e87840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bcc5a1b80 4 client.15378 mds_command: new command op to 24574 tid=0[{"prefix": "status"}] 2023-09-12T16:03:17.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.639+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/437023995 --> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562be6d678d0 con 0x562be6e8b350 2023-09-12T16:03:17.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bbe177700 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x562be6e8b350 0x562be6e87840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5ba1ffb700 10 client.15378 ms_handle_connect on v2:172.21.15.203:6836/3950241499 2023-09-12T16:03:17.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bbe177700 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x562be6e8b350 0x562be6e87840 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=994ed10b25c3b8a0 server_cookie=b9e231607aa97341 in_seq=0 out_seq=0 2023-09-12T16:03:17.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5ba1ffb700 1 -- 172.21.15.130:0/437023995 <== mds.2 v2:172.21.15.203:6836/3950241499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x562be6d678d0 con 0x562be6e8b350 2023-09-12T16:03:17.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5ba1ffb700 10 client.15378 handle_command_reply: tid=0 2023-09-12T16:03:17.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 client.15378 shutdown 2023-09-12T16:03:17.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/437023995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1e550 msgr2=0x562be6e85b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1e550 0x562be6e85b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00cac60 tx=0x7f5bb009a4e0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/437023995 shutdown_connections 2023-09-12T16:03:17.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x562be6e8b350 0x562be6e87840 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562be6e1e550 0x562be6e85b90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5ba1ffb700 0 client.15378 ms_handle_reset on v2:172.21.15.203:6836/3950241499 2023-09-12T16:03:17.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562be6e1cdb0 0x562be6e2aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 --2- 172.21.15.130:0/437023995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562be6e1dbb0 0x562be6e2afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/437023995 >> 172.21.15.130:0/437023995 conn(0x562be6d91040 msgr2=0x562be6e16d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.640+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/437023995 shutdown_connections 2023-09-12T16:03:17.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bcc5a1b80 1 -- 172.21.15.130:0/437023995 wait complete. 2023-09-12T16:03:17.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bcc5a1b80 20 client.15378 trim_cache size 0 max 16384 2023-09-12T16:03:17.663 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:03:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "cluster_fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T16:03:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "whoami": 2, 2023-09-12T16:03:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 24574, 2023-09-12T16:03:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "want_state": "up:active", 2023-09-12T16:03:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "state": "up:active", 2023-09-12T16:03:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs", 2023-09-12T16:03:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "rank_uptime": 53.001272729, 2023-09-12T16:03:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "mdsmap_epoch": 24, 2023-09-12T16:03:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch": 88, 2023-09-12T16:03:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch_barrier": 86, 2023-09-12T16:03:17.666 INFO:teuthology.orchestra.run.smithi130.stdout: "uptime": 89.334110421000005 2023-09-12T16:03:17.666 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:03:17.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 -- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ba40a0750 msgr2=0x7f5ba40a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ba40a0750 0x7f5ba40a2c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8067540 tx=0x7f5bb4009260 comp rx=0 tx=0).stop 2023-09-12T16:03:17.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 -- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100af0 msgr2=0x7f5bb8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100af0 0x7f5bb8102070 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000b780 tx=0x7f5bb000bd60 comp rx=0 tx=0).stop 2023-09-12T16:03:17.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 -- 172.21.15.130:0/2920153238 shutdown_connections 2023-09-12T16:03:17.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ba40a0750 0x7f5ba40a2c10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8100af0 0x7f5bb8102070 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb80ff740 0x7f5bb81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 --2- 172.21.15.130:0/2920153238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bb8100100 0x7f5bb8101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 -- 172.21.15.130:0/2920153238 >> 172.21.15.130:0/2920153238 conn(0x7f5bb80fa1d0 msgr2=0x7f5bb80fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 -- 172.21.15.130:0/2920153238 shutdown_connections 2023-09-12T16:03:17.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.641+0000 7f5bc03db700 1 -- 172.21.15.130:0/2920153238 wait complete. 2023-09-12T16:03:17.669 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-12T16:03:17.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 -- 172.21.15.130:0/2524975115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01010b0 msgr2=0x7f74a01034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 --2- 172.21.15.130:0/2524975115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01010b0 0x7f74a01034a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7494009990 tx=0x7f7494009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 -- 172.21.15.130:0/2524975115 shutdown_connections 2023-09-12T16:03:17.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 --2- 172.21.15.130:0/2524975115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a0106310 0x7f74a0108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 --2- 172.21.15.130:0/2524975115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01010b0 0x7f74a01034a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 --2- 172.21.15.130:0/2524975115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a01039e0 0x7f74a0105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 -- 172.21.15.130:0/2524975115 >> 172.21.15.130:0/2524975115 conn(0x7f74a00faa80 msgr2=0x7f74a00fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 -- 172.21.15.130:0/2524975115 shutdown_connections 2023-09-12T16:03:17.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 -- 172.21.15.130:0/2524975115 wait complete. 2023-09-12T16:03:17.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.791+0000 7f74a5703700 1 Processor -- start 2023-09-12T16:03:17.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f74a5703700 1 -- start start 2023-09-12T16:03:17.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f74a5703700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a01010b0 0x7f74a0196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f74a5703700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01039e0 0x7f74a01967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f74a5703700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a0106310 0x7f74a019d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f74a5703700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a019ddf0 con 0x7f74a01039e0 2023-09-12T16:03:17.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f74a5703700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a019df30 con 0x7f74a01010b0 2023-09-12T16:03:17.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f74a5703700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a019e0c0 con 0x7f74a0106310 2023-09-12T16:03:17.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749e7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01039e0 0x7f74a01967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749e7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01039e0 0x7f74a01967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54546/0 (socket says 172.21.15.130:54546) 2023-09-12T16:03:17.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749e7fc700 1 -- 172.21.15.130:0/2912891613 learned_addr learned my addr 172.21.15.130:0/2912891613 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749f7fe700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a0106310 0x7f74a019d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749effd700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a01010b0 0x7f74a0196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749e7fc700 1 -- 172.21.15.130:0/2912891613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a0106310 msgr2=0x7f74a019d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749e7fc700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a0106310 0x7f74a019d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749e7fc700 1 -- 172.21.15.130:0/2912891613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a01010b0 msgr2=0x7f74a0196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749e7fc700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a01010b0 0x7f74a0196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749e7fc700 1 -- 172.21.15.130:0/2912891613 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940095f0 con 0x7f74a01039e0 2023-09-12T16:03:17.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.792+0000 7f749e7fc700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01039e0 0x7f74a01967b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f749000d950 tx=0x7f749000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.793+0000 7f7487fff700 1 -- 172.21.15.130:0/2912891613 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490009940 con 0x7f74a01039e0 2023-09-12T16:03:17.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.793+0000 7f7487fff700 1 -- 172.21.15.130:0/2912891613 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7490010460 con 0x7f74a01039e0 2023-09-12T16:03:17.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.793+0000 7f74a5703700 1 -- 172.21.15.130:0/2912891613 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a019e3a0 con 0x7f74a01039e0 2023-09-12T16:03:17.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.793+0000 7f7487fff700 1 -- 172.21.15.130:0/2912891613 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749000f620 con 0x7f74a01039e0 2023-09-12T16:03:17.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.793+0000 7f74a5703700 1 -- 172.21.15.130:0/2912891613 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a019e8b0 con 0x7f74a01039e0 2023-09-12T16:03:17.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.795+0000 7f7487fff700 1 -- 172.21.15.130:0/2912891613 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f74900108d0 con 0x7f74a01039e0 2023-09-12T16:03:17.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.795+0000 7f7487fff700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74880a06d0 0x7f74880a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.795+0000 7f749effd700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74880a06d0 0x7f74880a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.795+0000 7f7487fff700 1 -- 172.21.15.130:0/2912891613 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f74900cbe90 con 0x7f74a01039e0 2023-09-12T16:03:17.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f749effd700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74880a06d0 0x7f74880a2b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7494006010 tx=0x7f749400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f74b18c9b80 1 Processor -- start 2023-09-12T16:03:17.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f74b18c9b80 1 -- start start 2023-09-12T16:03:17.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 0x55c0273ebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273ea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273eb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0273436e0 con 0x55c0273eadb0 2023-09-12T16:03:17.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c027341000 con 0x55c0273ec550 2023-09-12T16:03:17.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c02733c8f0 con 0x55c0273ebbb0 2023-09-12T16:03:17.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f749f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f749f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273eb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54556/0 (socket says 172.21.15.130:54556) 2023-09-12T16:03:17.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f749f7fe700 1 -- 172.21.15.130:0/984891732 learned_addr learned my addr 172.21.15.130:0/984891732 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.796+0000 7f749effd700 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 0x55c0273ebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f749e7fc700 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273ea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f749f7fe700 1 -- 172.21.15.130:0/984891732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 msgr2=0x55c0273ebfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f749f7fe700 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 0x55c0273ebfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f749f7fe700 1 -- 172.21.15.130:0/984891732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 msgr2=0x55c0273ea7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f749f7fe700 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273ea7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f749f7fe700 1 -- 172.21.15.130:0/984891732 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c027168d90 con 0x55c0273eadb0 2023-09-12T16:03:17.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f749e7fc700 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273ea7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:17.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f749f7fe700 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273eb260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f749800f5a0 tx=0x7f749800f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17498107d846bdda server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f74857fa700 1 -- 172.21.15.130:0/984891732 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498014470 con 0x55c0273eadb0 2023-09-12T16:03:17.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f74857fa700 1 -- 172.21.15.130:0/984891732 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7498014a90 con 0x55c0273eadb0 2023-09-12T16:03:17.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.797+0000 7f74857fa700 1 -- 172.21.15.130:0/984891732 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74980055b0 con 0x55c0273eadb0 2023-09-12T16:03:17.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 -- 172.21.15.130:0/984891732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 msgr2=0x55c0273eb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273eb260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f749800f5a0 tx=0x7f749800f8b0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 -- 172.21.15.130:0/984891732 shutdown_connections 2023-09-12T16:03:17.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273eb260 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 0x55c0273ebfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/984891732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273ea7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 -- 172.21.15.130:0/984891732 >> 172.21.15.130:0/984891732 conn(0x55c02735f040 msgr2=0x55c0273e56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 -- 172.21.15.130:0/984891732 shutdown_connections 2023-09-12T16:03:17.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 -- 172.21.15.130:0/984891732 wait complete. 2023-09-12T16:03:17.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 -- start start 2023-09-12T16:03:17.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 Processor -- start 2023-09-12T16:03:17.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273f4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 0x55c0273f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273fba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0272dab20 con 0x55c0273eadb0 2023-09-12T16:03:17.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0272da820 con 0x55c0273ec550 2023-09-12T16:03:17.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0271be8d0 con 0x55c0273ebbb0 2023-09-12T16:03:17.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f749effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.798+0000 7f74b18c9b80 10 client.0 _ll_register_callbacks cb 0x55c027224a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:17.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273f4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54558/0 (socket says 172.21.15.130:54558) 2023-09-12T16:03:17.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749effd700 1 -- 172.21.15.130:0/3569579812 learned_addr learned my addr 172.21.15.130:0/3569579812 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749f7fe700 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273fba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749e7fc700 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 0x55c0273f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749effd700 1 -- 172.21.15.130:0/3569579812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 msgr2=0x55c0273f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749effd700 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 0x55c0273f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749effd700 1 -- 172.21.15.130:0/3569579812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 msgr2=0x55c0273fba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749effd700 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273fba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749effd700 1 -- 172.21.15.130:0/3569579812 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749800f1d0 con 0x55c0273eadb0 2023-09-12T16:03:17.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749f7fe700 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273fba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:17.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749e7fc700 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 0x55c0273f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:17.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f749effd700 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273f4510 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7494015000 tx=0x7f749401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4a60fe9d24a083 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f7482ff5700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:17.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f7482ff5700 1 -- 172.21.15.130:0/3569579812 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749402b050 con 0x55c0273eadb0 2023-09-12T16:03:17.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f7482ff5700 1 -- 172.21.15.130:0/3569579812 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7494024820 con 0x55c0273eadb0 2023-09-12T16:03:17.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3569579812 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0272944a0 con 0x55c0273eadb0 2023-09-12T16:03:17.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f7482ff5700 1 -- 172.21.15.130:0/3569579812 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74940249e0 con 0x55c0273eadb0 2023-09-12T16:03:17.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f74b18c9b80 10 client.15390 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:17.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3569579812 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c027151c10 con 0x55c0273eadb0 2023-09-12T16:03:17.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.799+0000 7f7482ff5700 1 -- 172.21.15.130:0/3569579812 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0272944a0 con 0x55c0273eadb0 2023-09-12T16:03:17.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f7482ff5700 1 -- 172.21.15.130:0/3569579812 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7494030c10 con 0x55c0273eadb0 2023-09-12T16:03:17.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f74b18c9b80 10 client.15390 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:17.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f74b18c9b80 10 client.15390 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-12T16:03:17.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f74b18c9b80 20 client.15390 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:17.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x55c0273ffc40 0x55c0273feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f74b18c9b80 4 client.15390 mds_command: new command op to 24592 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:03:17.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3569579812 --> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c027351850 con 0x55c0273ffc40 2023-09-12T16:03:17.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f749e7fc700 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x55c0273ffc40 0x55c0273feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f7482ff5700 10 client.15390 ms_handle_connect on v2:172.21.15.203:6838/2671073294 2023-09-12T16:03:17.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.800+0000 7f749e7fc700 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x55c0273ffc40 0x55c0273feb50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1d16c619103449 server_cookie=e6ef2706a21d5a65 in_seq=0 out_seq=0 2023-09-12T16:03:17.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f7482ff5700 1 -- 172.21.15.130:0/3569579812 <== mds.1 v2:172.21.15.203:6838/2671073294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c027351850 con 0x55c0273ffc40 2023-09-12T16:03:17.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f7482ff5700 10 client.15390 handle_command_reply: tid=0 2023-09-12T16:03:17.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 client.15390 shutdown 2023-09-12T16:03:17.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3569579812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 msgr2=0x55c0273f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273f4510 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7494015000 tx=0x7f749401a050 comp rx=0 tx=0).stop 2023-09-12T16:03:17.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3569579812 shutdown_connections 2023-09-12T16:03:17.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x55c0273ffc40 0x55c0273feb50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ec550 0x55c0273fba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f7482ff5700 0 client.15390 ms_handle_reset on v2:172.21.15.203:6838/2671073294 2023-09-12T16:03:17.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273eadb0 0x55c0273f4510 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3569579812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ebbb0 0x55c0273f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3569579812 >> 172.21.15.130:0/3569579812 conn(0x55c02735f040 msgr2=0x55c0273e4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3569579812 shutdown_connections 2023-09-12T16:03:17.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3569579812 wait complete. 2023-09-12T16:03:17.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.801+0000 7f74b18c9b80 20 client.15390 trim_cache size 0 max 16384 2023-09-12T16:03:17.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f74b18c9b80 1 Processor -- start 2023-09-12T16:03:17.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f74b18c9b80 1 -- start start 2023-09-12T16:03:17.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273eadb0 0x55c0273ea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ec550 0x55c0273f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c027305f70 con 0x55c0273ec550 2023-09-12T16:03:17.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0272f8ed0 con 0x55c0273ebbb0 2023-09-12T16:03:17.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0272d6e30 con 0x55c0273eadb0 2023-09-12T16:03:17.853 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ec550 0x55c0273f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.853 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ec550 0x55c0273f32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54570/0 (socket says 172.21.15.130:54570) 2023-09-12T16:03:17.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749f7fe700 1 -- 172.21.15.130:0/3035498640 learned_addr learned my addr 172.21.15.130:0/3035498640 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749effd700 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273eadb0 0x55c0273ea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749f7fe700 1 -- 172.21.15.130:0/3035498640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273eadb0 msgr2=0x55c0273ea6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749e7fc700 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749f7fe700 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273eadb0 0x55c0273ea6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749f7fe700 1 -- 172.21.15.130:0/3035498640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ebbb0 msgr2=0x55c0273fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749f7fe700 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749f7fe700 1 -- 172.21.15.130:0/3035498640 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c02718f200 con 0x55c0273ec550 2023-09-12T16:03:17.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749e7fc700 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:17.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f749f7fe700 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ec550 0x55c0273f32e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55c0273e0180 tx=0x7f7498007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82aa490ae89bce1b server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f7485ffb700 1 -- 172.21.15.130:0/3035498640 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498003a50 con 0x55c0273ec550 2023-09-12T16:03:17.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f7485ffb700 1 -- 172.21.15.130:0/3035498640 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7498003bb0 con 0x55c0273ec550 2023-09-12T16:03:17.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.819+0000 7f7485ffb700 1 -- 172.21.15.130:0/3035498640 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498005ad0 con 0x55c0273ec550 2023-09-12T16:03:17.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3035498640 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ec550 msgr2=0x55c0273f32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ec550 0x55c0273f32e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55c0273e0180 tx=0x7f7498007750 comp rx=0 tx=0).stop 2023-09-12T16:03:17.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3035498640 shutdown_connections 2023-09-12T16:03:17.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ec550 0x55c0273f32e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273eadb0 0x55c0273ea6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3035498640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3035498640 >> 172.21.15.130:0/3035498640 conn(0x55c02735f040 msgr2=0x55c0273e5680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3035498640 shutdown_connections 2023-09-12T16:03:17.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3035498640 wait complete. 2023-09-12T16:03:17.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 -- start start 2023-09-12T16:03:17.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 Processor -- start 2023-09-12T16:03:17.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273eadb0 0x55c0273f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.820+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f74b18c9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ec550 0x55c0273f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0273f1430 con 0x55c0273ebbb0 2023-09-12T16:03:17.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c027453c90 con 0x55c0273eadb0 2023-09-12T16:03:17.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f74b18c9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c027453e50 con 0x55c0273ec550 2023-09-12T16:03:17.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f74b18c9b80 10 client.0 _ll_register_callbacks cb 0x7f749400f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:17.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749e7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749e7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54574/0 (socket says 172.21.15.130:54574) 2023-09-12T16:03:17.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749e7fc700 1 -- 172.21.15.130:0/3983458911 learned_addr learned my addr 172.21.15.130:0/3983458911 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:17.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749effd700 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273eadb0 0x55c0273f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749f7fe700 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ec550 0x55c0273f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749e7fc700 1 -- 172.21.15.130:0/3983458911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ec550 msgr2=0x55c0273f0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749e7fc700 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ec550 0x55c0273f0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749e7fc700 1 -- 172.21.15.130:0/3983458911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273eadb0 msgr2=0x55c0273f07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:17.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749e7fc700 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273eadb0 0x55c0273f07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749e7fc700 1 -- 172.21.15.130:0/3983458911 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c027168d90 con 0x55c0273ebbb0 2023-09-12T16:03:17.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749f7fe700 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ec550 0x55c0273f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:17.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f749e7fc700 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7490010fd0 tx=0x7f749000d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3534c969e4543084 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:17.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f7482ff5700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:17.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f7482ff5700 1 -- 172.21.15.130:0/3983458911 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74900a1280 con 0x55c0273ebbb0 2023-09-12T16:03:17.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f7482ff5700 1 -- 172.21.15.130:0/3983458911 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7490099470 con 0x55c0273ebbb0 2023-09-12T16:03:17.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3983458911 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7498014a90 con 0x55c0273ebbb0 2023-09-12T16:03:17.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f7482ff5700 1 -- 172.21.15.130:0/3983458911 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7490099630 con 0x55c0273ebbb0 2023-09-12T16:03:17.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f74b18c9b80 10 client.15396 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:17.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.821+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3983458911 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c027167260 con 0x55c0273ebbb0 2023-09-12T16:03:17.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f7482ff5700 1 -- 172.21.15.130:0/3983458911 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7490063080 con 0x55c0273ebbb0 2023-09-12T16:03:17.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f7482ff5700 1 -- 172.21.15.130:0/3983458911 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74900a0df0 con 0x55c0273ebbb0 2023-09-12T16:03:17.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f74b18c9b80 10 client.15396 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:17.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f74b18c9b80 10 client.15396 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-12T16:03:17.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f74b18c9b80 20 client.15396 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:17.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x55c027459090 0x55c027455490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:17.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f74b18c9b80 4 client.15396 mds_command: new command op to 24592 tid=0[{"prefix": "status"}] 2023-09-12T16:03:17.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3983458911 --> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c0273358d0 con 0x55c027459090 2023-09-12T16:03:17.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f749effd700 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x55c027459090 0x55c027455490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:17.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f7482ff5700 10 client.15396 ms_handle_connect on v2:172.21.15.203:6838/2671073294 2023-09-12T16:03:17.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.822+0000 7f749effd700 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x55c027459090 0x55c027455490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=613c532c7a013ab3 server_cookie=4706e83690d452b5 in_seq=0 out_seq=0 2023-09-12T16:03:17.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f7482ff5700 1 -- 172.21.15.130:0/3983458911 <== mds.1 v2:172.21.15.203:6838/2671073294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c0273358d0 con 0x55c027459090 2023-09-12T16:03:17.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f7482ff5700 10 client.15396 handle_command_reply: tid=0 2023-09-12T16:03:17.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 client.15396 shutdown 2023-09-12T16:03:17.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3983458911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ebbb0 msgr2=0x55c0273fba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7490010fd0 tx=0x7f749000d8d0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3983458911 shutdown_connections 2023-09-12T16:03:17.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0273ec550 0x55c0273f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x55c027459090 0x55c027455490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0273eadb0 0x55c0273f07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 --2- 172.21.15.130:0/3983458911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c0273ebbb0 0x55c0273fba50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f7482ff5700 0 client.15396 ms_handle_reset on v2:172.21.15.203:6838/2671073294 2023-09-12T16:03:17.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3983458911 >> 172.21.15.130:0/3983458911 conn(0x55c02735f040 msgr2=0x55c0273eee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3983458911 shutdown_connections 2023-09-12T16:03:17.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 1 -- 172.21.15.130:0/3983458911 wait complete. 2023-09-12T16:03:17.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74b18c9b80 20 client.15396 trim_cache size 0 max 16384 2023-09-12T16:03:17.870 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:03:17.870 INFO:teuthology.orchestra.run.smithi130.stdout: "cluster_fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T16:03:17.870 INFO:teuthology.orchestra.run.smithi130.stdout: "whoami": 1, 2023-09-12T16:03:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 24592, 2023-09-12T16:03:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "want_state": "up:active", 2023-09-12T16:03:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "state": "up:active", 2023-09-12T16:03:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs", 2023-09-12T16:03:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "rank_uptime": 54.189688347999997, 2023-09-12T16:03:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "mdsmap_epoch": 24, 2023-09-12T16:03:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch": 88, 2023-09-12T16:03:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch_barrier": 86, 2023-09-12T16:03:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "uptime": 84.749978507999998 2023-09-12T16:03:17.873 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:03:17.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 -- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74880a06d0 msgr2=0x7f74880a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74880a06d0 0x7f74880a2b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7494006010 tx=0x7f749400b540 comp rx=0 tx=0).stop 2023-09-12T16:03:17.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 -- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01039e0 msgr2=0x7f74a01967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:17.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01039e0 0x7f74a01967b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f749000d950 tx=0x7f749000dc60 comp rx=0 tx=0).stop 2023-09-12T16:03:17.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 -- 172.21.15.130:0/2912891613 shutdown_connections 2023-09-12T16:03:17.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74880a06d0 0x7f74880a2b90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a0106310 0x7f74a019d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a01010b0 0x7f74a0196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 --2- 172.21.15.130:0/2912891613 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74a01039e0 0x7f74a01967b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:17.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 -- 172.21.15.130:0/2912891613 >> 172.21.15.130:0/2912891613 conn(0x7f74a00faa80 msgr2=0x7f74a0101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:17.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.823+0000 7f74a5703700 1 -- 172.21.15.130:0/2912891613 shutdown_connections 2023-09-12T16:03:17.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:17.824+0000 7f74a5703700 1 -- 172.21.15.130:0/2912891613 wait complete. 2023-09-12T16:03:17.879 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-12T16:03:18.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.001+0000 7f54a677f700 1 -- 172.21.15.130:0/220420039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a0102210 msgr2=0x7f54a0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:18.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.001+0000 7f54a677f700 1 --2- 172.21.15.130:0/220420039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a0102210 0x7f54a0108ac0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f549800b0a0 tx=0x7f549800b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.001+0000 7f54a677f700 1 -- 172.21.15.130:0/220420039 shutdown_connections 2023-09-12T16:03:18.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.001+0000 7f54a677f700 1 --2- 172.21.15.130:0/220420039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a0102210 0x7f54a0108ac0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.001+0000 7f54a677f700 1 --2- 172.21.15.130:0/220420039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a00fe8a0 0x7f54a00fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.001+0000 7f54a677f700 1 --2- 172.21.15.130:0/220420039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54a0101870 0x7f54a0101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.001+0000 7f54a677f700 1 -- 172.21.15.130:0/220420039 >> 172.21.15.130:0/220420039 conn(0x7f54a00fa1d0 msgr2=0x7f54a00fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:18.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.001+0000 7f54a677f700 1 -- 172.21.15.130:0/220420039 shutdown_connections 2023-09-12T16:03:18.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.001+0000 7f54a677f700 1 -- 172.21.15.130:0/220420039 wait complete. 2023-09-12T16:03:18.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f54a677f700 1 Processor -- start 2023-09-12T16:03:18.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f54a677f700 1 -- start start 2023-09-12T16:03:18.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f54a677f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a00fe8a0 0x7f54a01963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f54a677f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a0101870 0x7f54a0196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f54a677f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54a0102210 0x7f54a019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f54a677f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a019df60 con 0x7f54a00fe8a0 2023-09-12T16:03:18.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f54a677f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a019e0a0 con 0x7f54a0101870 2023-09-12T16:03:18.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f54a677f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a019e230 con 0x7f54a0102210 2023-09-12T16:03:18.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a00fe8a0 0x7f54a01963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a00fe8a0 0x7f54a01963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54586/0 (socket says 172.21.15.130:54586) 2023-09-12T16:03:18.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549ffff700 1 -- 172.21.15.130:0/3836240931 learned_addr learned my addr 172.21.15.130:0/3836240931 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:18.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f54a4d1c700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54a0102210 0x7f54a019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549f7fe700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a0101870 0x7f54a0196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549ffff700 1 -- 172.21.15.130:0/3836240931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54a0102210 msgr2=0x7f54a019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:18.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549ffff700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54a0102210 0x7f54a019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549ffff700 1 -- 172.21.15.130:0/3836240931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a0101870 msgr2=0x7f54a0196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:18.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549ffff700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a0101870 0x7f54a0196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549ffff700 1 -- 172.21.15.130:0/3836240931 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498009d00 con 0x7f54a00fe8a0 2023-09-12T16:03:18.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.002+0000 7f549ffff700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a00fe8a0 0x7f54a01963e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f549000d980 tx=0x7f549000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:18.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.003+0000 7f549d7fa700 1 -- 172.21.15.130:0/3836240931 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549000bb60 con 0x7f54a00fe8a0 2023-09-12T16:03:18.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.003+0000 7f54a677f700 1 -- 172.21.15.130:0/3836240931 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a019e510 con 0x7f54a00fe8a0 2023-09-12T16:03:18.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.003+0000 7f549d7fa700 1 -- 172.21.15.130:0/3836240931 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5490007620 con 0x7f54a00fe8a0 2023-09-12T16:03:18.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.003+0000 7f549d7fa700 1 -- 172.21.15.130:0/3836240931 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549000f620 con 0x7f54a00fe8a0 2023-09-12T16:03:18.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.003+0000 7f54a677f700 1 -- 172.21.15.130:0/3836240931 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a019ea70 con 0x7f54a00fe8a0 2023-09-12T16:03:18.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.005+0000 7f549d7fa700 1 -- 172.21.15.130:0/3836240931 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f5490010460 con 0x7f54a00fe8a0 2023-09-12T16:03:18.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.005+0000 7f549d7fa700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54880a06d0 0x7f54880a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.006+0000 7f549f7fe700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54880a06d0 0x7f54880a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.006+0000 7f549d7fa700 1 -- 172.21.15.130:0/3836240931 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f54900cc060 con 0x7f54a00fe8a0 2023-09-12T16:03:18.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.006+0000 7f549f7fe700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54880a06d0 0x7f54880a2b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5494009bd0 tx=0x7f5494009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:18.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.006+0000 7f54b2945b80 1 Processor -- start 2023-09-12T16:03:18.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.006+0000 7f54b2945b80 1 -- start start 2023-09-12T16:03:18.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4020 0x55a3167f4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 0x55a3167f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167f3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54b2945b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3167476e0 con 0x55a3167f31e0 2023-09-12T16:03:18.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54b2945b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a316745000 con 0x55a3167f4980 2023-09-12T16:03:18.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54b2945b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3167408f0 con 0x55a3167f4020 2023-09-12T16:03:18.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54a4d1c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54a4d1c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54598/0 (socket says 172.21.15.130:54598) 2023-09-12T16:03:18.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54a4d1c700 1 -- 172.21.15.130:0/3824839623 learned_addr learned my addr 172.21.15.130:0/3824839623 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:18.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f549ffff700 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4020 0x55a3167f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f549f7fe700 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 0x55a3167f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54a4d1c700 1 -- 172.21.15.130:0/3824839623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4020 msgr2=0x55a3167f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:18.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54a4d1c700 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4020 0x55a3167f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54a4d1c700 1 -- 172.21.15.130:0/3824839623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 msgr2=0x55a3167f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:18.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54a4d1c700 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 0x55a3167f2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54a4d1c700 1 -- 172.21.15.130:0/3824839623 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a31656cd90 con 0x55a3167f31e0 2023-09-12T16:03:18.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f549f7fe700 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 0x55a3167f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:18.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54a4d1c700 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167f3660 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f549800b780 tx=0x7f5498009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d7cea69523a5e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:18.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54867fc700 1 -- 172.21.15.130:0/3824839623 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549800e050 con 0x55a3167f31e0 2023-09-12T16:03:18.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54867fc700 1 -- 172.21.15.130:0/3824839623 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5498009460 con 0x55a3167f31e0 2023-09-12T16:03:18.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.007+0000 7f54867fc700 1 -- 172.21.15.130:0/3824839623 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54980052a0 con 0x55a3167f31e0 2023-09-12T16:03:18.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 -- 172.21.15.130:0/3824839623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 msgr2=0x55a3167f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:18.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167f3660 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f549800b780 tx=0x7f5498009c60 comp rx=0 tx=0).stop 2023-09-12T16:03:18.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 -- 172.21.15.130:0/3824839623 shutdown_connections 2023-09-12T16:03:18.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167f3660 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4020 0x55a3167f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3824839623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 0x55a3167f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 -- 172.21.15.130:0/3824839623 >> 172.21.15.130:0/3824839623 conn(0x55a316763040 msgr2=0x55a3167e96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:18.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 -- 172.21.15.130:0/3824839623 shutdown_connections 2023-09-12T16:03:18.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 -- 172.21.15.130:0/3824839623 wait complete. 2023-09-12T16:03:18.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.008+0000 7f54b2945b80 1 -- start start 2023-09-12T16:03:18.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f54b2945b80 1 Processor -- start 2023-09-12T16:03:18.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167fce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 0x55a316803e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f54b2945b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3166deb20 con 0x55a3167f31e0 2023-09-12T16:03:18.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f54b2945b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3166de820 con 0x55a3167f4020 2023-09-12T16:03:18.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f54b2945b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3165c28d0 con 0x55a3167f4980 2023-09-12T16:03:18.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f54b2945b80 10 client.0 _ll_register_callbacks cb 0x55a316628a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:18.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167fc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54600/0 (socket says 172.21.15.130:54600) 2023-09-12T16:03:18.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549ffff700 1 -- 172.21.15.130:0/1318513204 learned_addr learned my addr 172.21.15.130:0/1318513204 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:18.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f54a4d1c700 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 0x55a316803e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549f7fe700 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549ffff700 1 -- 172.21.15.130:0/1318513204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 msgr2=0x55a316803e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:18.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549ffff700 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 0x55a316803e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549ffff700 1 -- 172.21.15.130:0/1318513204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 msgr2=0x55a3167fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:18.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549ffff700 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549ffff700 1 -- 172.21.15.130:0/1318513204 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a01093c0 con 0x55a3167f31e0 2023-09-12T16:03:18.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549f7fe700 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:18.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f549ffff700 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167fc8d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5490010a70 tx=0x7f5490095850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f646a077564909 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:18.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f547f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:18.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.009+0000 7f547f7fe700 1 -- 172.21.15.130:0/1318513204 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900840b0 con 0x55a3167f31e0 2023-09-12T16:03:18.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f547f7fe700 1 -- 172.21.15.130:0/1318513204 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f54900993e0 con 0x55a3167f31e0 2023-09-12T16:03:18.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f54b2945b80 1 -- 172.21.15.130:0/1318513204 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3166984a0 con 0x55a3167f31e0 2023-09-12T16:03:18.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f547f7fe700 1 -- 172.21.15.130:0/1318513204 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f549009d210 con 0x55a3167f31e0 2023-09-12T16:03:18.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f54b2945b80 10 client.15408 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:18.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f54b2945b80 1 -- 172.21.15.130:0/1318513204 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a316555c10 con 0x55a3167f31e0 2023-09-12T16:03:18.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f547f7fe700 1 -- 172.21.15.130:0/1318513204 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3166984a0 con 0x55a3167f31e0 2023-09-12T16:03:18.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f547f7fe700 1 -- 172.21.15.130:0/1318513204 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f54900b7270 con 0x55a3167f31e0 2023-09-12T16:03:18.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f54b2945b80 10 client.15408 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:18.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f54b2945b80 10 client.15408 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-12T16:03:18.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f54b2945b80 20 client.15408 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:18.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f54b2945b80 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a316807f30 0x55a316806ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f54b2945b80 4 client.15408 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:03:18.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f54b2945b80 1 -- 172.21.15.130:0/1318513204 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a316755850 con 0x55a316807f30 2023-09-12T16:03:18.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.010+0000 7f549f7fe700 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a316807f30 0x55a316806ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.011+0000 7f547f7fe700 10 client.15408 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:18.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.011+0000 7f549f7fe700 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a316807f30 0x55a316806ec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac1a8dffd659c9ba server_cookie=2bdc3c83e0a1ba72 in_seq=0 out_seq=0 2023-09-12T16:03:18.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.011+0000 7f547f7fe700 1 -- 172.21.15.130:0/1318513204 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a316755850 con 0x55a316807f30 2023-09-12T16:03:18.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.011+0000 7f547f7fe700 10 client.15408 handle_command_reply: tid=0 2023-09-12T16:03:18.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.011+0000 7f54b2945b80 1 client.15408 shutdown 2023-09-12T16:03:18.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 -- 172.21.15.130:0/1318513204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 msgr2=0x55a3167fc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:18.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167fc8d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5490010a70 tx=0x7f5490095850 comp rx=0 tx=0).stop 2023-09-12T16:03:18.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 -- 172.21.15.130:0/1318513204 shutdown_connections 2023-09-12T16:03:18.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a316807f30 0x55a316806ec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 0x55a316803e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167fc8d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 --2- 172.21.15.130:0/1318513204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f547f7fe700 0 client.15408 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:18.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 -- 172.21.15.130:0/1318513204 >> 172.21.15.130:0/1318513204 conn(0x55a316763040 msgr2=0x55a3167e8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:18.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 -- 172.21.15.130:0/1318513204 shutdown_connections 2023-09-12T16:03:18.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 1 -- 172.21.15.130:0/1318513204 wait complete. 2023-09-12T16:03:18.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.012+0000 7f54b2945b80 20 client.15408 trim_cache size 0 max 16384 2023-09-12T16:03:18.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f54b2945b80 1 Processor -- start 2023-09-12T16:03:18.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f54b2945b80 1 -- start start 2023-09-12T16:03:18.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f31e0 0x55a3167f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f4020 0x55a316803e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 0x55a316806ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f54b2945b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a316709f70 con 0x55a3167f4020 2023-09-12T16:03:18.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f54b2945b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3166fced0 con 0x55a3167f4980 2023-09-12T16:03:18.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f54b2945b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3167fbd80 con 0x55a3167f31e0 2023-09-12T16:03:18.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f4020 0x55a316803e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f4020 0x55a316803e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54610/0 (socket says 172.21.15.130:54610) 2023-09-12T16:03:18.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f549f7fe700 1 -- 172.21.15.130:0/2276353303 learned_addr learned my addr 172.21.15.130:0/2276353303 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:18.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.029+0000 7f549ffff700 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f31e0 0x55a3167f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f54a4d1c700 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 0x55a316806ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f549f7fe700 1 -- 172.21.15.130:0/2276353303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f31e0 msgr2=0x55a3167f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:18.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f549f7fe700 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f31e0 0x55a3167f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f549f7fe700 1 -- 172.21.15.130:0/2276353303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 msgr2=0x55a316806ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:18.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f549f7fe700 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 0x55a316806ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f549f7fe700 1 -- 172.21.15.130:0/2276353303 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a316593200 con 0x55a3167f4020 2023-09-12T16:03:18.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f549ffff700 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f31e0 0x55a3167f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:18.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f549f7fe700 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f4020 0x55a316803e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5494010760 tx=0x7f5494013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c0d8526c720f8c server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:18.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f5486ffd700 1 -- 172.21.15.130:0/2276353303 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494026450 con 0x55a3167f4020 2023-09-12T16:03:18.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f5486ffd700 1 -- 172.21.15.130:0/2276353303 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5494025040 con 0x55a3167f4020 2023-09-12T16:03:18.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f5486ffd700 1 -- 172.21.15.130:0/2276353303 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549401eb20 con 0x55a3167f4020 2023-09-12T16:03:18.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f54b2945b80 1 -- 172.21.15.130:0/2276353303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f4020 msgr2=0x55a316803e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:18.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f54b2945b80 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f4020 0x55a316803e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5494010760 tx=0x7f5494013000 comp rx=0 tx=0).stop 2023-09-12T16:03:18.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f54b2945b80 1 -- 172.21.15.130:0/2276353303 shutdown_connections 2023-09-12T16:03:18.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f54b2945b80 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4980 0x55a316806ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f54b2945b80 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f31e0 0x55a3167f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f54b2945b80 1 --2- 172.21.15.130:0/2276353303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f4020 0x55a316803e10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.030+0000 7f54b2945b80 1 -- 172.21.15.130:0/2276353303 >> 172.21.15.130:0/2276353303 conn(0x55a316763040 msgr2=0x55a3167fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:18.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 -- 172.21.15.130:0/2276353303 shutdown_connections 2023-09-12T16:03:18.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 -- 172.21.15.130:0/2276353303 wait complete. 2023-09-12T16:03:18.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 -- start start 2023-09-12T16:03:18.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 Processor -- start 2023-09-12T16:03:18.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167feba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 0x55a31685bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3167ff620 con 0x55a3167f31e0 2023-09-12T16:03:18.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3167ff760 con 0x55a3167f4020 2023-09-12T16:03:18.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31685c4c0 con 0x55a3167f4980 2023-09-12T16:03:18.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f54b2945b80 10 client.0 _ll_register_callbacks cb 0x7f5490098df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:18.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f549ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f549ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167feba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54614/0 (socket says 172.21.15.130:54614) 2023-09-12T16:03:18.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.031+0000 7f549ffff700 1 -- 172.21.15.130:0/3551521006 learned_addr learned my addr 172.21.15.130:0/3551521006 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:18.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54a4d1c700 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 0x55a31685bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f549f7fe700 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f549ffff700 1 -- 172.21.15.130:0/3551521006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 msgr2=0x55a31685bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:18.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f549ffff700 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 0x55a31685bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f549ffff700 1 -- 172.21.15.130:0/3551521006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 msgr2=0x55a3167ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:18.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f549ffff700 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f549ffff700 1 -- 172.21.15.130:0/3551521006 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494010b80 con 0x55a3167f31e0 2023-09-12T16:03:18.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f549f7fe700 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:18.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54a4d1c700 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 0x55a31685bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:18.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f549ffff700 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167feba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f5490022ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01ec80b69471bcb server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:18.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f547f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:18.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f547f7fe700 1 -- 172.21.15.130:0/3551521006 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900a11c0 con 0x55a3167f31e0 2023-09-12T16:03:18.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f547f7fe700 1 -- 172.21.15.130:0/3551521006 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f54900b7020 con 0x55a3167f31e0 2023-09-12T16:03:18.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54b2945b80 1 -- 172.21.15.130:0/3551521006 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5498009460 con 0x55a3167f31e0 2023-09-12T16:03:18.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f547f7fe700 1 -- 172.21.15.130:0/3551521006 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5490099800 con 0x55a3167f31e0 2023-09-12T16:03:18.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54b2945b80 10 client.15414 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:18.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54b2945b80 1 -- 172.21.15.130:0/3551521006 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a31656b260 con 0x55a3167f31e0 2023-09-12T16:03:18.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f547f7fe700 1 -- 172.21.15.130:0/3551521006 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5498009460 con 0x55a3167f31e0 2023-09-12T16:03:18.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f547f7fe700 1 -- 172.21.15.130:0/3551521006 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f54900988d0 con 0x55a3167f31e0 2023-09-12T16:03:18.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54b2945b80 10 client.15414 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:18.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54b2945b80 10 client.15414 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-12T16:03:18.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54b2945b80 20 client.15414 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:18.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a316861180 0x55a31685d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:18.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54b2945b80 4 client.15414 mds_command: new command op to 24593 tid=0[{"prefix": "status"}] 2023-09-12T16:03:18.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.032+0000 7f54b2945b80 1 -- 172.21.15.130:0/3551521006 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a3167398d0 con 0x55a316861180 2023-09-12T16:03:18.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.033+0000 7f549f7fe700 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a316861180 0x55a31685d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:18.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.033+0000 7f547f7fe700 10 client.15414 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:18.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.033+0000 7f549f7fe700 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a316861180 0x55a31685d650 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae6f3e74d920acf0 server_cookie=aa8fc6345ffee4d2 in_seq=0 out_seq=0 2023-09-12T16:03:18.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.033+0000 7f547f7fe700 1 -- 172.21.15.130:0/3551521006 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a3167398d0 con 0x55a316861180 2023-09-12T16:03:18.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.033+0000 7f547f7fe700 10 client.15414 handle_command_reply: tid=0 2023-09-12T16:03:18.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.033+0000 7f54b2945b80 1 client.15414 shutdown 2023-09-12T16:03:18.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 -- 172.21.15.130:0/3551521006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 msgr2=0x55a3167feba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:18.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167feba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f5490022ff0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 -- 172.21.15.130:0/3551521006 shutdown_connections 2023-09-12T16:03:18.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a316861180 0x55a31685d650 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3167f4980 0x55a31685bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3167f31e0 0x55a3167feba0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f547f7fe700 0 client.15414 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:18.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 --2- 172.21.15.130:0/3551521006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3167f4020 0x55a3167ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 -- 172.21.15.130:0/3551521006 >> 172.21.15.130:0/3551521006 conn(0x55a316763040 msgr2=0x55a3167f35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:18.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 -- 172.21.15.130:0/3551521006 shutdown_connections 2023-09-12T16:03:18.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 1 -- 172.21.15.130:0/3551521006 wait complete. 2023-09-12T16:03:18.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54b2945b80 20 client.15414 trim_cache size 0 max 16384 2023-09-12T16:03:18.057 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:03:18.057 INFO:teuthology.orchestra.run.smithi130.stdout: "cluster_fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T16:03:18.057 INFO:teuthology.orchestra.run.smithi130.stdout: "whoami": 0, 2023-09-12T16:03:18.057 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 24593, 2023-09-12T16:03:18.057 INFO:teuthology.orchestra.run.smithi130.stdout: "want_state": "up:active", 2023-09-12T16:03:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "state": "up:active", 2023-09-12T16:03:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs", 2023-09-12T16:03:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "rank_uptime": 73.008967636999998, 2023-09-12T16:03:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "mdsmap_epoch": 24, 2023-09-12T16:03:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch": 88, 2023-09-12T16:03:18.059 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch_barrier": 86, 2023-09-12T16:03:18.059 INFO:teuthology.orchestra.run.smithi130.stdout: "uptime": 78.478333000999996 2023-09-12T16:03:18.059 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:03:18.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 -- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54880a06d0 msgr2=0x7f54880a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:18.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54880a06d0 0x7f54880a2b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5494009bd0 tx=0x7f5494009260 comp rx=0 tx=0).stop 2023-09-12T16:03:18.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 -- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a00fe8a0 msgr2=0x7f54a01963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:18.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a00fe8a0 0x7f54a01963e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f549000d980 tx=0x7f549000dd40 comp rx=0 tx=0).stop 2023-09-12T16:03:18.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 -- 172.21.15.130:0/3836240931 shutdown_connections 2023-09-12T16:03:18.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f54880a06d0 0x7f54880a2b90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54a0102210 0x7f54a019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a00fe8a0 0x7f54a01963e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 --2- 172.21.15.130:0/3836240931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a0101870 0x7f54a0196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:18.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 -- 172.21.15.130:0/3836240931 >> 172.21.15.130:0/3836240931 conn(0x7f54a00fa1d0 msgr2=0x7f54a00fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:18.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 -- 172.21.15.130:0/3836240931 shutdown_connections 2023-09-12T16:03:18.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:18.034+0000 7f54a677f700 1 -- 172.21.15.130:0/3836240931 wait complete. 2023-09-12T16:03:18.063 INFO:tasks.cephadm:Setup complete, yielding 2023-09-12T16:03:18.064 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-12T16:03:18.074 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi130.front.sepia.ceph.com 2023-09-12T16:03:18.074 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch status' 2023-09-12T16:03:18.098 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:18 smithi130 ceph-mon[98670]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-12T16:03:18.098 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:18 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4271345652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:18.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:18 smithi159 ceph-mon[102522]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-12T16:03:18.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:18 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4271345652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:18.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:18 smithi203 ceph-mon[103431]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-12T16:03:18.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:18 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4271345652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:18.627 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:19.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.851+0000 7f78f718c700 1 -- 172.21.15.130:0/2855980458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0101890 msgr2=0x7f78f0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:19.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.851+0000 7f78f718c700 1 --2- 172.21.15.130:0/2855980458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0101890 0x7f78f0101cf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0099c0 tx=0x7f78dc009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:19.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.851+0000 7f78f718c700 1 -- 172.21.15.130:0/2855980458 shutdown_connections 2023-09-12T16:03:19.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.851+0000 7f78f718c700 1 --2- 172.21.15.130:0/2855980458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f0102230 0x7f78f0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:19.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.851+0000 7f78f718c700 1 --2- 172.21.15.130:0/2855980458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f0100690 0x7f78f0100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:19.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.851+0000 7f78f718c700 1 --2- 172.21.15.130:0/2855980458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0101890 0x7f78f0101cf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:19.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.851+0000 7f78f718c700 1 -- 172.21.15.130:0/2855980458 >> 172.21.15.130:0/2855980458 conn(0x7f78f00fbbf0 msgr2=0x7f78f00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:19.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.851+0000 7f78f718c700 1 -- 172.21.15.130:0/2855980458 shutdown_connections 2023-09-12T16:03:19.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.851+0000 7f78f718c700 1 -- 172.21.15.130:0/2855980458 wait complete. 2023-09-12T16:03:19.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f718c700 1 Processor -- start 2023-09-12T16:03:19.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f718c700 1 -- start start 2023-09-12T16:03:19.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f718c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f0100690 0x7f78f00743a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:19.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f718c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f0101890 0x7f78f00729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:19.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f718c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0102230 0x7f78f0072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:19.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f718c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00736b0 con 0x7f78f0102230 2023-09-12T16:03:19.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f718c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f0073820 con 0x7f78f0100690 2023-09-12T16:03:19.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f718c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f01a0320 con 0x7f78f0101890 2023-09-12T16:03:19.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f5729700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0102230 0x7f78f0072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:19.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f5729700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0102230 0x7f78f0072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54632/0 (socket says 172.21.15.130:54632) 2023-09-12T16:03:19.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f5729700 1 -- 172.21.15.130:0/1546817931 learned_addr learned my addr 172.21.15.130:0/1546817931 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:19.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f4f28700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f0100690 0x7f78f00743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:19.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78effff700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f0101890 0x7f78f00729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:19.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f5729700 1 -- 172.21.15.130:0/1546817931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f0101890 msgr2=0x7f78f00729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:19.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f5729700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f0101890 0x7f78f00729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:19.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f5729700 1 -- 172.21.15.130:0/1546817931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f0100690 msgr2=0x7f78f00743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:19.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f5729700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f0100690 0x7f78f00743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:19.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.852+0000 7f78f5729700 1 -- 172.21.15.130:0/1546817931 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0095f0 con 0x7f78f0102230 2023-09-12T16:03:19.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.853+0000 7f78f5729700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0102230 0x7f78f0072f30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f78e8007d70 tx=0x7f78e800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:19.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.853+0000 7f78edffb700 1 -- 172.21.15.130:0/1546817931 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e800ac90 con 0x7f78f0102230 2023-09-12T16:03:19.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.853+0000 7f78edffb700 1 -- 172.21.15.130:0/1546817931 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e800adf0 con 0x7f78f0102230 2023-09-12T16:03:19.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.853+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f01a0600 con 0x7f78f0102230 2023-09-12T16:03:19.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.853+0000 7f78edffb700 1 -- 172.21.15.130:0/1546817931 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e80115e0 con 0x7f78f0102230 2023-09-12T16:03:19.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.853+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f01a0ab0 con 0x7f78f0102230 2023-09-12T16:03:19.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.854+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f010be90 con 0x7f78f0102230 2023-09-12T16:03:19.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.855+0000 7f78edffb700 1 -- 172.21.15.130:0/1546817931 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f78e8012420 con 0x7f78f0102230 2023-09-12T16:03:19.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.855+0000 7f78edffb700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e00a06d0 0x7f78e00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:19.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.856+0000 7f78edffb700 1 -- 172.21.15.130:0/1546817931 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f78e80cc190 con 0x7f78f0102230 2023-09-12T16:03:19.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.856+0000 7f78f4f28700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e00a06d0 0x7f78e00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:19.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.856+0000 7f78f4f28700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e00a06d0 0x7f78e00a2b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f78f01016f0 tx=0x7f78e400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:19.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:19.860+0000 7f78edffb700 1 -- 172.21.15.130:0/1546817931 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f78e8095900 con 0x7f78f0102230 2023-09-12T16:03:20.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.066+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78f0060a40 con 0x7f78e00a06d0 2023-09-12T16:03:20.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.067+0000 7f78edffb700 1 -- 172.21.15.130:0/1546817931 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f78f0060a40 con 0x7f78e00a06d0 2023-09-12T16:03:20.068 INFO:teuthology.orchestra.run.smithi130.stdout:Backend: cephadm 2023-09-12T16:03:20.068 INFO:teuthology.orchestra.run.smithi130.stdout:Available: Yes 2023-09-12T16:03:20.068 INFO:teuthology.orchestra.run.smithi130.stdout:Paused: No 2023-09-12T16:03:20.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.069+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e00a06d0 msgr2=0x7f78e00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:20.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.069+0000 7f78f718c700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e00a06d0 0x7f78e00a2b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f78f01016f0 tx=0x7f78e400b410 comp rx=0 tx=0).stop 2023-09-12T16:03:20.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.069+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0102230 msgr2=0x7f78f0072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:20.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.069+0000 7f78f718c700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0102230 0x7f78f0072f30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f78e8007d70 tx=0x7f78e800a340 comp rx=0 tx=0).stop 2023-09-12T16:03:20.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.069+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 shutdown_connections 2023-09-12T16:03:20.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.070+0000 7f78f718c700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e00a06d0 0x7f78e00a2b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:20.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.070+0000 7f78f718c700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78f0102230 0x7f78f0072f30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:20.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.070+0000 7f78f718c700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f0100690 0x7f78f00743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:20.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.070+0000 7f78f718c700 1 --2- 172.21.15.130:0/1546817931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f0101890 0x7f78f00729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:20.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.070+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 >> 172.21.15.130:0/1546817931 conn(0x7f78f00fbbf0 msgr2=0x7f78f00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:20.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.070+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 shutdown_connections 2023-09-12T16:03:20.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:20.070+0000 7f78f718c700 1 -- 172.21.15.130:0/1546817931 wait complete. 2023-09-12T16:03:20.383 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:20 smithi130 ceph-mon[98670]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:03:20.384 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch ps' 2023-09-12T16:03:20.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:20 smithi159 ceph-mon[102522]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:03:20.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:20 smithi203 ceph-mon[103431]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:03:20.986 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:22.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.176+0000 7f5e43142700 1 -- 172.21.15.130:0/3478401558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c100790 msgr2=0x7f5e3c100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:22.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.176+0000 7f5e43142700 1 --2- 172.21.15.130:0/3478401558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c100790 0x7f5e3c100bf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e280099c0 tx=0x7f5e28009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.176+0000 7f5e43142700 1 -- 172.21.15.130:0/3478401558 shutdown_connections 2023-09-12T16:03:22.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.176+0000 7f5e43142700 1 --2- 172.21.15.130:0/3478401558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e3c101130 0x7f5e3c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.176+0000 7f5e43142700 1 --2- 172.21.15.130:0/3478401558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c0ff590 0x7f5e3c0ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.176+0000 7f5e43142700 1 --2- 172.21.15.130:0/3478401558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c100790 0x7f5e3c100bf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.176+0000 7f5e43142700 1 -- 172.21.15.130:0/3478401558 >> 172.21.15.130:0/3478401558 conn(0x7f5e3c0fab10 msgr2=0x7f5e3c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:22.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.176+0000 7f5e43142700 1 -- 172.21.15.130:0/3478401558 shutdown_connections 2023-09-12T16:03:22.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.176+0000 7f5e43142700 1 -- 172.21.15.130:0/3478401558 wait complete. 2023-09-12T16:03:22.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e43142700 1 Processor -- start 2023-09-12T16:03:22.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e43142700 1 -- start start 2023-09-12T16:03:22.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e43142700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c0ff590 0x7f5e3c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:22.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e43142700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e3c100790 0x7f5e3c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:22.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e43142700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c101130 0x7f5e3c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:22.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e43142700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c19dfb0 con 0x7f5e3c0ff590 2023-09-12T16:03:22.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e43142700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c19e0f0 con 0x7f5e3c100790 2023-09-12T16:03:22.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e43142700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c19e280 con 0x7f5e3c101130 2023-09-12T16:03:22.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e40ede700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c0ff590 0x7f5e3c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:22.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e40ede700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c0ff590 0x7f5e3c196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54640/0 (socket says 172.21.15.130:54640) 2023-09-12T16:03:22.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e40ede700 1 -- 172.21.15.130:0/1384347455 learned_addr learned my addr 172.21.15.130:0/1384347455 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:22.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e3bfff700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e3c100790 0x7f5e3c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:22.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e416df700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c101130 0x7f5e3c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:22.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e40ede700 1 -- 172.21.15.130:0/1384347455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c101130 msgr2=0x7f5e3c19d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:22.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e40ede700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c101130 0x7f5e3c19d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e40ede700 1 -- 172.21.15.130:0/1384347455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e3c100790 msgr2=0x7f5e3c196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:22.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e40ede700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e3c100790 0x7f5e3c196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e40ede700 1 -- 172.21.15.130:0/1384347455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e280095f0 con 0x7f5e3c0ff590 2023-09-12T16:03:22.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.177+0000 7f5e40ede700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c0ff590 0x7f5e3c196420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b530 tx=0x7f5e3000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:22.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.178+0000 7f5e39ffb700 1 -- 172.21.15.130:0/1384347455 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30011830 con 0x7f5e3c0ff590 2023-09-12T16:03:22.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.178+0000 7f5e39ffb700 1 -- 172.21.15.130:0/1384347455 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e30011e50 con 0x7f5e3c0ff590 2023-09-12T16:03:22.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.178+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c19e560 con 0x7f5e3c0ff590 2023-09-12T16:03:22.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.178+0000 7f5e39ffb700 1 -- 172.21.15.130:0/1384347455 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000f2e0 con 0x7f5e3c0ff590 2023-09-12T16:03:22.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.178+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c19eb30 con 0x7f5e3c0ff590 2023-09-12T16:03:22.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.179+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c10aeb0 con 0x7f5e3c0ff590 2023-09-12T16:03:22.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.180+0000 7f5e39ffb700 1 -- 172.21.15.130:0/1384347455 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f5e30010580 con 0x7f5e3c0ff590 2023-09-12T16:03:22.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.180+0000 7f5e39ffb700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5e2c0a0750 0x7f5e2c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:22.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.180+0000 7f5e3bfff700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5e2c0a0750 0x7f5e2c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:22.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.180+0000 7f5e39ffb700 1 -- 172.21.15.130:0/1384347455 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5e300cb900 con 0x7f5e3c0ff590 2023-09-12T16:03:22.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.181+0000 7f5e3bfff700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5e2c0a0750 0x7f5e2c0a2c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800b580 tx=0x7f5e28000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:22.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.185+0000 7f5e39ffb700 1 -- 172.21.15.130:0/1384347455 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5e300950d0 con 0x7f5e3c0ff590 2023-09-12T16:03:22.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.383+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e3c068830 con 0x7f5e2c0a0750 2023-09-12T16:03:22.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.395+0000 7f5e39ffb700 1 -- 172.21.15.130:0/1384347455 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f5e3c068830 con 0x7f5e2c0a0750 2023-09-12T16:03:22.396 INFO:teuthology.orchestra.run.smithi130.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-09-12T16:03:22.396 INFO:teuthology.orchestra.run.smithi130.stdout:mds.a smithi130 running (85s) - 85s 16.4M - 18.0.0-6088-g2110e007 bf15e540d611 5885aca20228 2023-09-12T16:03:22.396 INFO:teuthology.orchestra.run.smithi130.stdout:mds.b smithi159 running (83s) - 82s 31.0M - 18.0.0-6088-g2110e007 bf15e540d611 fa58e30589ac 2023-09-12T16:03:22.397 INFO:teuthology.orchestra.run.smithi130.stdout:mds.c smithi203 running (94s) - 94s 20.9M - 18.0.0-6088-g2110e007 bf15e540d611 f6a60204aa53 2023-09-12T16:03:22.397 INFO:teuthology.orchestra.run.smithi130.stdout:mds.d smithi130 running (101s) - 101s 15.2M - 18.0.0-6088-g2110e007 bf15e540d611 594c30dd43bd 2023-09-12T16:03:22.397 INFO:teuthology.orchestra.run.smithi130.stdout:mds.e smithi159 running (99s) - 98s 17.7M - 18.0.0-6088-g2110e007 bf15e540d611 04fc5a2baef3 2023-09-12T16:03:22.397 INFO:teuthology.orchestra.run.smithi130.stdout:mds.f smithi203 running (96s) - 96s 18.5M - 18.0.0-6088-g2110e007 bf15e540d611 35eb579dcf1e 2023-09-12T16:03:22.397 INFO:teuthology.orchestra.run.smithi130.stdout:mds.g smithi130 running (87s) - 87s 17.2M - 18.0.0-6088-g2110e007 bf15e540d611 6c022b3a6f99 2023-09-12T16:03:22.397 INFO:teuthology.orchestra.run.smithi130.stdout:mds.h smithi159 running (91s) - 91s 20.0M - 18.0.0-6088-g2110e007 bf15e540d611 c9ccabea5aef 2023-09-12T16:03:22.398 INFO:teuthology.orchestra.run.smithi130.stdout:mds.i smithi203 running (89s) - 89s 21.7M - 18.0.0-6088-g2110e007 bf15e540d611 22af2c5b0e14 2023-09-12T16:03:22.398 INFO:teuthology.orchestra.run.smithi130.stdout:mds.j smithi130 running (103s) - 103s 17.5M - 18.0.0-6088-g2110e007 bf15e540d611 5a7e5e931527 2023-09-12T16:03:22.398 INFO:teuthology.orchestra.run.smithi130.stdout:mds.k smithi159 running (105s) - 105s 14.2M - 18.0.0-6088-g2110e007 bf15e540d611 f741ecea79c8 2023-09-12T16:03:22.398 INFO:teuthology.orchestra.run.smithi130.stdout:mds.l smithi203 running (107s) - 107s 14.2M - 18.0.0-6088-g2110e007 bf15e540d611 234b96c09b89 2023-09-12T16:03:22.398 INFO:teuthology.orchestra.run.smithi130.stdout:mgr.x smithi130 *:9283,8765 running (8m) - 8m 522M - 18.0.0-6088-g2110e007 bf15e540d611 7d7d2256e25c 2023-09-12T16:03:22.399 INFO:teuthology.orchestra.run.smithi130.stdout:mgr.y smithi159 *:8443,8765 running (6m) - 6m 452M - 18.0.0-6088-g2110e007 bf15e540d611 0d23bafa3256 2023-09-12T16:03:22.399 INFO:teuthology.orchestra.run.smithi130.stdout:mgr.z smithi203 *:8443,8765 running (6m) - 6m 452M - 18.0.0-6088-g2110e007 bf15e540d611 5df2b073fd91 2023-09-12T16:03:22.399 INFO:teuthology.orchestra.run.smithi130.stdout:mon.a smithi130 running (8m) - 8m 96.9M 2048M 18.0.0-6088-g2110e007 bf15e540d611 2e2780516902 2023-09-12T16:03:22.399 INFO:teuthology.orchestra.run.smithi130.stdout:mon.b smithi159 running (6m) - 6m 61.2M 2048M 18.0.0-6088-g2110e007 bf15e540d611 9a218cd014a6 2023-09-12T16:03:22.399 INFO:teuthology.orchestra.run.smithi130.stdout:mon.c smithi203 running (6m) - 6m 63.1M 2048M 18.0.0-6088-g2110e007 bf15e540d611 db561d8c3723 2023-09-12T16:03:22.400 INFO:teuthology.orchestra.run.smithi130.stdout:osd.0 smithi130 running (5m) - 5m 70.7M 4292M 18.0.0-6088-g2110e007 bf15e540d611 f98358043400 2023-09-12T16:03:22.400 INFO:teuthology.orchestra.run.smithi130.stdout:osd.1 smithi159 running (5m) - 5m 77.1M 4292M 18.0.0-6088-g2110e007 bf15e540d611 7ae9c6fc751b 2023-09-12T16:03:22.400 INFO:teuthology.orchestra.run.smithi130.stdout:osd.2 smithi203 running (5m) - 5m 73.7M 4292M 18.0.0-6088-g2110e007 bf15e540d611 a19f3d9eec0c 2023-09-12T16:03:22.400 INFO:teuthology.orchestra.run.smithi130.stdout:osd.3 smithi130 running (4m) - 4m 66.3M 4292M 18.0.0-6088-g2110e007 bf15e540d611 7591d86bda48 2023-09-12T16:03:22.400 INFO:teuthology.orchestra.run.smithi130.stdout:osd.4 smithi159 running (4m) - 4m 64.5M 4292M 18.0.0-6088-g2110e007 bf15e540d611 43ceb77e5fc8 2023-09-12T16:03:22.401 INFO:teuthology.orchestra.run.smithi130.stdout:osd.5 smithi203 running (4m) - 4m 68.3M 4292M 18.0.0-6088-g2110e007 bf15e540d611 48a5743f1225 2023-09-12T16:03:22.401 INFO:teuthology.orchestra.run.smithi130.stdout:osd.6 smithi130 running (3m) - 3m 64.4M 4292M 18.0.0-6088-g2110e007 bf15e540d611 b72e3f4575dd 2023-09-12T16:03:22.401 INFO:teuthology.orchestra.run.smithi130.stdout:osd.7 smithi159 running (3m) - 3m 65.8M 4292M 18.0.0-6088-g2110e007 bf15e540d611 abde56cb63f0 2023-09-12T16:03:22.401 INFO:teuthology.orchestra.run.smithi130.stdout:osd.8 smithi203 running (3m) - 3m 64.2M 4292M 18.0.0-6088-g2110e007 bf15e540d611 a08b87c32752 2023-09-12T16:03:22.401 INFO:teuthology.orchestra.run.smithi130.stdout:osd.9 smithi130 running (2m) - 2m 61.4M 4292M 18.0.0-6088-g2110e007 bf15e540d611 3e5062da318c 2023-09-12T16:03:22.402 INFO:teuthology.orchestra.run.smithi130.stdout:osd.10 smithi159 running (2m) - 2m 61.8M 4292M 18.0.0-6088-g2110e007 bf15e540d611 722a8ace4e88 2023-09-12T16:03:22.402 INFO:teuthology.orchestra.run.smithi130.stdout:osd.11 smithi203 running (2m) - 2m 55.2M 4292M 18.0.0-6088-g2110e007 bf15e540d611 f57f2ef228ea 2023-09-12T16:03:22.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.397+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5e2c0a0750 msgr2=0x7f5e2c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:22.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.397+0000 7f5e43142700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5e2c0a0750 0x7f5e2c0a2c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800b580 tx=0x7f5e28000bc0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.397+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c0ff590 msgr2=0x7f5e3c196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:22.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.397+0000 7f5e43142700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c0ff590 0x7f5e3c196420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b530 tx=0x7f5e3000b840 comp rx=0 tx=0).stop 2023-09-12T16:03:22.403 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.398+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 shutdown_connections 2023-09-12T16:03:22.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.398+0000 7f5e43142700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5e2c0a0750 0x7f5e2c0a2c10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.398+0000 7f5e43142700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c101130 0x7f5e3c19d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.398+0000 7f5e43142700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e3c0ff590 0x7f5e3c196420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.398+0000 7f5e43142700 1 --2- 172.21.15.130:0/1384347455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e3c100790 0x7f5e3c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:22.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.398+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 >> 172.21.15.130:0/1384347455 conn(0x7f5e3c0fab10 msgr2=0x7f5e3c068120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:22.404 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.398+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 shutdown_connections 2023-09-12T16:03:22.405 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:22.398+0000 7f5e43142700 1 -- 172.21.15.130:0/1384347455 wait complete. 2023-09-12T16:03:22.421 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:22 smithi130 ceph-mon[98670]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:22.421 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:22 smithi130 ceph-mon[98670]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:22.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:22 smithi159 ceph-mon[102522]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:22.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:22 smithi159 ceph-mon[102522]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:22.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:22 smithi203 ceph-mon[103431]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:22.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:22 smithi203 ceph-mon[103431]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:22.749 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch ls' 2023-09-12T16:03:23.309 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:23.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:23 smithi130 ceph-mon[98670]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:23.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:23 smithi130 ceph-mon[98670]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:23.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:23 smithi159 ceph-mon[102522]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:23.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:23 smithi159 ceph-mon[102522]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:23.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:23 smithi203 ceph-mon[103431]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:23.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:23 smithi203 ceph-mon[103431]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:24.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.452+0000 7f0197625700 1 -- 172.21.15.130:0/1194112309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01900ff7b0 msgr2=0x7f01900ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:24.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.452+0000 7f0197625700 1 --2- 172.21.15.130:0/1194112309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01900ff7b0 0x7f01900ffc30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f018c009990 tx=0x7f018c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.452+0000 7f0197625700 1 -- 172.21.15.130:0/1194112309 shutdown_connections 2023-09-12T16:03:24.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.452+0000 7f0197625700 1 --2- 172.21.15.130:0/1194112309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0190100170 0x7f0190107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.452+0000 7f0197625700 1 --2- 172.21.15.130:0/1194112309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0190100be0 0x7f0190101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.452+0000 7f0197625700 1 --2- 172.21.15.130:0/1194112309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01900ff7b0 0x7f01900ffc30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.452+0000 7f0197625700 1 -- 172.21.15.130:0/1194112309 >> 172.21.15.130:0/1194112309 conn(0x7f01900fa280 msgr2=0x7f01900fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:24.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.452+0000 7f0197625700 1 -- 172.21.15.130:0/1194112309 shutdown_connections 2023-09-12T16:03:24.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.452+0000 7f0197625700 1 -- 172.21.15.130:0/1194112309 wait complete. 2023-09-12T16:03:24.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0197625700 1 Processor -- start 2023-09-12T16:03:24.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0197625700 1 -- start start 2023-09-12T16:03:24.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0197625700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900ff7b0 0x7f0190103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:24.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0197625700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0190100170 0x7f0190101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:24.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0197625700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0190100be0 0x7f01901020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:24.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0197625700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190102860 con 0x7f0190100be0 2023-09-12T16:03:24.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0197625700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01901029d0 con 0x7f0190100170 2023-09-12T16:03:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0197625700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01901a0360 con 0x7f01900ff7b0 2023-09-12T16:03:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0195bc2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0190100be0 0x7f01901020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0195bc2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0190100be0 0x7f01901020e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54658/0 (socket says 172.21.15.130:54658) 2023-09-12T16:03:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0195bc2700 1 -- 172.21.15.130:0/777304022 learned_addr learned my addr 172.21.15.130:0/777304022 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0194bc0700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0190100170 0x7f0190101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f01953c1700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900ff7b0 0x7f0190103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0195bc2700 1 -- 172.21.15.130:0/777304022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900ff7b0 msgr2=0x7f0190103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.453+0000 7f0195bc2700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900ff7b0 0x7f0190103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.454+0000 7f0195bc2700 1 -- 172.21.15.130:0/777304022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0190100170 msgr2=0x7f0190101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.454+0000 7f0195bc2700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0190100170 0x7f0190101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.454+0000 7f0195bc2700 1 -- 172.21.15.130:0/777304022 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c0095f0 con 0x7f0190100be0 2023-09-12T16:03:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.454+0000 7f0195bc2700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0190100be0 0x7f01901020e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0188007ee0 tx=0x7f018800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.454+0000 7f01827fc700 1 -- 172.21.15.130:0/777304022 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188017070 con 0x7f0190100be0 2023-09-12T16:03:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.454+0000 7f01827fc700 1 -- 172.21.15.130:0/777304022 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f018800f040 con 0x7f0190100be0 2023-09-12T16:03:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.454+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01901a0640 con 0x7f0190100be0 2023-09-12T16:03:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.454+0000 7f01827fc700 1 -- 172.21.15.130:0/777304022 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188005740 con 0x7f0190100be0 2023-09-12T16:03:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.454+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01901a0af0 con 0x7f0190100be0 2023-09-12T16:03:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.455+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019010ade0 con 0x7f0190100be0 2023-09-12T16:03:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.456+0000 7f01827fc700 1 -- 172.21.15.130:0/777304022 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f018800dc60 con 0x7f0190100be0 2023-09-12T16:03:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.456+0000 7f01827fc700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f017c0a06d0 0x7f017c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.456+0000 7f01953c1700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f017c0a06d0 0x7f017c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.456+0000 7f01827fc700 1 -- 172.21.15.130:0/777304022 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f01880cbed0 con 0x7f0190100be0 2023-09-12T16:03:24.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.457+0000 7f01953c1700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f017c0a06d0 0x7f017c0a2b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0184005950 tx=0x7f01840058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:24.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.460+0000 7f01827fc700 1 -- 172.21.15.130:0/777304022 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0188095590 con 0x7f0190100be0 2023-09-12T16:03:24.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.657+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f01901a0d10 con 0x7f017c0a06d0 2023-09-12T16:03:24.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.662+0000 7f01827fc700 1 -- 172.21.15.130:0/777304022 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f01901a0d10 con 0x7f017c0a06d0 2023-09-12T16:03:24.663 INFO:teuthology.orchestra.run.smithi130.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-09-12T16:03:24.663 INFO:teuthology.orchestra.run.smithi130.stdout:mds.all 12/12 - 111s smithi130=a;smithi130=d;smithi130=g;smithi130=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2023-09-12T16:03:24.663 INFO:teuthology.orchestra.run.smithi130.stdout:mgr 3/3 - 6m smithi130=x;smithi159=y;smithi203=z;count:3 2023-09-12T16:03:24.664 INFO:teuthology.orchestra.run.smithi130.stdout:mon 3/3 - 7m smithi130:172.21.15.130=a;smithi159:172.21.15.159=b;smithi203:172.21.15.203=c;count:3 2023-09-12T16:03:24.664 INFO:teuthology.orchestra.run.smithi130.stdout:osd 12 - - 2023-09-12T16:03:24.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.664+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f017c0a06d0 msgr2=0x7f017c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:24.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.664+0000 7f0197625700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f017c0a06d0 0x7f017c0a2b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0184005950 tx=0x7f01840058e0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.664+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0190100be0 msgr2=0x7f01901020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:24.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.664+0000 7f0197625700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0190100be0 0x7f01901020e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0188007ee0 tx=0x7f018800d3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.665+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 shutdown_connections 2023-09-12T16:03:24.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.665+0000 7f0197625700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f017c0a06d0 0x7f017c0a2b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.665+0000 7f0197625700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0190100be0 0x7f01901020e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.665+0000 7f0197625700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900ff7b0 0x7f0190103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.665+0000 7f0197625700 1 --2- 172.21.15.130:0/777304022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0190100170 0x7f0190101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:24.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.665+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 >> 172.21.15.130:0/777304022 conn(0x7f01900fa280 msgr2=0x7f01900fab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:24.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.665+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 shutdown_connections 2023-09-12T16:03:24.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:24.665+0000 7f0197625700 1 -- 172.21.15.130:0/777304022 wait complete. 2023-09-12T16:03:25.070 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch host ls' 2023-09-12T16:03:25.625 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:26.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:25 smithi130 ceph-mon[98670]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:26.158 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:25 smithi130 ceph-mon[98670]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:26.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:25 smithi159 ceph-mon[102522]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:25 smithi159 ceph-mon[102522]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:25 smithi203 ceph-mon[103431]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:25 smithi203 ceph-mon[103431]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:26.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.848+0000 7f1f2eec1700 1 -- 172.21.15.130:0/4022600520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f28100790 msgr2=0x7f1f28100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:26.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.848+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/4022600520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f28100790 0x7f1f28100bf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1f140099c0 tx=0x7f1f14009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:26.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.848+0000 7f1f2eec1700 1 -- 172.21.15.130:0/4022600520 shutdown_connections 2023-09-12T16:03:26.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.848+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/4022600520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f28101130 0x7f1f281079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:26.850 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.848+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/4022600520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f280ff590 0x7f1f280ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:26.850 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.848+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/4022600520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f28100790 0x7f1f28100bf0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:26.850 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.848+0000 7f1f2eec1700 1 -- 172.21.15.130:0/4022600520 >> 172.21.15.130:0/4022600520 conn(0x7f1f280fab10 msgr2=0x7f1f280fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:26.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.848+0000 7f1f2eec1700 1 -- 172.21.15.130:0/4022600520 shutdown_connections 2023-09-12T16:03:26.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.848+0000 7f1f2eec1700 1 -- 172.21.15.130:0/4022600520 wait complete. 2023-09-12T16:03:26.853 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2eec1700 1 Processor -- start 2023-09-12T16:03:26.853 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2eec1700 1 -- start start 2023-09-12T16:03:26.853 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2eec1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f280ff590 0x7f1f28198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:26.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2eec1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f28100790 0x7f1f28198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:26.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2eec1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f28101130 0x7f1f2819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:26.854 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2eec1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f281a00b0 con 0x7f1f280ff590 2023-09-12T16:03:26.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2eec1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f281a01f0 con 0x7f1f28100790 2023-09-12T16:03:26.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2eec1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f281a0380 con 0x7f1f28101130 2023-09-12T16:03:26.855 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2cc5d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f280ff590 0x7f1f28198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:26.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2cc5d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f280ff590 0x7f1f28198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54668/0 (socket says 172.21.15.130:54668) 2023-09-12T16:03:26.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2cc5d700 1 -- 172.21.15.130:0/1427753781 learned_addr learned my addr 172.21.15.130:0/1427753781 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:26.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2d45e700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f28101130 0x7f1f2819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:26.856 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f27fff700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f28100790 0x7f1f28198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:26.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2cc5d700 1 -- 172.21.15.130:0/1427753781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f28101130 msgr2=0x7f1f2819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:26.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2cc5d700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f28101130 0x7f1f2819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:26.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2cc5d700 1 -- 172.21.15.130:0/1427753781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f28100790 msgr2=0x7f1f28198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:26.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2cc5d700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f28100790 0x7f1f28198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:26.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.849+0000 7f1f2cc5d700 1 -- 172.21.15.130:0/1427753781 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f140095f0 con 0x7f1f280ff590 2023-09-12T16:03:26.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.850+0000 7f1f2cc5d700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f280ff590 0x7f1f28198520 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00e9a0 tx=0x7f1f1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:26.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.850+0000 7f1f25ffb700 1 -- 172.21.15.130:0/1427753781 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c009790 con 0x7f1f280ff590 2023-09-12T16:03:26.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.850+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f281a0660 con 0x7f1f280ff590 2023-09-12T16:03:26.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.850+0000 7f1f25ffb700 1 -- 172.21.15.130:0/1427753781 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f1c0044e0 con 0x7f1f280ff590 2023-09-12T16:03:26.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.850+0000 7f1f25ffb700 1 -- 172.21.15.130:0/1427753781 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c003d20 con 0x7f1f280ff590 2023-09-12T16:03:26.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.850+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f281a0b60 con 0x7f1f280ff590 2023-09-12T16:03:26.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.851+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f281054c0 con 0x7f1f280ff590 2023-09-12T16:03:26.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.852+0000 7f1f25ffb700 1 -- 172.21.15.130:0/1427753781 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f1f1c003e80 con 0x7f1f280ff590 2023-09-12T16:03:26.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.853+0000 7f1f25ffb700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f180a06d0 0x7f1f180a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:26.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.853+0000 7f1f27fff700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f180a06d0 0x7f1f180a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:26.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.853+0000 7f1f25ffb700 1 -- 172.21.15.130:0/1427753781 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1f1c014070 con 0x7f1f280ff590 2023-09-12T16:03:26.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.853+0000 7f1f27fff700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f180a06d0 0x7f1f180a2b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400b580 tx=0x7f1f1401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:26.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:26.856+0000 7f1f25ffb700 1 -- 172.21.15.130:0/1427753781 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1f1c095a60 con 0x7f1f280ff590 2023-09-12T16:03:27.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.051+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f2802cf80 con 0x7f1f180a06d0 2023-09-12T16:03:27.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.053+0000 7f1f25ffb700 1 -- 172.21.15.130:0/1427753781 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f1f2802cf80 con 0x7f1f180a06d0 2023-09-12T16:03:27.053 INFO:teuthology.orchestra.run.smithi130.stdout:HOST ADDR LABELS STATUS 2023-09-12T16:03:27.053 INFO:teuthology.orchestra.run.smithi130.stdout:smithi130 172.21.15.130 2023-09-12T16:03:27.054 INFO:teuthology.orchestra.run.smithi130.stdout:smithi159 172.21.15.159 2023-09-12T16:03:27.054 INFO:teuthology.orchestra.run.smithi130.stdout:smithi203 172.21.15.203 2023-09-12T16:03:27.054 INFO:teuthology.orchestra.run.smithi130.stdout:3 hosts in cluster 2023-09-12T16:03:27.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f180a06d0 msgr2=0x7f1f180a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:27.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f180a06d0 0x7f1f180a2b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400b580 tx=0x7f1f1401a040 comp rx=0 tx=0).stop 2023-09-12T16:03:27.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f280ff590 msgr2=0x7f1f28198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:27.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f280ff590 0x7f1f28198520 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00e9a0 tx=0x7f1f1c00ecb0 comp rx=0 tx=0).stop 2023-09-12T16:03:27.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 shutdown_connections 2023-09-12T16:03:27.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f180a06d0 0x7f1f180a2b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:27.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f28101130 0x7f1f2819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:27.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f280ff590 0x7f1f28198520 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:27.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 --2- 172.21.15.130:0/1427753781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f28100790 0x7f1f28198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:27.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 >> 172.21.15.130:0/1427753781 conn(0x7f1f280fab10 msgr2=0x7f1f280fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:27.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 shutdown_connections 2023-09-12T16:03:27.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:27.055+0000 7f1f2eec1700 1 -- 172.21.15.130:0/1427753781 wait complete. 2023-09-12T16:03:27.480 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch device ls' 2023-09-12T16:03:27.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:27 smithi130 ceph-mon[98670]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-12T16:03:27.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:27 smithi130 ceph-mon[98670]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:27.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:27 smithi159 ceph-mon[102522]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-12T16:03:27.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:27 smithi159 ceph-mon[102522]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:27 smithi203 ceph-mon[103431]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-12T16:03:27.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:27 smithi203 ceph-mon[103431]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:28.034 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:29.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.263+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3744509113 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc1018c0 msgr2=0x7f50dc101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:29.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.263+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3744509113 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc1018c0 0x7f50dc101d20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f50d00099c0 tx=0x7f50d0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.263+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3744509113 shutdown_connections 2023-09-12T16:03:29.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.263+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3744509113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50dc102260 0x7f50dc108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.263+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3744509113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc1006c0 0x7f50dc100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.263+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3744509113 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc1018c0 0x7f50dc101d20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.263+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3744509113 >> 172.21.15.130:0/3744509113 conn(0x7f50dc0fbc80 msgr2=0x7f50dc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:29.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.263+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3744509113 shutdown_connections 2023-09-12T16:03:29.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.263+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3744509113 wait complete. 2023-09-12T16:03:29.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50e0dcd700 1 Processor -- start 2023-09-12T16:03:29.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50e0dcd700 1 -- start start 2023-09-12T16:03:29.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50e0dcd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc1006c0 0x7f50dc196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:29.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50e0dcd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50dc1018c0 0x7f50dc196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:29.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50e0dcd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc102260 0x7f50dc19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:29.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50e0dcd700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc19dec0 con 0x7f50dc102260 2023-09-12T16:03:29.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50e0dcd700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc19e000 con 0x7f50dc1006c0 2023-09-12T16:03:29.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50e0dcd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc19e190 con 0x7f50dc1018c0 2023-09-12T16:03:29.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50dad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc102260 0x7f50dc19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:29.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50dad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc102260 0x7f50dc19d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54684/0 (socket says 172.21.15.130:54684) 2023-09-12T16:03:29.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50dad9d700 1 -- 172.21.15.130:0/3127859864 learned_addr learned my addr 172.21.15.130:0/3127859864 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:29.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50da59c700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc1006c0 0x7f50dc196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:29.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50d9d9b700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50dc1018c0 0x7f50dc196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:29.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50dad9d700 1 -- 172.21.15.130:0/3127859864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50dc1018c0 msgr2=0x7f50dc196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:29.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50dad9d700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50dc1018c0 0x7f50dc196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50dad9d700 1 -- 172.21.15.130:0/3127859864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc1006c0 msgr2=0x7f50dc196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:29.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50dad9d700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc1006c0 0x7f50dc196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50dad9d700 1 -- 172.21.15.130:0/3127859864 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00095f0 con 0x7f50dc102260 2023-09-12T16:03:29.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.264+0000 7f50dad9d700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc102260 0x7f50dc19d8f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f50d400add0 tx=0x7f50d400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:29.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.265+0000 7f50cb7fe700 1 -- 172.21.15.130:0/3127859864 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d400f070 con 0x7f50dc102260 2023-09-12T16:03:29.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.265+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc19e470 con 0x7f50dc102260 2023-09-12T16:03:29.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.265+0000 7f50cb7fe700 1 -- 172.21.15.130:0/3127859864 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d4007b30 con 0x7f50dc102260 2023-09-12T16:03:29.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.265+0000 7f50cb7fe700 1 -- 172.21.15.130:0/3127859864 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4005280 con 0x7f50dc102260 2023-09-12T16:03:29.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.265+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc19e970 con 0x7f50dc102260 2023-09-12T16:03:29.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.266+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50dc107c90 con 0x7f50dc102260 2023-09-12T16:03:29.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.267+0000 7f50cb7fe700 1 -- 172.21.15.130:0/3127859864 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f50d400c890 con 0x7f50dc102260 2023-09-12T16:03:29.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.268+0000 7f50cb7fe700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50c40a06d0 0x7f50c40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:29.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.268+0000 7f50da59c700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50c40a06d0 0x7f50c40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:29.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.268+0000 7f50cb7fe700 1 -- 172.21.15.130:0/3127859864 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50d40cbb10 con 0x7f50dc102260 2023-09-12T16:03:29.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.268+0000 7f50da59c700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50c40a06d0 0x7f50c40a2b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f50dc101720 tx=0x7f50cc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:29.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.271+0000 7f50cb7fe700 1 -- 172.21.15.130:0/3127859864 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f50d4095260 con 0x7f50dc102260 2023-09-12T16:03:29.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.469+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f50dc02cfa0 con 0x7f50c40a06d0 2023-09-12T16:03:29.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.471+0000 7f50cb7fe700 1 -- 172.21.15.130:0/3127859864 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f50dc02cfa0 con 0x7f50c40a06d0 2023-09-12T16:03:29.472 INFO:teuthology.orchestra.run.smithi130.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-09-12T16:03:29.472 INFO:teuthology.orchestra.run.smithi130.stdout:smithi130 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500Q1400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-12T16:03:29.472 INFO:teuthology.orchestra.run.smithi130.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-12T16:03:29.472 INFO:teuthology.orchestra.run.smithi130.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-12T16:03:29.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50c40a06d0 msgr2=0x7f50c40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:29.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50c40a06d0 0x7f50c40a2b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f50dc101720 tx=0x7f50cc00a380 comp rx=0 tx=0).stop 2023-09-12T16:03:29.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc102260 msgr2=0x7f50dc19d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:29.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc102260 0x7f50dc19d8f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f50d400add0 tx=0x7f50d400c330 comp rx=0 tx=0).stop 2023-09-12T16:03:29.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 shutdown_connections 2023-09-12T16:03:29.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50c40a06d0 0x7f50c40a2b90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50dc102260 0x7f50dc19d8f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc1006c0 0x7f50dc196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 --2- 172.21.15.130:0/3127859864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50dc1018c0 0x7f50dc196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 >> 172.21.15.130:0/3127859864 conn(0x7f50dc0fbc80 msgr2=0x7f50dc104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 shutdown_connections 2023-09-12T16:03:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:29.473+0000 7f50e0dcd700 1 -- 172.21.15.130:0/3127859864 wait complete. 2023-09-12T16:03:29.798 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-12T16:03:29.807 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi130.front.sepia.ceph.com 2023-09-12T16:03:29.807 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph fs dump' 2023-09-12T16:03:30.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:29 smithi130 ceph-mon[98670]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:03:30.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:29 smithi159 ceph-mon[102522]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:03:30.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:29 smithi203 ceph-mon[103431]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:03:30.364 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:31.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:30 smithi130 ceph-mon[98670]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:31.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:30 smithi159 ceph-mon[102522]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:31.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:30 smithi203 ceph-mon[103431]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-12T16:03:31.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.542+0000 7f8e4b881700 1 -- 172.21.15.130:0/465042114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 msgr2=0x7f8e44103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:31.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.542+0000 7f8e4b881700 1 --2- 172.21.15.130:0/465042114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 0x7f8e44103510 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009990 tx=0x7f8e40009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.542+0000 7f8e4b881700 1 -- 172.21.15.130:0/465042114 shutdown_connections 2023-09-12T16:03:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.542+0000 7f8e4b881700 1 --2- 172.21.15.130:0/465042114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44106380 0x7f8e441087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.542+0000 7f8e4b881700 1 --2- 172.21.15.130:0/465042114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 0x7f8e44103510 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.542+0000 7f8e4b881700 1 --2- 172.21.15.130:0/465042114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44103a50 0x7f8e44105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.542+0000 7f8e4b881700 1 -- 172.21.15.130:0/465042114 >> 172.21.15.130:0/465042114 conn(0x7f8e440fab10 msgr2=0x7f8e440fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:31.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.542+0000 7f8e4b881700 1 -- 172.21.15.130:0/465042114 shutdown_connections 2023-09-12T16:03:31.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.542+0000 7f8e4b881700 1 -- 172.21.15.130:0/465042114 wait complete. 2023-09-12T16:03:31.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.543+0000 7f8e4b881700 1 Processor -- start 2023-09-12T16:03:31.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.543+0000 7f8e4b881700 1 -- start start 2023-09-12T16:03:31.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.543+0000 7f8e4b881700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 0x7f8e441963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:31.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.543+0000 7f8e4b881700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44103a50 0x7f8e44196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:31.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.543+0000 7f8e4b881700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106380 0x7f8e4419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:31.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4b881700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4419df50 con 0x7f8e44101120 2023-09-12T16:03:31.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4b881700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4419e090 con 0x7f8e44106380 2023-09-12T16:03:31.549 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4b881700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4419e220 con 0x7f8e44103a50 2023-09-12T16:03:31.549 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4961d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 0x7f8e441963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:31.549 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4961d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 0x7f8e441963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54692/0 (socket says 172.21.15.130:54692) 2023-09-12T16:03:31.549 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4961d700 1 -- 172.21.15.130:0/3108248492 learned_addr learned my addr 172.21.15.130:0/3108248492 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:31.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e48e1c700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44103a50 0x7f8e44196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:31.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4961d700 1 -- 172.21.15.130:0/3108248492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44103a50 msgr2=0x7f8e44196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:31.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4961d700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44103a50 0x7f8e44196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4961d700 1 -- 172.21.15.130:0/3108248492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106380 msgr2=0x7f8e4419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:31.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4961d700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106380 0x7f8e4419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4961d700 1 -- 172.21.15.130:0/3108248492 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e400095f0 con 0x7f8e44101120 2023-09-12T16:03:31.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e48e1c700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44103a50 0x7f8e44196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:31.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4961d700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 0x7f8e441963c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005910 tx=0x7f8e4000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:31.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e3a7fc700 1 -- 172.21.15.130:0/3108248492 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4001d070 con 0x7f8e44101120 2023-09-12T16:03:31.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e3a7fc700 1 -- 172.21.15.130:0/3108248492 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e4000fbc0 con 0x7f8e44101120 2023-09-12T16:03:31.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4419e4a0 con 0x7f8e44101120 2023-09-12T16:03:31.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e3a7fc700 1 -- 172.21.15.130:0/3108248492 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40017650 con 0x7f8e44101120 2023-09-12T16:03:31.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.544+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4419e990 con 0x7f8e44101120 2023-09-12T16:03:31.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.545+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4410bf20 con 0x7f8e44101120 2023-09-12T16:03:31.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.546+0000 7f8e3a7fc700 1 -- 172.21.15.130:0/3108248492 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f8e40022470 con 0x7f8e44101120 2023-09-12T16:03:31.553 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.547+0000 7f8e3a7fc700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8e300a0750 0x7f8e300a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:31.553 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.547+0000 7f8e48e1c700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8e300a0750 0x7f8e300a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:31.553 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.547+0000 7f8e3a7fc700 1 -- 172.21.15.130:0/3108248492 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8e400cd6a0 con 0x7f8e44101120 2023-09-12T16:03:31.554 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.547+0000 7f8e48e1c700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8e300a0750 0x7f8e300a2c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8e340099d0 tx=0x7f8e34008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:31.554 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.550+0000 7f8e3a7fc700 1 -- 172.21.15.130:0/3108248492 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8e40096d90 con 0x7f8e44101120 2023-09-12T16:03:31.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.780+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8e4410c160 con 0x7f8e44101120 2023-09-12T16:03:31.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.781+0000 7f8e3a7fc700 1 -- 172.21.15.130:0/3108248492 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2958 (secure 0 0 0) 0x7f8e40027ba0 con 0x7f8e44101120 2023-09-12T16:03:31.782 INFO:teuthology.orchestra.run.smithi130.stdout:e24 2023-09-12T16:03:31.782 INFO:teuthology.orchestra.run.smithi130.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-09-12T16:03:31.783 INFO:teuthology.orchestra.run.smithi130.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-12T16:03:31.783 INFO:teuthology.orchestra.run.smithi130.stdout:legacy client fscid: 1 2023-09-12T16:03:31.783 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:31.783 INFO:teuthology.orchestra.run.smithi130.stdout:Filesystem 'cephfs' (1) 2023-09-12T16:03:31.784 INFO:teuthology.orchestra.run.smithi130.stdout:fs_name cephfs 2023-09-12T16:03:31.784 INFO:teuthology.orchestra.run.smithi130.stdout:epoch 24 2023-09-12T16:03:31.784 INFO:teuthology.orchestra.run.smithi130.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-09-12T16:03:31.784 INFO:teuthology.orchestra.run.smithi130.stdout:created 2023-09-12T16:02:04.422106+0000 2023-09-12T16:03:31.784 INFO:teuthology.orchestra.run.smithi130.stdout:modified 2023-09-12T16:02:25.646573+0000 2023-09-12T16:03:31.785 INFO:teuthology.orchestra.run.smithi130.stdout:tableserver 0 2023-09-12T16:03:31.785 INFO:teuthology.orchestra.run.smithi130.stdout:root 0 2023-09-12T16:03:31.785 INFO:teuthology.orchestra.run.smithi130.stdout:session_timeout 300 2023-09-12T16:03:31.785 INFO:teuthology.orchestra.run.smithi130.stdout:session_autoclose 300 2023-09-12T16:03:31.785 INFO:teuthology.orchestra.run.smithi130.stdout:max_file_size 1099511627776 2023-09-12T16:03:31.786 INFO:teuthology.orchestra.run.smithi130.stdout:max_xattr_size 65536 2023-09-12T16:03:31.786 INFO:teuthology.orchestra.run.smithi130.stdout:required_client_features {} 2023-09-12T16:03:31.786 INFO:teuthology.orchestra.run.smithi130.stdout:last_failure 0 2023-09-12T16:03:31.786 INFO:teuthology.orchestra.run.smithi130.stdout:last_failure_osd_epoch 0 2023-09-12T16:03:31.786 INFO:teuthology.orchestra.run.smithi130.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-12T16:03:31.787 INFO:teuthology.orchestra.run.smithi130.stdout:max_mds 3 2023-09-12T16:03:31.787 INFO:teuthology.orchestra.run.smithi130.stdout:in 0,1,2 2023-09-12T16:03:31.787 INFO:teuthology.orchestra.run.smithi130.stdout:up {0=24593,1=24592,2=24574} 2023-09-12T16:03:31.787 INFO:teuthology.orchestra.run.smithi130.stdout:failed 2023-09-12T16:03:31.787 INFO:teuthology.orchestra.run.smithi130.stdout:damaged 2023-09-12T16:03:31.788 INFO:teuthology.orchestra.run.smithi130.stdout:stopped 2023-09-12T16:03:31.788 INFO:teuthology.orchestra.run.smithi130.stdout:data_pools [3,4] 2023-09-12T16:03:31.788 INFO:teuthology.orchestra.run.smithi130.stdout:metadata_pool 2 2023-09-12T16:03:31.788 INFO:teuthology.orchestra.run.smithi130.stdout:inline_data disabled 2023-09-12T16:03:31.788 INFO:teuthology.orchestra.run.smithi130.stdout:balancer 2023-09-12T16:03:31.789 INFO:teuthology.orchestra.run.smithi130.stdout:bal_rank_mask -1 2023-09-12T16:03:31.789 INFO:teuthology.orchestra.run.smithi130.stdout:standby_count_wanted 0 2023-09-12T16:03:31.789 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.b{0:24593} state up:active seq 3 addr [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.789 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.f{0:24562} state up:standby-replay seq 1 addr [v2:172.21.15.203:6834/1801270748,v1:172.21.15.203:6835/1801270748] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.790 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.i{1:24592} state up:active seq 9 addr [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.790 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.h{1:24557} state up:standby-replay seq 1 addr [v2:172.21.15.159:6836/3413713845,v1:172.21.15.159:6837/3413713845] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.790 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.c{2:24574} state up:active seq 11 addr [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.790 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.e{2:24533} state up:standby-replay seq 1 addr [v2:172.21.15.159:6834/3667735298,v1:172.21.15.159:6835/3667735298] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.790 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:31.791 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:31.791 INFO:teuthology.orchestra.run.smithi130.stdout:Standby daemons: 2023-09-12T16:03:31.791 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:31.791 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.j{-1:14709} state up:standby seq 1 addr [v2:172.21.15.130:6834/2074628565,v1:172.21.15.130:6835/2074628565] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.792 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.d{-1:14715} state up:standby seq 1 addr [v2:172.21.15.130:6836/372380878,v1:172.21.15.130:6837/372380878] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.792 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.g{-1:14772} state up:standby seq 1 addr [v2:172.21.15.130:6838/2430896629,v1:172.21.15.130:6839/2430896629] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.792 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.a{-1:14784} state up:standby seq 1 addr [v2:172.21.15.130:6840/1672964981,v1:172.21.15.130:6841/1672964981] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.792 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.k{-1:24512} state up:standby seq 1 addr [v2:172.21.15.159:6832/2681608700,v1:172.21.15.159:6833/2681608700] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.793 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.l{-1:24520} state up:standby seq 1 addr [v2:172.21.15.203:6832/2012369878,v1:172.21.15.203:6833/2012369878] compat {c=[1],r=[1],i=[fff]}] 2023-09-12T16:03:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8e300a0750 msgr2=0x7f8e300a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8e300a0750 0x7f8e300a2c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8e340099d0 tx=0x7f8e34008040 comp rx=0 tx=0).stop 2023-09-12T16:03:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 msgr2=0x7f8e441963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 0x7f8e441963c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005910 tx=0x7f8e4000f710 comp rx=0 tx=0).stop 2023-09-12T16:03:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 shutdown_connections 2023-09-12T16:03:31.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8e300a0750 0x7f8e300a2c10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106380 0x7f8e4419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e44101120 0x7f8e441963c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 --2- 172.21.15.130:0/3108248492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44103a50 0x7f8e44196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 >> 172.21.15.130:0/3108248492 conn(0x7f8e440fab10 msgr2=0x7f8e44102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 shutdown_connections 2023-09-12T16:03:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:31.783+0000 7f8e4b881700 1 -- 172.21.15.130:0/3108248492 wait complete. 2023-09-12T16:03:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 24 2023-09-12T16:03:32.124 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:31 smithi130 ceph-mon[98670]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:32.124 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:31 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3108248492' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-12T16:03:32.125 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph osd dump' 2023-09-12T16:03:32.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:31 smithi159 ceph-mon[102522]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:32.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:31 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3108248492' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-12T16:03:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:31 smithi203 ceph-mon[103431]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:31 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3108248492' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-12T16:03:32.683 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:33.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.879+0000 7f429e11f700 1 -- 172.21.15.130:0/3205387652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298103ab0 msgr2=0x7f4298105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.879+0000 7f429e11f700 1 --2- 172.21.15.130:0/3205387652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298103ab0 0x7f4298105ea0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4288009990 tx=0x7f4288009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.879+0000 7f429e11f700 1 -- 172.21.15.130:0/3205387652 shutdown_connections 2023-09-12T16:03:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.879+0000 7f429e11f700 1 --2- 172.21.15.130:0/3205387652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42981063e0 0x7f42981087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.879+0000 7f429e11f700 1 --2- 172.21.15.130:0/3205387652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4298101180 0x7f4298103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.879+0000 7f429e11f700 1 --2- 172.21.15.130:0/3205387652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298103ab0 0x7f4298105ea0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.879+0000 7f429e11f700 1 -- 172.21.15.130:0/3205387652 >> 172.21.15.130:0/3205387652 conn(0x7f42980fab10 msgr2=0x7f42980fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:33.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.879+0000 7f429e11f700 1 -- 172.21.15.130:0/3205387652 shutdown_connections 2023-09-12T16:03:33.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.879+0000 7f429e11f700 1 -- 172.21.15.130:0/3205387652 wait complete. 2023-09-12T16:03:33.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f429e11f700 1 Processor -- start 2023-09-12T16:03:33.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f429e11f700 1 -- start start 2023-09-12T16:03:33.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f429e11f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298101180 0x7f42981962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:33.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f429e11f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4298103ab0 0x7f4298196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:33.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f429e11f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42981063e0 0x7f429819d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:33.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f429e11f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429819de60 con 0x7f4298101180 2023-09-12T16:03:33.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f429e11f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429819dfa0 con 0x7f4298103ab0 2023-09-12T16:03:33.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f429e11f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429819e130 con 0x7f42981063e0 2023-09-12T16:03:33.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f42977fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298101180 0x7f42981962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:33.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f42977fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298101180 0x7f42981962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54704/0 (socket says 172.21.15.130:54704) 2023-09-12T16:03:33.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f42977fe700 1 -- 172.21.15.130:0/1131766312 learned_addr learned my addr 172.21.15.130:0/1131766312 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:33.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.880+0000 7f4296ffd700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4298103ab0 0x7f4298196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:33.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f42977fe700 1 -- 172.21.15.130:0/1131766312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42981063e0 msgr2=0x7f429819d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:33.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f4297fff700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42981063e0 0x7f429819d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:33.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f42977fe700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42981063e0 0x7f429819d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:33.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f42977fe700 1 -- 172.21.15.130:0/1131766312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4298103ab0 msgr2=0x7f4298196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:33.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f42977fe700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4298103ab0 0x7f4298196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:33.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f42977fe700 1 -- 172.21.15.130:0/1131766312 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42880095f0 con 0x7f4298101180 2023-09-12T16:03:33.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f4297fff700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42981063e0 0x7f429819d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:33.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f42977fe700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298101180 0x7f42981962d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f428000c9e0 tx=0x7f428000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:33.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f4294ff9700 1 -- 172.21.15.130:0/1131766312 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280007ac0 con 0x7f4298101180 2023-09-12T16:03:33.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f4294ff9700 1 -- 172.21.15.130:0/1131766312 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4280007c20 con 0x7f4298101180 2023-09-12T16:03:33.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429819e410 con 0x7f4298101180 2023-09-12T16:03:33.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f4294ff9700 1 -- 172.21.15.130:0/1131766312 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280018660 con 0x7f4298101180 2023-09-12T16:03:33.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.881+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429819e910 con 0x7f4298101180 2023-09-12T16:03:33.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.882+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42980fc690 con 0x7f4298101180 2023-09-12T16:03:33.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.883+0000 7f4294ff9700 1 -- 172.21.15.130:0/1131766312 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f428000f450 con 0x7f4298101180 2023-09-12T16:03:33.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.884+0000 7f4294ff9700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f42840a06d0 0x7f42840a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:33.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.884+0000 7f4294ff9700 1 -- 172.21.15.130:0/1131766312 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f42800cc260 con 0x7f4298101180 2023-09-12T16:03:33.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.884+0000 7f4296ffd700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f42840a06d0 0x7f42840a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:33.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.884+0000 7f4296ffd700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f42840a06d0 0x7f42840a2b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f428800b5c0 tx=0x7f4288005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:33.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:33.887+0000 7f4294ff9700 1 -- 172.21.15.130:0/1131766312 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f42800959b0 con 0x7f4298101180 2023-09-12T16:03:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:33 smithi130 ceph-mon[98670]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:34.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.076+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f429810c190 con 0x7f4298101180 2023-09-12T16:03:34.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.077+0000 7f4294ff9700 1 -- 172.21.15.130:0/1131766312 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5619 (secure 0 0 0) 0x7f4280098fd0 con 0x7f4298101180 2023-09-12T16:03:34.078 INFO:teuthology.orchestra.run.smithi130.stdout:epoch 88 2023-09-12T16:03:34.078 INFO:teuthology.orchestra.run.smithi130.stdout:fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T16:03:34.078 INFO:teuthology.orchestra.run.smithi130.stdout:created 2023-09-12T15:55:02.305146+0000 2023-09-12T16:03:34.078 INFO:teuthology.orchestra.run.smithi130.stdout:modified 2023-09-12T16:02:51.826597+0000 2023-09-12T16:03:34.078 INFO:teuthology.orchestra.run.smithi130.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-09-12T16:03:34.079 INFO:teuthology.orchestra.run.smithi130.stdout:crush_version 27 2023-09-12T16:03:34.079 INFO:teuthology.orchestra.run.smithi130.stdout:full_ratio 0.95 2023-09-12T16:03:34.079 INFO:teuthology.orchestra.run.smithi130.stdout:backfillfull_ratio 0.9 2023-09-12T16:03:34.079 INFO:teuthology.orchestra.run.smithi130.stdout:nearfull_ratio 0.85 2023-09-12T16:03:34.079 INFO:teuthology.orchestra.run.smithi130.stdout:require_min_compat_client luminous 2023-09-12T16:03:34.080 INFO:teuthology.orchestra.run.smithi130.stdout:min_compat_client luminous 2023-09-12T16:03:34.080 INFO:teuthology.orchestra.run.smithi130.stdout:require_osd_release reef 2023-09-12T16:03:34.080 INFO:teuthology.orchestra.run.smithi130.stdout:stretch_mode_enabled false 2023-09-12T16:03:34.080 INFO:teuthology.orchestra.run.smithi130.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-09-12T16:03:34.080 INFO:teuthology.orchestra.run.smithi130.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-09-12T16:03:34.080 INFO:teuthology.orchestra.run.smithi130.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-09-12T16:03:34.081 INFO:teuthology.orchestra.run.smithi130.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-09-12T16:03:34.081 INFO:teuthology.orchestra.run.smithi130.stdout:max_osd 12 2023-09-12T16:03:34.081 INFO:teuthology.orchestra.run.smithi130.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6802/1767854979,v1:172.21.15.130:6803/1767854979] [v2:172.21.15.130:6804/1767854979,v1:172.21.15.130:6805/1767854979] exists,up 10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c 2023-09-12T16:03:34.081 INFO:teuthology.orchestra.run.smithi130.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/1684620878,v1:172.21.15.159:6801/1684620878] [v2:172.21.15.159:6802/1684620878,v1:172.21.15.159:6803/1684620878] exists,up 8fe17faa-52a3-4d63-878a-da07c02679b1 2023-09-12T16:03:34.081 INFO:teuthology.orchestra.run.smithi130.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6800/3797136832,v1:172.21.15.203:6801/3797136832] [v2:172.21.15.203:6802/3797136832,v1:172.21.15.203:6803/3797136832] exists,up 3c101604-31dd-462a-88ee-7d55829e0d2d 2023-09-12T16:03:34.082 INFO:teuthology.orchestra.run.smithi130.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6810/3784588793,v1:172.21.15.130:6811/3784588793] [v2:172.21.15.130:6812/3784588793,v1:172.21.15.130:6813/3784588793] exists,up 7045b414-e695-4f08-9764-72c802812750 2023-09-12T16:03:34.082 INFO:teuthology.orchestra.run.smithi130.stdout:osd.4 up in weight 1 up_from 32 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/2237508615,v1:172.21.15.159:6809/2237508615] [v2:172.21.15.159:6810/2237508615,v1:172.21.15.159:6811/2237508615] exists,up 8e61d2d4-2479-45ab-a033-28ae1aeb8202 2023-09-12T16:03:34.082 INFO:teuthology.orchestra.run.smithi130.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6808/2191150970,v1:172.21.15.203:6809/2191150970] [v2:172.21.15.203:6810/2191150970,v1:172.21.15.203:6811/2191150970] exists,up 5fc371aa-08c6-49f0-8f7f-0ce25705e987 2023-09-12T16:03:34.082 INFO:teuthology.orchestra.run.smithi130.stdout:osd.6 up in weight 1 up_from 44 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6818/291320749,v1:172.21.15.130:6819/291320749] [v2:172.21.15.130:6820/291320749,v1:172.21.15.130:6821/291320749] exists,up 6cb4de31-02a3-4822-944d-14a9fe33f79c 2023-09-12T16:03:34.082 INFO:teuthology.orchestra.run.smithi130.stdout:osd.7 up in weight 1 up_from 49 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/315414716,v1:172.21.15.159:6817/315414716] [v2:172.21.15.159:6818/315414716,v1:172.21.15.159:6819/315414716] exists,up d8c5305d-d340-496a-8967-461ee63ee0c9 2023-09-12T16:03:34.083 INFO:teuthology.orchestra.run.smithi130.stdout:osd.8 up in weight 1 up_from 56 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6816/2706023432,v1:172.21.15.203:6817/2706023432] [v2:172.21.15.203:6818/2706023432,v1:172.21.15.203:6819/2706023432] exists,up 52e84740-7c08-4017-b5d4-3a20f0ecad18 2023-09-12T16:03:34.083 INFO:teuthology.orchestra.run.smithi130.stdout:osd.9 up in weight 1 up_from 63 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6826/715792473,v1:172.21.15.130:6827/715792473] [v2:172.21.15.130:6828/715792473,v1:172.21.15.130:6829/715792473] exists,up 9c95841a-2cba-411e-b7e6-f958256adc5c 2023-09-12T16:03:34.083 INFO:teuthology.orchestra.run.smithi130.stdout:osd.10 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/1888655979,v1:172.21.15.159:6825/1888655979] [v2:172.21.15.159:6826/1888655979,v1:172.21.15.159:6827/1888655979] exists,up 742fd1e6-2b06-438c-8d3d-d7249d475f3e 2023-09-12T16:03:34.083 INFO:teuthology.orchestra.run.smithi130.stdout:osd.11 up in weight 1 up_from 76 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6824/768600250,v1:172.21.15.203:6825/768600250] [v2:172.21.15.203:6826/768600250,v1:172.21.15.203:6827/768600250] exists,up b32e5d8c-4556-4d0e-83ce-9f35138be3f4 2023-09-12T16:03:34.083 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 2.14 [2,4] 2023-09-12T16:03:34.084 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 2.1f [8,1] 2023-09-12T16:03:34.084 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 3.0 [4,8] 2023-09-12T16:03:34.084 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 3.1 [4,9] 2023-09-12T16:03:34.084 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 3.10 [5,9] 2023-09-12T16:03:34.084 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 3.13 [4,9] 2023-09-12T16:03:34.084 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 4.9 [5,8] 2023-09-12T16:03:34.085 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 4.b [9,8] 2023-09-12T16:03:34.085 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 4.17 [9,10] 2023-09-12T16:03:34.085 INFO:teuthology.orchestra.run.smithi130.stdout:pg_upmap_items 4.1b [9,8] 2023-09-12T16:03:34.085 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:0/1706265745 expires 2023-09-13T15:55:50.680182+0000 2023-09-12T16:03:34.085 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:6800/3042728256 expires 2023-09-13T15:55:50.680182+0000 2023-09-12T16:03:34.086 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:0/3742416190 expires 2023-09-13T15:55:50.680182+0000 2023-09-12T16:03:34.086 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:0/2502139426 expires 2023-09-13T15:55:28.728403+0000 2023-09-12T16:03:34.087 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:6801/3042728256 expires 2023-09-13T15:55:50.680182+0000 2023-09-12T16:03:34.087 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:6800/840569029 expires 2023-09-13T15:55:28.728403+0000 2023-09-12T16:03:34.087 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:6801/840569029 expires 2023-09-13T15:55:28.728403+0000 2023-09-12T16:03:34.087 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:0/410531865 expires 2023-09-13T15:55:50.680182+0000 2023-09-12T16:03:34.087 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:0/288931411 expires 2023-09-13T15:55:28.728403+0000 2023-09-12T16:03:34.088 INFO:teuthology.orchestra.run.smithi130.stdout:blocklist 172.21.15.130:0/1303099884 expires 2023-09-13T15:55:28.728403+0000 2023-09-12T16:03:34.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f42840a06d0 msgr2=0x7f42840a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f42840a06d0 0x7f42840a2b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f428800b5c0 tx=0x7f4288005960 comp rx=0 tx=0).stop 2023-09-12T16:03:34.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298101180 msgr2=0x7f42981962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298101180 0x7f42981962d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f428000c9e0 tx=0x7f428000ccf0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 shutdown_connections 2023-09-12T16:03:34.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f42840a06d0 0x7f42840a2b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42981063e0 0x7f429819d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4298101180 0x7f42981962d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 --2- 172.21.15.130:0/1131766312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4298103ab0 0x7f4298196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 >> 172.21.15.130:0/1131766312 conn(0x7f42980fab10 msgr2=0x7f4298101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:34.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 shutdown_connections 2023-09-12T16:03:34.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.079+0000 7f429e11f700 1 -- 172.21.15.130:0/1131766312 wait complete. 2023-09-12T16:03:34.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:33 smithi159 ceph-mon[102522]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:33 smithi203 ceph-mon[103431]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:34.441 INFO:teuthology.run_tasks:Running task fs.ready... 2023-09-12T16:03:34.456 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:34.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.581+0000 7f78c2aac700 1 -- 172.21.15.130:0/254998169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 msgr2=0x7f78bc0ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.581+0000 7f78c2aac700 1 --2- 172.21.15.130:0/254998169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 0x7f78bc0ff930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f78b00099c0 tx=0x7f78b0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.581+0000 7f78c2aac700 1 -- 172.21.15.130:0/254998169 shutdown_connections 2023-09-12T16:03:34.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.581+0000 7f78c2aac700 1 --2- 172.21.15.130:0/254998169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78bc1010a0 0x7f78bc107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.581+0000 7f78c2aac700 1 --2- 172.21.15.130:0/254998169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 0x7f78bc0ff930 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.581+0000 7f78c2aac700 1 --2- 172.21.15.130:0/254998169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc100700 0x7f78bc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.581+0000 7f78c2aac700 1 -- 172.21.15.130:0/254998169 >> 172.21.15.130:0/254998169 conn(0x7f78bc0faa80 msgr2=0x7f78bc0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:34.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 -- 172.21.15.130:0/254998169 shutdown_connections 2023-09-12T16:03:34.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 -- 172.21.15.130:0/254998169 wait complete. 2023-09-12T16:03:34.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 Processor -- start 2023-09-12T16:03:34.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 -- start start 2023-09-12T16:03:34.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 0x7f78bc069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:34.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78bc100700 0x7f78bc19b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:34.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc1010a0 0x7f78bc19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:34.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc19de60 con 0x7f78bc0ff510 2023-09-12T16:03:34.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc19dfd0 con 0x7f78bc100700 2023-09-12T16:03:34.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c2aac700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc19e190 con 0x7f78bc1010a0 2023-09-12T16:03:34.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c0848700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 0x7f78bc069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:34.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c0848700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 0x7f78bc069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54718/0 (socket says 172.21.15.130:54718) 2023-09-12T16:03:34.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.582+0000 7f78c0848700 1 -- 172.21.15.130:0/3915977977 learned_addr learned my addr 172.21.15.130:0/3915977977 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:34.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78c0848700 1 -- 172.21.15.130:0/3915977977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc1010a0 msgr2=0x7f78bc19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78c0848700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc1010a0 0x7f78bc19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78c0848700 1 -- 172.21.15.130:0/3915977977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78bc100700 msgr2=0x7f78bc19b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78bbfff700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78bc100700 0x7f78bc19b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:34.592 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78c0848700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78bc100700 0x7f78bc19b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.592 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78c0848700 1 -- 172.21.15.130:0/3915977977 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b00095f0 con 0x7f78bc0ff510 2023-09-12T16:03:34.592 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78c0848700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 0x7f78bc069510 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f78b00036f0 tx=0x7f78b0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:34.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78b9ffb700 1 -- 172.21.15.130:0/3915977977 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b001d070 con 0x7f78bc0ff510 2023-09-12T16:03:34.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78bc19e410 con 0x7f78bc0ff510 2023-09-12T16:03:34.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78b9ffb700 1 -- 172.21.15.130:0/3915977977 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78b0004110 con 0x7f78bc0ff510 2023-09-12T16:03:34.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78b9ffb700 1 -- 172.21.15.130:0/3915977977 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0021410 con 0x7f78bc0ff510 2023-09-12T16:03:34.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.583+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78bc19e890 con 0x7f78bc0ff510 2023-09-12T16:03:34.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.584+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78bc106a50 con 0x7f78bc0ff510 2023-09-12T16:03:34.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.585+0000 7f78b9ffb700 1 -- 172.21.15.130:0/3915977977 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f78b000f660 con 0x7f78bc0ff510 2023-09-12T16:03:34.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.586+0000 7f78b9ffb700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78ac0a0710 0x7f78ac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:34.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.586+0000 7f78bbfff700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78ac0a0710 0x7f78ac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:34.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.586+0000 7f78b9ffb700 1 -- 172.21.15.130:0/3915977977 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f78b00cd030 con 0x7f78bc0ff510 2023-09-12T16:03:34.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.586+0000 7f78bbfff700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78ac0a0710 0x7f78ac0a2bd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f78a8005950 tx=0x7f78a80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:34.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.590+0000 7f78b9ffb700 1 -- 172.21.15.130:0/3915977977 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f78b0096780 con 0x7f78bc0ff510 2023-09-12T16:03:34.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.782+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78bc10ad90 con 0x7f78bc0ff510 2023-09-12T16:03:34.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.783+0000 7f78b9ffb700 1 -- 172.21.15.130:0/3915977977 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24473 (secure 0 0 0) 0x7f78b0099da0 con 0x7f78bc0ff510 2023-09-12T16:03:34.784 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:34.785 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":88,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:51.826597+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:34.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78ac0a0710 msgr2=0x7f78ac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78ac0a0710 0x7f78ac0a2bd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f78a8005950 tx=0x7f78a80058e0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 msgr2=0x7f78bc069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 0x7f78bc069510 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f78b00036f0 tx=0x7f78b0009580 comp rx=0 tx=0).stop 2023-09-12T16:03:34.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 shutdown_connections 2023-09-12T16:03:34.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78ac0a0710 0x7f78ac0a2bd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc1010a0 0x7f78bc19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78bc0ff510 0x7f78bc069510 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 --2- 172.21.15.130:0/3915977977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78bc100700 0x7f78bc19b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 >> 172.21.15.130:0/3915977977 conn(0x7f78bc0faa80 msgr2=0x7f78bc103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:34.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 shutdown_connections 2023-09-12T16:03:34.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.786+0000 7f78c2aac700 1 -- 172.21.15.130:0/3915977977 wait complete. 2023-09-12T16:03:34.799 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:03:34.800 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:03:34.930 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.927+0000 7f4f714f2700 1 -- 172.21.15.130:0/1386985926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c0ff720 msgr2=0x7f4f6c0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.930 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.927+0000 7f4f714f2700 1 --2- 172.21.15.130:0/1386985926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c0ff720 0x7f4f6c0ffba0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009990 tx=0x7f4f5c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.931 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.927+0000 7f4f714f2700 1 -- 172.21.15.130:0/1386985926 shutdown_connections 2023-09-12T16:03:34.931 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.927+0000 7f4f714f2700 1 --2- 172.21.15.130:0/1386985926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f6c1000e0 0x7f4f6c107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.931 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.927+0000 7f4f714f2700 1 --2- 172.21.15.130:0/1386985926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c100ad0 0x7f4f6c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.931 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.927+0000 7f4f714f2700 1 --2- 172.21.15.130:0/1386985926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c0ff720 0x7f4f6c0ffba0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.931 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.927+0000 7f4f714f2700 1 -- 172.21.15.130:0/1386985926 >> 172.21.15.130:0/1386985926 conn(0x7f4f6c0fa1f0 msgr2=0x7f4f6c0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:34.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.928+0000 7f4f714f2700 1 -- 172.21.15.130:0/1386985926 shutdown_connections 2023-09-12T16:03:34.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.928+0000 7f4f714f2700 1 -- 172.21.15.130:0/1386985926 wait complete. 2023-09-12T16:03:34.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.928+0000 7f4f714f2700 1 Processor -- start 2023-09-12T16:03:34.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.928+0000 7f4f714f2700 1 -- start start 2023-09-12T16:03:34.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.928+0000 7f4f714f2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f6c0ff720 0x7f4f6c0ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:34.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.928+0000 7f4f714f2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c1000e0 0x7f4f6c19b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:34.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.928+0000 7f4f714f2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c100ad0 0x7f4f6c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:34.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.928+0000 7f4f714f2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19deb0 con 0x7f4f6c1000e0 2023-09-12T16:03:34.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f714f2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19e020 con 0x7f4f6c0ff720 2023-09-12T16:03:34.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f714f2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19e1e0 con 0x7f4f6c100ad0 2023-09-12T16:03:34.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6a7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c1000e0 0x7f4f6c19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:34.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6a7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c1000e0 0x7f4f6c19b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54732/0 (socket says 172.21.15.130:54732) 2023-09-12T16:03:34.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6a7fc700 1 -- 172.21.15.130:0/4260598691 learned_addr learned my addr 172.21.15.130:0/4260598691 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:34.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6b7fe700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c100ad0 0x7f4f6c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:34.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6affd700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f6c0ff720 0x7f4f6c0ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:34.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6a7fc700 1 -- 172.21.15.130:0/4260598691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c100ad0 msgr2=0x7f4f6c19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6a7fc700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c100ad0 0x7f4f6c19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6a7fc700 1 -- 172.21.15.130:0/4260598691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f6c0ff720 msgr2=0x7f4f6c0ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:34.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6a7fc700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f6c0ff720 0x7f4f6c0ff1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:34.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6a7fc700 1 -- 172.21.15.130:0/4260598691 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0095f0 con 0x7f4f6c1000e0 2023-09-12T16:03:34.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6b7fe700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c100ad0 0x7f4f6c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:34.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f6a7fc700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c1000e0 0x7f4f6c19b090 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c005910 tx=0x7f4f5c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:34.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f63fff700 1 -- 172.21.15.130:0/4260598691 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c01d070 con 0x7f4f6c1000e0 2023-09-12T16:03:34.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f63fff700 1 -- 172.21.15.130:0/4260598691 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f5c004060 con 0x7f4f6c1000e0 2023-09-12T16:03:34.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c19e460 con 0x7f4f6c1000e0 2023-09-12T16:03:34.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f63fff700 1 -- 172.21.15.130:0/4260598691 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c022e10 con 0x7f4f6c1000e0 2023-09-12T16:03:34.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.929+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c19e880 con 0x7f4f6c1000e0 2023-09-12T16:03:34.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.930+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c106ad0 con 0x7f4f6c1000e0 2023-09-12T16:03:34.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.931+0000 7f4f63fff700 1 -- 172.21.15.130:0/4260598691 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f4f5c022470 con 0x7f4f6c1000e0 2023-09-12T16:03:34.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.932+0000 7f4f63fff700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4f580a06d0 0x7f4f580a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:34.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.932+0000 7f4f6affd700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4f580a06d0 0x7f4f580a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:34.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.932+0000 7f4f63fff700 1 -- 172.21.15.130:0/4260598691 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4f5c0cd500 con 0x7f4f6c1000e0 2023-09-12T16:03:34.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.932+0000 7f4f6affd700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4f580a06d0 0x7f4f580a2b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4f54005fd0 tx=0x7f4f54005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:34.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:34.936+0000 7f4f63fff700 1 -- 172.21.15.130:0/4260598691 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4f5c096c80 con 0x7f4f6c1000e0 2023-09-12T16:03:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.126+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f6c10af50 con 0x7f4f6c1000e0 2023-09-12T16:03:35.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.126+0000 7f4f63fff700 1 -- 172.21.15.130:0/4260598691 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f5c09a2a0 con 0x7f4f6c1000e0 2023-09-12T16:03:35.127 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:03:35.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4f580a06d0 msgr2=0x7f4f580a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4f580a06d0 0x7f4f580a2b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4f54005fd0 tx=0x7f4f54005ee0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c1000e0 msgr2=0x7f4f6c19b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c1000e0 0x7f4f6c19b090 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c005910 tx=0x7f4f5c005e80 comp rx=0 tx=0).stop 2023-09-12T16:03:35.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 shutdown_connections 2023-09-12T16:03:35.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4f580a06d0 0x7f4f580a2b90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c100ad0 0x7f4f6c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f6c0ff720 0x7f4f6c0ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 --2- 172.21.15.130:0/4260598691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f6c1000e0 0x7f4f6c19b090 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 >> 172.21.15.130:0/4260598691 conn(0x7f4f6c0fa1f0 msgr2=0x7f4f6c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:35.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 shutdown_connections 2023-09-12T16:03:35.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.128+0000 7f4f714f2700 1 -- 172.21.15.130:0/4260598691 wait complete. 2023-09-12T16:03:35.140 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:34 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1131766312' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-12T16:03:35.140 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:34 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3915977977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:35.141 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:03:35.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:34 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1131766312' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-12T16:03:35.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:34 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3915977977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:35.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:34 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1131766312' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-12T16:03:35.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:34 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3915977977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:35.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.265+0000 7f8f1c258700 1 -- 172.21.15.130:0/2943360125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 msgr2=0x7f8f140ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 --2- 172.21.15.130:0/2943360125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 0x7f8f140ff910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8f040099c0 tx=0x7f8f04009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 -- 172.21.15.130:0/2943360125 shutdown_connections 2023-09-12T16:03:35.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 --2- 172.21.15.130:0/2943360125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f14101080 0x7f8f14107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 --2- 172.21.15.130:0/2943360125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 0x7f8f140ff910 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 --2- 172.21.15.130:0/2943360125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f141006e0 0x7f8f14100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 -- 172.21.15.130:0/2943360125 >> 172.21.15.130:0/2943360125 conn(0x7f8f140faa60 msgr2=0x7f8f140fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:35.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 -- 172.21.15.130:0/2943360125 shutdown_connections 2023-09-12T16:03:35.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 -- 172.21.15.130:0/2943360125 wait complete. 2023-09-12T16:03:35.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 Processor -- start 2023-09-12T16:03:35.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 -- start start 2023-09-12T16:03:35.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 0x7f8f141952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f141006e0 0x7f8f141957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f14101080 0x7f8f1419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1419ce30 con 0x7f8f140ff4f0 2023-09-12T16:03:35.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1419cf70 con 0x7f8f14101080 2023-09-12T16:03:35.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.266+0000 7f8f1c258700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1419d100 con 0x7f8f141006e0 2023-09-12T16:03:35.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f19ff4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 0x7f8f141952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f19ff4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 0x7f8f141952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54742/0 (socket says 172.21.15.130:54742) 2023-09-12T16:03:35.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f19ff4700 1 -- 172.21.15.130:0/1115348053 learned_addr learned my addr 172.21.15.130:0/1115348053 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:35.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f197f3700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f141006e0 0x7f8f141957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f1a7f5700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f14101080 0x7f8f1419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f19ff4700 1 -- 172.21.15.130:0/1115348053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f141006e0 msgr2=0x7f8f141957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f19ff4700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f141006e0 0x7f8f141957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f19ff4700 1 -- 172.21.15.130:0/1115348053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f14101080 msgr2=0x7f8f1419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f19ff4700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f14101080 0x7f8f1419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f19ff4700 1 -- 172.21.15.130:0/1115348053 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f040095f0 con 0x7f8f140ff4f0 2023-09-12T16:03:35.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f19ff4700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 0x7f8f141952b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009990 tx=0x7f8f0400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:35.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f0affd700 1 -- 172.21.15.130:0/1115348053 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0401d070 con 0x7f8f140ff4f0 2023-09-12T16:03:35.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f0affd700 1 -- 172.21.15.130:0/1115348053 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f040054f0 con 0x7f8f140ff4f0 2023-09-12T16:03:35.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f0affd700 1 -- 172.21.15.130:0/1115348053 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04004bd0 con 0x7f8f140ff4f0 2023-09-12T16:03:35.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f14067760 con 0x7f8f140ff4f0 2023-09-12T16:03:35.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.267+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f14067bc0 con 0x7f8f140ff4f0 2023-09-12T16:03:35.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.268+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1410ad60 con 0x7f8f140ff4f0 2023-09-12T16:03:35.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.269+0000 7f8f0affd700 1 -- 172.21.15.130:0/1115348053 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f8f0400fb80 con 0x7f8f140ff4f0 2023-09-12T16:03:35.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.269+0000 7f8f0affd700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8f000a06d0 0x7f8f000a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.270+0000 7f8f0affd700 1 -- 172.21.15.130:0/1115348053 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8f040cd2b0 con 0x7f8f140ff4f0 2023-09-12T16:03:35.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.270+0000 7f8f197f3700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8f000a06d0 0x7f8f000a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.270+0000 7f8f197f3700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8f000a06d0 0x7f8f000a2b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f10005950 tx=0x7f8f10011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:35.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.273+0000 7f8f0affd700 1 -- 172.21.15.130:0/1115348053 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8f04096a00 con 0x7f8f140ff4f0 2023-09-12T16:03:35.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.466+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f14068090 con 0x7f8f140ff4f0 2023-09-12T16:03:35.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.467+0000 7f8f0affd700 1 -- 172.21.15.130:0/1115348053 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f0409a020 con 0x7f8f140ff4f0 2023-09-12T16:03:35.467 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:35.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8f000a06d0 msgr2=0x7f8f000a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8f000a06d0 0x7f8f000a2b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f10005950 tx=0x7f8f10011040 comp rx=0 tx=0).stop 2023-09-12T16:03:35.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 msgr2=0x7f8f141952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 0x7f8f141952b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009990 tx=0x7f8f0400f690 comp rx=0 tx=0).stop 2023-09-12T16:03:35.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 shutdown_connections 2023-09-12T16:03:35.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8f000a06d0 0x7f8f000a2b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f14101080 0x7f8f1419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f140ff4f0 0x7f8f141952b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 --2- 172.21.15.130:0/1115348053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f141006e0 0x7f8f141957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 >> 172.21.15.130:0/1115348053 conn(0x7f8f140faa60 msgr2=0x7f8f140fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:35.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 shutdown_connections 2023-09-12T16:03:35.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.469+0000 7f8f1c258700 1 -- 172.21.15.130:0/1115348053 wait complete. 2023-09-12T16:03:35.482 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:03:35.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.606+0000 7f10d8104700 1 -- 172.21.15.130:0/2237190472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d00ff4f0 msgr2=0x7f10d00ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.606+0000 7f10d8104700 1 --2- 172.21.15.130:0/2237190472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d00ff4f0 0x7f10d00ff910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0099c0 tx=0x7f10cc009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.606+0000 7f10d8104700 1 -- 172.21.15.130:0/2237190472 shutdown_connections 2023-09-12T16:03:35.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.606+0000 7f10d8104700 1 --2- 172.21.15.130:0/2237190472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d0101080 0x7f10d0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.606+0000 7f10d8104700 1 --2- 172.21.15.130:0/2237190472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d00ff4f0 0x7f10d00ff910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.606+0000 7f10d8104700 1 --2- 172.21.15.130:0/2237190472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d01006e0 0x7f10d0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.606+0000 7f10d8104700 1 -- 172.21.15.130:0/2237190472 >> 172.21.15.130:0/2237190472 conn(0x7f10d00faa60 msgr2=0x7f10d00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:35.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 -- 172.21.15.130:0/2237190472 shutdown_connections 2023-09-12T16:03:35.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 -- 172.21.15.130:0/2237190472 wait complete. 2023-09-12T16:03:35.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 Processor -- start 2023-09-12T16:03:35.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 -- start start 2023-09-12T16:03:35.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d00ff4f0 0x7f10d0195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d01006e0 0x7f10d01957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d0101080 0x7f10d0068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0068d60 con 0x7f10d0101080 2023-09-12T16:03:35.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0068ed0 con 0x7f10d00ff4f0 2023-09-12T16:03:35.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.607+0000 7f10d8104700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0069090 con 0x7f10d01006e0 2023-09-12T16:03:35.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d66a1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d0101080 0x7f10d0068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d66a1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d0101080 0x7f10d0068790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54758/0 (socket says 172.21.15.130:54758) 2023-09-12T16:03:35.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d66a1700 1 -- 172.21.15.130:0/301512073 learned_addr learned my addr 172.21.15.130:0/301512073 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:35.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d569f700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d01006e0 0x7f10d01957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d5ea0700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d00ff4f0 0x7f10d0195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d66a1700 1 -- 172.21.15.130:0/301512073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d01006e0 msgr2=0x7f10d01957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d66a1700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d01006e0 0x7f10d01957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d66a1700 1 -- 172.21.15.130:0/301512073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d00ff4f0 msgr2=0x7f10d0195270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d66a1700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d00ff4f0 0x7f10d0195270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d66a1700 1 -- 172.21.15.130:0/301512073 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10cc0095f0 con 0x7f10d0101080 2023-09-12T16:03:35.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d569f700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d01006e0 0x7f10d01957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:03:35.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d66a1700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d0101080 0x7f10d0068790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f10c800ef30 tx=0x7f10c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:35.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10c6ffd700 1 -- 172.21.15.130:0/301512073 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c8018070 con 0x7f10d0101080 2023-09-12T16:03:35.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d00692b0 con 0x7f10d0101080 2023-09-12T16:03:35.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10c6ffd700 1 -- 172.21.15.130:0/301512073 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c800f040 con 0x7f10d0101080 2023-09-12T16:03:35.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10c6ffd700 1 -- 172.21.15.130:0/301512073 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c800cd00 con 0x7f10d0101080 2023-09-12T16:03:35.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.608+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d0198800 con 0x7f10d0101080 2023-09-12T16:03:35.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.609+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d01064a0 con 0x7f10d0101080 2023-09-12T16:03:35.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.610+0000 7f10c6ffd700 1 -- 172.21.15.130:0/301512073 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f10c8003a00 con 0x7f10d0101080 2023-09-12T16:03:35.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.611+0000 7f10c6ffd700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f10bc0a06d0 0x7f10bc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.611+0000 7f10d5ea0700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f10bc0a06d0 0x7f10bc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.611+0000 7f10c6ffd700 1 -- 172.21.15.130:0/301512073 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f10c80cbb60 con 0x7f10d0101080 2023-09-12T16:03:35.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.611+0000 7f10d5ea0700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f10bc0a06d0 0x7f10bc0a2b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009990 tx=0x7f10cc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:35.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.615+0000 7f10c6ffd700 1 -- 172.21.15.130:0/301512073 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f10c80952b0 con 0x7f10d0101080 2023-09-12T16:03:35.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.807+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10d010afd0 con 0x7f10d0101080 2023-09-12T16:03:35.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.807+0000 7f10c6ffd700 1 -- 172.21.15.130:0/301512073 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10c80988d0 con 0x7f10d0101080 2023-09-12T16:03:35.808 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:03:35.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f10bc0a06d0 msgr2=0x7f10bc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f10bc0a06d0 0x7f10bc0a2b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009990 tx=0x7f10cc005fb0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d0101080 msgr2=0x7f10d0068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d0101080 0x7f10d0068790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f10c800ef30 tx=0x7f10c800c3a0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 shutdown_connections 2023-09-12T16:03:35.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f10bc0a06d0 0x7f10bc0a2b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10d0101080 0x7f10d0068790 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d00ff4f0 0x7f10d0195270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 --2- 172.21.15.130:0/301512073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d01006e0 0x7f10d01957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 >> 172.21.15.130:0/301512073 conn(0x7f10d00faa60 msgr2=0x7f10d00fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:35.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 shutdown_connections 2023-09-12T16:03:35.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.809+0000 7f10d8104700 1 -- 172.21.15.130:0/301512073 wait complete. 2023-09-12T16:03:35.822 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:03:35.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.945+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/4105132926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd8101080 msgr2=0x7f3bd8107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.945+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/4105132926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd8101080 0x7f3bd8107930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000b0d0 tx=0x7f3bd000b3e0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/4105132926 shutdown_connections 2023-09-12T16:03:35.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/4105132926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd8101080 0x7f3bd8107930 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/4105132926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd80ff4f0 0x7f3bd80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/4105132926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bd81006e0 0x7f3bd8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/4105132926 >> 172.21.15.130:0/4105132926 conn(0x7f3bd80faa60 msgr2=0x7f3bd80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:35.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/4105132926 shutdown_connections 2023-09-12T16:03:35.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/4105132926 wait complete. 2023-09-12T16:03:35.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 Processor -- start 2023-09-12T16:03:35.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 -- start start 2023-09-12T16:03:35.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd80ff4f0 0x7f3bd81963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bd81006e0 0x7f3bd81968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd8101080 0x7f3bd819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd819df30 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd819e070 con 0x7f3bd81006e0 2023-09-12T16:03:35.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.946+0000 7f3bdf5f5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd819e200 con 0x7f3bd8101080 2023-09-12T16:03:35.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdd391700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd80ff4f0 0x7f3bd81963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bddb92700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd8101080 0x7f3bd819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdd391700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd80ff4f0 0x7f3bd81963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54768/0 (socket says 172.21.15.130:54768) 2023-09-12T16:03:35.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdd391700 1 -- 172.21.15.130:0/244779516 learned_addr learned my addr 172.21.15.130:0/244779516 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:35.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdcb90700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bd81006e0 0x7f3bd81968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdd391700 1 -- 172.21.15.130:0/244779516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd8101080 msgr2=0x7f3bd819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdd391700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd8101080 0x7f3bd819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdd391700 1 -- 172.21.15.130:0/244779516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bd81006e0 msgr2=0x7f3bd81968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:35.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdd391700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bd81006e0 0x7f3bd81968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:35.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdd391700 1 -- 172.21.15.130:0/244779516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd0009d00 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdcb90700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bd81006e0 0x7f3bd81968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:35.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdd391700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd80ff4f0 0x7f3bd81963b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400d980 tx=0x7f3bd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:35.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bce7fc700 1 -- 172.21.15.130:0/244779516 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400b950 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bce7fc700 1 -- 172.21.15.130:0/244779516 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bd4007620 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd819e4e0 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bce7fc700 1 -- 172.21.15.130:0/244779516 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400f3c0 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.947+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd819ea40 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.948+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd810ad00 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.949+0000 7f3bce7fc700 1 -- 172.21.15.130:0/244779516 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f3bd4010640 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.950+0000 7f3bce7fc700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3bc40a0750 0x7f3bc40a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:35.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.950+0000 7f3bce7fc700 1 -- 172.21.15.130:0/244779516 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3bd40cc750 con 0x7f3bd80ff4f0 2023-09-12T16:03:35.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.950+0000 7f3bdcb90700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3bc40a0750 0x7f3bc40a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:35.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.950+0000 7f3bdcb90700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3bc40a0750 0x7f3bc40a2c10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8005fd0 tx=0x7f3bc8005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:35.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:35.953+0000 7f3bce7fc700 1 -- 172.21.15.130:0/244779516 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3bd4095e10 con 0x7f3bd80ff4f0 2023-09-12T16:03:36.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.142+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3bd819ec60 con 0x7f3bd80ff4f0 2023-09-12T16:03:36.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.143+0000 7f3bce7fc700 1 -- 172.21.15.130:0/244779516 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3bd40160a0 con 0x7f3bd80ff4f0 2023-09-12T16:03:36.144 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:36.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3bc40a0750 msgr2=0x7f3bc40a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3bc40a0750 0x7f3bc40a2c10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8005fd0 tx=0x7f3bc8005db0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd80ff4f0 msgr2=0x7f3bd81963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd80ff4f0 0x7f3bd81963b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400d980 tx=0x7f3bd400dc90 comp rx=0 tx=0).stop 2023-09-12T16:03:36.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 shutdown_connections 2023-09-12T16:03:36.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3bc40a0750 0x7f3bc40a2c10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd8101080 0x7f3bd819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3bd80ff4f0 0x7f3bd81963b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 --2- 172.21.15.130:0/244779516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bd81006e0 0x7f3bd81968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 >> 172.21.15.130:0/244779516 conn(0x7f3bd80faa60 msgr2=0x7f3bd80fb280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:36.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 shutdown_connections 2023-09-12T16:03:36.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.145+0000 7f3bdf5f5700 1 -- 172.21.15.130:0/244779516 wait complete. 2023-09-12T16:03:36.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:35 smithi130 ceph-mon[98670]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:36.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:35 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4260598691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:36.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:35 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1115348053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:36.157 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:35 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/301512073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:36.158 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:03:36.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:35 smithi159 ceph-mon[102522]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:36.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:35 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4260598691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:36.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:35 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1115348053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:36.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:35 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/301512073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:35 smithi203 ceph-mon[103431]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:35 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4260598691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:36.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:35 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1115348053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:36.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:35 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/301512073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:36.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.279+0000 7fb25c8db700 1 -- 172.21.15.130:0/2566250983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb258101080 msgr2=0x7fb258107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.279+0000 7fb25c8db700 1 --2- 172.21.15.130:0/2566250983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb258101080 0x7fb258107930 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb25000b0d0 tx=0x7fb25000b3e0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.279+0000 7fb25c8db700 1 -- 172.21.15.130:0/2566250983 shutdown_connections 2023-09-12T16:03:36.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.279+0000 7fb25c8db700 1 --2- 172.21.15.130:0/2566250983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb258101080 0x7fb258107930 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.279+0000 7fb25c8db700 1 --2- 172.21.15.130:0/2566250983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2580ff4f0 0x7fb2580ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.279+0000 7fb25c8db700 1 --2- 172.21.15.130:0/2566250983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2581006e0 0x7fb258100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.279+0000 7fb25c8db700 1 -- 172.21.15.130:0/2566250983 >> 172.21.15.130:0/2566250983 conn(0x7fb2580faa60 msgr2=0x7fb2580fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:36.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.279+0000 7fb25c8db700 1 -- 172.21.15.130:0/2566250983 shutdown_connections 2023-09-12T16:03:36.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.279+0000 7fb25c8db700 1 -- 172.21.15.130:0/2566250983 wait complete. 2023-09-12T16:03:36.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25c8db700 1 Processor -- start 2023-09-12T16:03:36.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25c8db700 1 -- start start 2023-09-12T16:03:36.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25c8db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2580ff4f0 0x7fb2581952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:36.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25c8db700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2581006e0 0x7fb2581957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:36.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25c8db700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb258101080 0x7fb25819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:36.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25c8db700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25819ce30 con 0x7fb2580ff4f0 2023-09-12T16:03:36.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25c8db700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25819cf70 con 0x7fb258101080 2023-09-12T16:03:36.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25c8db700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25819d100 con 0x7fb2581006e0 2023-09-12T16:03:36.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2580ff4f0 0x7fb2581952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:36.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2580ff4f0 0x7fb2581952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54782/0 (socket says 172.21.15.130:54782) 2023-09-12T16:03:36.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb25659c700 1 -- 172.21.15.130:0/1189273197 learned_addr learned my addr 172.21.15.130:0/1189273197 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:36.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.280+0000 7fb256d9d700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb258101080 0x7fb25819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:36.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb255d9b700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2581006e0 0x7fb2581957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:36.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb25659c700 1 -- 172.21.15.130:0/1189273197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2581006e0 msgr2=0x7fb2581957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb25659c700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2581006e0 0x7fb2581957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb25659c700 1 -- 172.21.15.130:0/1189273197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb258101080 msgr2=0x7fb25819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb25659c700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb258101080 0x7fb25819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb25659c700 1 -- 172.21.15.130:0/1189273197 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb250009d00 con 0x7fb2580ff4f0 2023-09-12T16:03:36.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb25659c700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2580ff4f0 0x7fb2581952b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d980 tx=0x7fb24c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:36.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb2477fe700 1 -- 172.21.15.130:0/1189273197 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c004020 con 0x7fb2580ff4f0 2023-09-12T16:03:36.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb2477fe700 1 -- 172.21.15.130:0/1189273197 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb24c00f690 con 0x7fb2580ff4f0 2023-09-12T16:03:36.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25819d3e0 con 0x7fb2580ff4f0 2023-09-12T16:03:36.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb2477fe700 1 -- 172.21.15.130:0/1189273197 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c003a60 con 0x7fb2580ff4f0 2023-09-12T16:03:36.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.281+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25819d8f0 con 0x7fb2580ff4f0 2023-09-12T16:03:36.290 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.282+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb258106990 con 0x7fb2580ff4f0 2023-09-12T16:03:36.290 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.283+0000 7fb2477fe700 1 -- 172.21.15.130:0/1189273197 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fb24c00fb00 con 0x7fb2580ff4f0 2023-09-12T16:03:36.290 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.284+0000 7fb2477fe700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb2400a0710 0x7fb2400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:36.290 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.284+0000 7fb255d9b700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb2400a0710 0x7fb2400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:36.290 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.284+0000 7fb2477fe700 1 -- 172.21.15.130:0/1189273197 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb24c0cbc30 con 0x7fb2580ff4f0 2023-09-12T16:03:36.290 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.284+0000 7fb255d9b700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb2400a0710 0x7fb2400a2bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb248007be0 tx=0x7fb2480073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:36.291 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.288+0000 7fb2477fe700 1 -- 172.21.15.130:0/1189273197 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb24c095290 con 0x7fb2580ff4f0 2023-09-12T16:03:36.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.523+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb25810aff0 con 0x7fb2580ff4f0 2023-09-12T16:03:36.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.523+0000 7fb2477fe700 1 -- 172.21.15.130:0/1189273197 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11342 (secure 0 0 0) 0x7fb24c0988b0 con 0x7fb2580ff4f0 2023-09-12T16:03:36.524 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:36.525 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:25.646573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:03:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb2400a0710 msgr2=0x7fb2400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb2400a0710 0x7fb2400a2bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb248007be0 tx=0x7fb2480073d0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2580ff4f0 msgr2=0x7fb2581952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2580ff4f0 0x7fb2581952b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d980 tx=0x7fb24c00dc90 comp rx=0 tx=0).stop 2023-09-12T16:03:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 shutdown_connections 2023-09-12T16:03:36.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb2400a0710 0x7fb2400a2bd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb258101080 0x7fb25819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2580ff4f0 0x7fb2581952b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 --2- 172.21.15.130:0/1189273197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2581006e0 0x7fb2581957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 >> 172.21.15.130:0/1189273197 conn(0x7fb2580faa60 msgr2=0x7fb258103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:36.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 shutdown_connections 2023-09-12T16:03:36.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.526+0000 7fb25c8db700 1 -- 172.21.15.130:0/1189273197 wait complete. 2023-09-12T16:03:36.529 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 24 2023-09-12T16:03:36.539 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:36.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.661+0000 7f78a1b80700 1 -- 172.21.15.130:0/954483558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100190 msgr2=0x7f789c107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.661+0000 7f78a1b80700 1 --2- 172.21.15.130:0/954483558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100190 0x7f789c107a50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f789400b0a0 tx=0x7f789400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.661+0000 7f78a1b80700 1 -- 172.21.15.130:0/954483558 shutdown_connections 2023-09-12T16:03:36.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.661+0000 7f78a1b80700 1 --2- 172.21.15.130:0/954483558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100190 0x7f789c107a50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.661+0000 7f78a1b80700 1 --2- 172.21.15.130:0/954483558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f789c100ac0 0x7f789c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.661+0000 7f78a1b80700 1 --2- 172.21.15.130:0/954483558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f789c0ff7f0 0x7f789c0ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.661+0000 7f78a1b80700 1 -- 172.21.15.130:0/954483558 >> 172.21.15.130:0/954483558 conn(0x7f789c0fa200 msgr2=0x7f789c0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:36.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.661+0000 7f78a1b80700 1 -- 172.21.15.130:0/954483558 shutdown_connections 2023-09-12T16:03:36.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.661+0000 7f78a1b80700 1 -- 172.21.15.130:0/954483558 wait complete. 2023-09-12T16:03:36.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f78a1b80700 1 Processor -- start 2023-09-12T16:03:36.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f78a1b80700 1 -- start start 2023-09-12T16:03:36.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f78a1b80700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f789c0ff7f0 0x7f789c0ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:36.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f78a1b80700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f789c100190 0x7f789c19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:36.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f78a1b80700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100ac0 0x7f789c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:36.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f78a1b80700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c19de70 con 0x7f789c100ac0 2023-09-12T16:03:36.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f78a1b80700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c19dfe0 con 0x7f789c0ff7f0 2023-09-12T16:03:36.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f78a1b80700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c19e1a0 con 0x7f789c100190 2023-09-12T16:03:36.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f789bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100ac0 0x7f789c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:36.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f789bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100ac0 0x7f789c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54800/0 (socket says 172.21.15.130:54800) 2023-09-12T16:03:36.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f789bfff700 1 -- 172.21.15.130:0/3996984709 learned_addr learned my addr 172.21.15.130:0/3996984709 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:36.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.662+0000 7f789affd700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f789c100190 0x7f789c19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:36.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f789b7fe700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f789c0ff7f0 0x7f789c0ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:36.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f789bfff700 1 -- 172.21.15.130:0/3996984709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f789c100190 msgr2=0x7f789c19b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f789bfff700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f789c100190 0x7f789c19b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f789bfff700 1 -- 172.21.15.130:0/3996984709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f789c0ff7f0 msgr2=0x7f789c0ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f789bfff700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f789c0ff7f0 0x7f789c0ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f789bfff700 1 -- 172.21.15.130:0/3996984709 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7894009d00 con 0x7f789c100ac0 2023-09-12T16:03:36.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f789bfff700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100ac0 0x7f789c19d6a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f789400b780 tx=0x7f789400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:36.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f7898ff9700 1 -- 172.21.15.130:0/3996984709 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789400e040 con 0x7f789c100ac0 2023-09-12T16:03:36.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f7898ff9700 1 -- 172.21.15.130:0/3996984709 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78940094a0 con 0x7f789c100ac0 2023-09-12T16:03:36.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c19e420 con 0x7f789c100ac0 2023-09-12T16:03:36.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f7898ff9700 1 -- 172.21.15.130:0/3996984709 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894005330 con 0x7f789c100ac0 2023-09-12T16:03:36.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.663+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c19e8a0 con 0x7f789c100ac0 2023-09-12T16:03:36.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.664+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789c10ad40 con 0x7f789c100ac0 2023-09-12T16:03:36.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.665+0000 7f7898ff9700 1 -- 172.21.15.130:0/3996984709 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f789401da30 con 0x7f789c100ac0 2023-09-12T16:03:36.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.666+0000 7f7898ff9700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78840a0710 0x7f78840a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:36.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.666+0000 7f789b7fe700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78840a0710 0x7f78840a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:36.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.666+0000 7f7898ff9700 1 -- 172.21.15.130:0/3996984709 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f78940cd750 con 0x7f789c100ac0 2023-09-12T16:03:36.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.666+0000 7f789b7fe700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78840a0710 0x7f78840a2bd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f788c005fd0 tx=0x7f788c005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:36.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.670+0000 7f7898ff9700 1 -- 172.21.15.130:0/3996984709 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7894096e40 con 0x7f789c100ac0 2023-09-12T16:03:36.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.864+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f789c19eac0 con 0x7f789c100ac0 2023-09-12T16:03:36.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.866+0000 7f7898ff9700 1 -- 172.21.15.130:0/3996984709 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24473 (secure 0 0 0) 0x7f789409a460 con 0x7f789c100ac0 2023-09-12T16:03:36.867 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:36.868 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":88,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:51.826597+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:36.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78840a0710 msgr2=0x7f78840a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78840a0710 0x7f78840a2bd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f788c005fd0 tx=0x7f788c005dd0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100ac0 msgr2=0x7f789c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:36.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100ac0 0x7f789c19d6a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f789400b780 tx=0x7f789400bd60 comp rx=0 tx=0).stop 2023-09-12T16:03:36.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 shutdown_connections 2023-09-12T16:03:36.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78840a0710 0x7f78840a2bd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f789c100ac0 0x7f789c19d6a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f789c0ff7f0 0x7f789c0ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 --2- 172.21.15.130:0/3996984709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f789c100190 0x7f789c19b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:36.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 >> 172.21.15.130:0/3996984709 conn(0x7f789c0fa200 msgr2=0x7f789c0fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:36.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 shutdown_connections 2023-09-12T16:03:36.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:36.868+0000 7f78a1b80700 1 -- 172.21.15.130:0/3996984709 wait complete. 2023-09-12T16:03:36.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:03:36.884 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:03:37.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.019+0000 7f400f74d700 1 -- 172.21.15.130:0/819295001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40080fe8c0 msgr2=0x7f40080fece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.019+0000 7f400f74d700 1 --2- 172.21.15.130:0/819295001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40080fe8c0 0x7f40080fece0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009990 tx=0x7f3ffc009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.019+0000 7f400f74d700 1 -- 172.21.15.130:0/819295001 shutdown_connections 2023-09-12T16:03:37.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.019+0000 7f400f74d700 1 --2- 172.21.15.130:0/819295001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40081022d0 0x7f4008108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.019+0000 7f400f74d700 1 --2- 172.21.15.130:0/819295001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40080fe8c0 0x7f40080fece0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.019+0000 7f400f74d700 1 --2- 172.21.15.130:0/819295001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4008101930 0x7f4008101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.019+0000 7f400f74d700 1 -- 172.21.15.130:0/819295001 >> 172.21.15.130:0/819295001 conn(0x7f40080fa1d0 msgr2=0x7f40080fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:37.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.019+0000 7f400f74d700 1 -- 172.21.15.130:0/819295001 shutdown_connections 2023-09-12T16:03:37.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.019+0000 7f400f74d700 1 -- 172.21.15.130:0/819295001 wait complete. 2023-09-12T16:03:37.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400f74d700 1 Processor -- start 2023-09-12T16:03:37.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400f74d700 1 -- start start 2023-09-12T16:03:37.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400f74d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40080fe8c0 0x7f40081962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400f74d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4008101930 0x7f4008196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400f74d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40081022d0 0x7f400819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400f74d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400819de50 con 0x7f4008101930 2023-09-12T16:03:37.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400f74d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400819df90 con 0x7f40081022d0 2023-09-12T16:03:37.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400f74d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400819e120 con 0x7f40080fe8c0 2023-09-12T16:03:37.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400cce8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4008101930 0x7f4008196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400cce8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4008101930 0x7f4008196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54812/0 (socket says 172.21.15.130:54812) 2023-09-12T16:03:37.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400cce8700 1 -- 172.21.15.130:0/913450345 learned_addr learned my addr 172.21.15.130:0/913450345 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:37.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400d4e9700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40080fe8c0 0x7f40081962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400cce8700 1 -- 172.21.15.130:0/913450345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40080fe8c0 msgr2=0x7f40081962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400cce8700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40080fe8c0 0x7f40081962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400cce8700 1 -- 172.21.15.130:0/913450345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40081022d0 msgr2=0x7f400819d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.020+0000 7f400dcea700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40081022d0 0x7f400819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.021+0000 7f400cce8700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40081022d0 0x7f400819d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.021+0000 7f400cce8700 1 -- 172.21.15.130:0/913450345 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0095f0 con 0x7f4008101930 2023-09-12T16:03:37.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.021+0000 7f400d4e9700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40080fe8c0 0x7f40081962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:37.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.021+0000 7f400cce8700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4008101930 0x7f4008196810 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f400400e9c0 tx=0x7f400400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:37.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.021+0000 7f3ffa7fc700 1 -- 172.21.15.130:0/913450345 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40040097b0 con 0x7f4008101930 2023-09-12T16:03:37.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.021+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400819e400 con 0x7f4008101930 2023-09-12T16:03:37.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.021+0000 7f3ffa7fc700 1 -- 172.21.15.130:0/913450345 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4004004cf0 con 0x7f4008101930 2023-09-12T16:03:37.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.021+0000 7f3ffa7fc700 1 -- 172.21.15.130:0/913450345 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4004005700 con 0x7f4008101930 2023-09-12T16:03:37.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.021+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400819e910 con 0x7f4008101930 2023-09-12T16:03:37.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.022+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40081076c0 con 0x7f4008101930 2023-09-12T16:03:37.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.023+0000 7f3ffa7fc700 1 -- 172.21.15.130:0/913450345 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f400400cbd0 con 0x7f4008101930 2023-09-12T16:03:37.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.023+0000 7f3ffa7fc700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3ff40a06d0 0x7f3ff40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.024+0000 7f400d4e9700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3ff40a06d0 0x7f3ff40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.024+0000 7f3ffa7fc700 1 -- 172.21.15.130:0/913450345 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4004014070 con 0x7f4008101930 2023-09-12T16:03:37.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.024+0000 7f400d4e9700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3ff40a06d0 0x7f3ff40a2b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00b2e0 tx=0x7f3ffc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:37.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.028+0000 7f3ffa7fc700 1 -- 172.21.15.130:0/913450345 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4004095ae0 con 0x7f4008101930 2023-09-12T16:03:37.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.219+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f400802ce30 con 0x7f4008101930 2023-09-12T16:03:37.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.219+0000 7f3ffa7fc700 1 -- 172.21.15.130:0/913450345 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4004004e60 con 0x7f4008101930 2023-09-12T16:03:37.220 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:03:37.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3ff40a06d0 msgr2=0x7f3ff40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3ff40a06d0 0x7f3ff40a2b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00b2e0 tx=0x7f3ffc005fb0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4008101930 msgr2=0x7f4008196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4008101930 0x7f4008196810 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f400400e9c0 tx=0x7f400400ed80 comp rx=0 tx=0).stop 2023-09-12T16:03:37.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 shutdown_connections 2023-09-12T16:03:37.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f3ff40a06d0 0x7f3ff40a2b90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40081022d0 0x7f400819d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40080fe8c0 0x7f40081962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 --2- 172.21.15.130:0/913450345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4008101930 0x7f4008196810 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 >> 172.21.15.130:0/913450345 conn(0x7f40080fa1d0 msgr2=0x7f40080fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:37.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 shutdown_connections 2023-09-12T16:03:37.229 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.221+0000 7f400f74d700 1 -- 172.21.15.130:0/913450345 wait complete. 2023-09-12T16:03:37.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:36 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/244779516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:37.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:36 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1189273197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:37.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:36 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/244779516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:37.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:36 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1189273197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:37.246 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:03:37.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:36 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/244779516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:37.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:36 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1189273197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:37.377 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.376+0000 7f5ab6301700 1 -- 172.21.15.130:0/3767545332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 msgr2=0x7f5ab0100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.377 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.376+0000 7f5ab6301700 1 --2- 172.21.15.130:0/3767545332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 0x7f5ab0100b70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40099c0 tx=0x7f5aa4009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.378 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.376+0000 7f5ab6301700 1 -- 172.21.15.130:0/3767545332 shutdown_connections 2023-09-12T16:03:37.378 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.376+0000 7f5ab6301700 1 --2- 172.21.15.130:0/3767545332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ab01010b0 0x7f5ab0107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.378 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.376+0000 7f5ab6301700 1 --2- 172.21.15.130:0/3767545332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab00ff520 0x7f5ab00ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.378 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.376+0000 7f5ab6301700 1 --2- 172.21.15.130:0/3767545332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 0x7f5ab0100b70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.376+0000 7f5ab6301700 1 -- 172.21.15.130:0/3767545332 >> 172.21.15.130:0/3767545332 conn(0x7f5ab00faa90 msgr2=0x7f5ab00fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:37.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.376+0000 7f5ab6301700 1 -- 172.21.15.130:0/3767545332 shutdown_connections 2023-09-12T16:03:37.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.377+0000 7f5ab6301700 1 -- 172.21.15.130:0/3767545332 wait complete. 2023-09-12T16:03:37.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.377+0000 7f5ab6301700 1 Processor -- start 2023-09-12T16:03:37.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.377+0000 7f5ab6301700 1 -- start start 2023-09-12T16:03:37.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5ab6301700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ab00ff520 0x7f5ab00692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5ab6301700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 0x7f5ab019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5ab6301700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab01010b0 0x7f5ab019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5ab6301700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab019de80 con 0x7f5ab0100710 2023-09-12T16:03:37.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5ab6301700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab019dff0 con 0x7f5ab01010b0 2023-09-12T16:03:37.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5ab6301700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab019e1b0 con 0x7f5ab00ff520 2023-09-12T16:03:37.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaf7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 0x7f5ab019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaf7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 0x7f5ab019b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54826/0 (socket says 172.21.15.130:54826) 2023-09-12T16:03:37.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaf7fe700 1 -- 172.21.15.130:0/4207012359 learned_addr learned my addr 172.21.15.130:0/4207012359 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:37.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaffff700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ab00ff520 0x7f5ab00692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5ab489e700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab01010b0 0x7f5ab019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaf7fe700 1 -- 172.21.15.130:0/4207012359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ab00ff520 msgr2=0x7f5ab00692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaf7fe700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ab00ff520 0x7f5ab00692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaf7fe700 1 -- 172.21.15.130:0/4207012359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab01010b0 msgr2=0x7f5ab019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaf7fe700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab01010b0 0x7f5ab019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.387 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaf7fe700 1 -- 172.21.15.130:0/4207012359 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa40095f0 con 0x7f5ab0100710 2023-09-12T16:03:37.387 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5ab489e700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab01010b0 0x7f5ab019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:37.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aaf7fe700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 0x7f5ab019b060 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009990 tx=0x7f5aa400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:37.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aad7fa700 1 -- 172.21.15.130:0/4207012359 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa401d070 con 0x7f5ab0100710 2023-09-12T16:03:37.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab019e430 con 0x7f5ab0100710 2023-09-12T16:03:37.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aad7fa700 1 -- 172.21.15.130:0/4207012359 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5aa40054f0 con 0x7f5ab0100710 2023-09-12T16:03:37.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.378+0000 7f5aad7fa700 1 -- 172.21.15.130:0/4207012359 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4004c70 con 0x7f5ab0100710 2023-09-12T16:03:37.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.379+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab019e8b0 con 0x7f5ab0100710 2023-09-12T16:03:37.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.379+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab01064a0 con 0x7f5ab0100710 2023-09-12T16:03:37.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.381+0000 7f5aad7fa700 1 -- 172.21.15.130:0/4207012359 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f5aa400fc30 con 0x7f5ab0100710 2023-09-12T16:03:37.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.381+0000 7f5aad7fa700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5a980a06d0 0x7f5a980a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.381+0000 7f5aaffff700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5a980a06d0 0x7f5a980a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.381+0000 7f5aad7fa700 1 -- 172.21.15.130:0/4207012359 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5aa40cd2e0 con 0x7f5ab0100710 2023-09-12T16:03:37.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.381+0000 7f5aaffff700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5a980a06d0 0x7f5a980a2b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0100570 tx=0x7f5aa0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:37.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.384+0000 7f5aad7fa700 1 -- 172.21.15.130:0/4207012359 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5aa4096ad0 con 0x7f5ab0100710 2023-09-12T16:03:37.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.573+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ab010ad90 con 0x7f5ab0100710 2023-09-12T16:03:37.575 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.573+0000 7f5aad7fa700 1 -- 172.21.15.130:0/4207012359 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5aa4026020 con 0x7f5ab0100710 2023-09-12T16:03:37.575 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:37.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5a980a06d0 msgr2=0x7f5a980a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5a980a06d0 0x7f5a980a2b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0100570 tx=0x7f5aa0009260 comp rx=0 tx=0).stop 2023-09-12T16:03:37.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 msgr2=0x7f5ab019b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 0x7f5ab019b060 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009990 tx=0x7f5aa400f740 comp rx=0 tx=0).stop 2023-09-12T16:03:37.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 shutdown_connections 2023-09-12T16:03:37.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5a980a06d0 0x7f5a980a2b90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab01010b0 0x7f5ab019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ab00ff520 0x7f5ab00692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 --2- 172.21.15.130:0/4207012359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ab0100710 0x7f5ab019b060 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 >> 172.21.15.130:0/4207012359 conn(0x7f5ab00faa90 msgr2=0x7f5ab00fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:37.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 shutdown_connections 2023-09-12T16:03:37.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.576+0000 7f5ab6301700 1 -- 172.21.15.130:0/4207012359 wait complete. 2023-09-12T16:03:37.589 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:03:37.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.714+0000 7f23a1425700 1 -- 172.21.15.130:0/1968265793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c101130 msgr2=0x7f239c103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.714+0000 7f23a1425700 1 --2- 172.21.15.130:0/1968265793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c101130 0x7f239c103520 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f238c009990 tx=0x7f238c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.714+0000 7f23a1425700 1 -- 172.21.15.130:0/1968265793 shutdown_connections 2023-09-12T16:03:37.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.714+0000 7f23a1425700 1 --2- 172.21.15.130:0/1968265793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f239c106390 0x7f239c108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.714+0000 7f23a1425700 1 --2- 172.21.15.130:0/1968265793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c101130 0x7f239c103520 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.714+0000 7f23a1425700 1 --2- 172.21.15.130:0/1968265793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f239c103a60 0x7f239c105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.714+0000 7f23a1425700 1 -- 172.21.15.130:0/1968265793 >> 172.21.15.130:0/1968265793 conn(0x7f239c0faac0 msgr2=0x7f239c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:37.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.714+0000 7f23a1425700 1 -- 172.21.15.130:0/1968265793 shutdown_connections 2023-09-12T16:03:37.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.714+0000 7f23a1425700 1 -- 172.21.15.130:0/1968265793 wait complete. 2023-09-12T16:03:37.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f23a1425700 1 Processor -- start 2023-09-12T16:03:37.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f23a1425700 1 -- start start 2023-09-12T16:03:37.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f23a1425700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f239c101130 0x7f239c1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f23a1425700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f239c103a60 0x7f239c1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f23a1425700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c106390 0x7f239c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f23a1425700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c19de30 con 0x7f239c106390 2023-09-12T16:03:37.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f23a1425700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c19df70 con 0x7f239c101130 2023-09-12T16:03:37.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f23a1425700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c19e100 con 0x7f239c103a60 2023-09-12T16:03:37.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f239b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c106390 0x7f239c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.715+0000 7f239b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c106390 0x7f239c19d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54842/0 (socket says 172.21.15.130:54842) 2023-09-12T16:03:37.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f239b7fe700 1 -- 172.21.15.130:0/2424883474 learned_addr learned my addr 172.21.15.130:0/2424883474 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:37.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f239a7fc700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f239c103a60 0x7f239c1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f239affd700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f239c101130 0x7f239c1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f239b7fe700 1 -- 172.21.15.130:0/2424883474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f239c103a60 msgr2=0x7f239c1967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f239b7fe700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f239c103a60 0x7f239c1967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f239b7fe700 1 -- 172.21.15.130:0/2424883474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f239c101130 msgr2=0x7f239c1962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f239b7fe700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f239c101130 0x7f239c1962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f239b7fe700 1 -- 172.21.15.130:0/2424883474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c0095f0 con 0x7f239c106390 2023-09-12T16:03:37.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f239b7fe700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c106390 0x7f239c19d860 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f239400ada0 tx=0x7f239400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:37.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f2383fff700 1 -- 172.21.15.130:0/2424883474 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239400f070 con 0x7f239c106390 2023-09-12T16:03:37.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239c19e3e0 con 0x7f239c106390 2023-09-12T16:03:37.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f2383fff700 1 -- 172.21.15.130:0/2424883474 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2394007b30 con 0x7f239c106390 2023-09-12T16:03:37.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f2383fff700 1 -- 172.21.15.130:0/2424883474 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23940053e0 con 0x7f239c106390 2023-09-12T16:03:37.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.716+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239c19e8a0 con 0x7f239c106390 2023-09-12T16:03:37.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.717+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239c0fc640 con 0x7f239c106390 2023-09-12T16:03:37.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.718+0000 7f2383fff700 1 -- 172.21.15.130:0/2424883474 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f239400c980 con 0x7f239c106390 2023-09-12T16:03:37.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.719+0000 7f2383fff700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23840a0750 0x7f23840a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:37.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.719+0000 7f239affd700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23840a0750 0x7f23840a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:37.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.719+0000 7f2383fff700 1 -- 172.21.15.130:0/2424883474 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f23940cbc60 con 0x7f239c106390 2023-09-12T16:03:37.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.719+0000 7f239affd700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23840a0750 0x7f23840a2c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f238c006010 tx=0x7f238c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:37.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.722+0000 7f2383fff700 1 -- 172.21.15.130:0/2424883474 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2394095350 con 0x7f239c106390 2023-09-12T16:03:37.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.913+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f239c10bfa0 con 0x7f239c106390 2023-09-12T16:03:37.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.914+0000 7f2383fff700 1 -- 172.21.15.130:0/2424883474 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f239c10bfa0 con 0x7f239c106390 2023-09-12T16:03:37.915 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:03:37.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23840a0750 msgr2=0x7f23840a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23840a0750 0x7f23840a2c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f238c006010 tx=0x7f238c00b540 comp rx=0 tx=0).stop 2023-09-12T16:03:37.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c106390 msgr2=0x7f239c19d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:37.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c106390 0x7f239c19d860 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f239400ada0 tx=0x7f239400c330 comp rx=0 tx=0).stop 2023-09-12T16:03:37.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 shutdown_connections 2023-09-12T16:03:37.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23840a0750 0x7f23840a2c10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f239c106390 0x7f239c19d860 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f239c101130 0x7f239c1962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 --2- 172.21.15.130:0/2424883474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f239c103a60 0x7f239c1967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:37.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 >> 172.21.15.130:0/2424883474 conn(0x7f239c0faac0 msgr2=0x7f239c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:37.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 shutdown_connections 2023-09-12T16:03:37.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:37.916+0000 7f23a1425700 1 -- 172.21.15.130:0/2424883474 wait complete. 2023-09-12T16:03:37.929 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:03:38.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.052+0000 7f9134490700 1 -- 172.21.15.130:0/2953968110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c101070 msgr2=0x7f912c103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.052+0000 7f9134490700 1 --2- 172.21.15.130:0/2953968110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c101070 0x7f912c103460 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9128009990 tx=0x7f9128009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 -- 172.21.15.130:0/2953968110 shutdown_connections 2023-09-12T16:03:38.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 --2- 172.21.15.130:0/2953968110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f912c1062d0 0x7f912c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 --2- 172.21.15.130:0/2953968110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c101070 0x7f912c103460 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 --2- 172.21.15.130:0/2953968110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c1039a0 0x7f912c105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 -- 172.21.15.130:0/2953968110 >> 172.21.15.130:0/2953968110 conn(0x7f912c0faa60 msgr2=0x7f912c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:38.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 -- 172.21.15.130:0/2953968110 shutdown_connections 2023-09-12T16:03:38.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 -- 172.21.15.130:0/2953968110 wait complete. 2023-09-12T16:03:38.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 Processor -- start 2023-09-12T16:03:38.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 -- start start 2023-09-12T16:03:38.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f912c101070 0x7f912c196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c1039a0 0x7f912c196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c1062d0 0x7f912c19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19deb0 con 0x7f912c1062d0 2023-09-12T16:03:38.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19dff0 con 0x7f912c101070 2023-09-12T16:03:38.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9134490700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19e180 con 0x7f912c1039a0 2023-09-12T16:03:38.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9132a2d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c1062d0 0x7f912c19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9132a2d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c1062d0 0x7f912c19d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54856/0 (socket says 172.21.15.130:54856) 2023-09-12T16:03:38.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9132a2d700 1 -- 172.21.15.130:0/2622434232 learned_addr learned my addr 172.21.15.130:0/2622434232 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:38.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.053+0000 7f9131a2b700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c1039a0 0x7f912c196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f913222c700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f912c101070 0x7f912c196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f9132a2d700 1 -- 172.21.15.130:0/2622434232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c1039a0 msgr2=0x7f912c196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f9132a2d700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c1039a0 0x7f912c196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f9132a2d700 1 -- 172.21.15.130:0/2622434232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f912c101070 msgr2=0x7f912c196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f9132a2d700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f912c101070 0x7f912c196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f9132a2d700 1 -- 172.21.15.130:0/2622434232 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280095f0 con 0x7f912c1062d0 2023-09-12T16:03:38.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f913222c700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f912c101070 0x7f912c196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:38.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f9132a2d700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c1062d0 0x7f912c19d8e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f912400ada0 tx=0x7f912400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:38.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f91237fe700 1 -- 172.21.15.130:0/2622434232 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912400f070 con 0x7f912c1062d0 2023-09-12T16:03:38.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f91237fe700 1 -- 172.21.15.130:0/2622434232 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9124007b30 con 0x7f912c1062d0 2023-09-12T16:03:38.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c19e460 con 0x7f912c1062d0 2023-09-12T16:03:38.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f91237fe700 1 -- 172.21.15.130:0/2622434232 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240053e0 con 0x7f912c1062d0 2023-09-12T16:03:38.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.054+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c19e920 con 0x7f912c1062d0 2023-09-12T16:03:38.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.055+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c0fc5e0 con 0x7f912c1062d0 2023-09-12T16:03:38.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.056+0000 7f91237fe700 1 -- 172.21.15.130:0/2622434232 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f912400c980 con 0x7f912c1062d0 2023-09-12T16:03:38.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.057+0000 7f91237fe700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91180a06d0 0x7f91180a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.057+0000 7f913222c700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91180a06d0 0x7f91180a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.057+0000 7f91237fe700 1 -- 172.21.15.130:0/2622434232 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f91240cbc00 con 0x7f912c1062d0 2023-09-12T16:03:38.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.057+0000 7f913222c700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91180a06d0 0x7f91180a2b90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9128005f50 tx=0x7f9128005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:38.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.060+0000 7f91237fe700 1 -- 172.21.15.130:0/2622434232 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9124095350 con 0x7f912c1062d0 2023-09-12T16:03:38.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:37 smithi130 ceph-mon[98670]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-12T16:03:38.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:37 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3996984709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:38.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:37 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/913450345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:38.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:37 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4207012359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:38.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:37 smithi159 ceph-mon[102522]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-12T16:03:38.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:37 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3996984709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:38.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:37 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/913450345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:38.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:37 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4207012359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:38.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:37 smithi203 ceph-mon[103431]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-09-12T16:03:38.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:37 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3996984709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:37 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/913450345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:38.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:37 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4207012359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:38.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.250+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f912c10c090 con 0x7f912c1062d0 2023-09-12T16:03:38.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.250+0000 7f91237fe700 1 -- 172.21.15.130:0/2622434232 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f912401a310 con 0x7f912c1062d0 2023-09-12T16:03:38.251 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:38.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.252+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91180a06d0 msgr2=0x7f91180a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.252+0000 7f9134490700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91180a06d0 0x7f91180a2b90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9128005f50 tx=0x7f9128005e80 comp rx=0 tx=0).stop 2023-09-12T16:03:38.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.252+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c1062d0 msgr2=0x7f912c19d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.252+0000 7f9134490700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c1062d0 0x7f912c19d8e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f912400ada0 tx=0x7f912400c330 comp rx=0 tx=0).stop 2023-09-12T16:03:38.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.252+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 shutdown_connections 2023-09-12T16:03:38.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.252+0000 7f9134490700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f91180a06d0 0x7f91180a2b90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.252+0000 7f9134490700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f912c1062d0 0x7f912c19d8e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.253+0000 7f9134490700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f912c101070 0x7f912c196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.253+0000 7f9134490700 1 --2- 172.21.15.130:0/2622434232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c1039a0 0x7f912c196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.253+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 >> 172.21.15.130:0/2622434232 conn(0x7f912c0faa60 msgr2=0x7f912c101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:38.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.253+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 shutdown_connections 2023-09-12T16:03:38.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.253+0000 7f9134490700 1 -- 172.21.15.130:0/2622434232 wait complete. 2023-09-12T16:03:38.265 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:03:38.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.387+0000 7f87d2dbf700 1 -- 172.21.15.130:0/2675836126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc0ff750 msgr2=0x7f87cc0ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.387+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/2675836126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc0ff750 0x7f87cc0ffbd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009990 tx=0x7f87b8009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.387+0000 7f87d2dbf700 1 -- 172.21.15.130:0/2675836126 shutdown_connections 2023-09-12T16:03:38.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.387+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/2675836126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87cc100110 0x7f87cc1079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.387+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/2675836126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87cc100b00 0x7f87cc100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.387+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/2675836126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc0ff750 0x7f87cc0ffbd0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.387+0000 7f87d2dbf700 1 -- 172.21.15.130:0/2675836126 >> 172.21.15.130:0/2675836126 conn(0x7f87cc0fa1c0 msgr2=0x7f87cc0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:38.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.387+0000 7f87d2dbf700 1 -- 172.21.15.130:0/2675836126 shutdown_connections 2023-09-12T16:03:38.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.387+0000 7f87d2dbf700 1 -- 172.21.15.130:0/2675836126 wait complete. 2023-09-12T16:03:38.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d2dbf700 1 Processor -- start 2023-09-12T16:03:38.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d2dbf700 1 -- start start 2023-09-12T16:03:38.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d2dbf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87cc0ff750 0x7f87cc195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d2dbf700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87cc100110 0x7f87cc195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d2dbf700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc100b00 0x7f87cc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d2dbf700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc19ccb0 con 0x7f87cc100b00 2023-09-12T16:03:38.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d2dbf700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc19cdf0 con 0x7f87cc100110 2023-09-12T16:03:38.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d2dbf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc19cf80 con 0x7f87cc0ff750 2023-09-12T16:03:38.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d135c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc100b00 0x7f87cc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d135c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc100b00 0x7f87cc19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54870/0 (socket says 172.21.15.130:54870) 2023-09-12T16:03:38.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d135c700 1 -- 172.21.15.130:0/3422461010 learned_addr learned my addr 172.21.15.130:0/3422461010 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:38.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d0b5b700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87cc0ff750 0x7f87cc195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87cbfff700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87cc100110 0x7f87cc195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d135c700 1 -- 172.21.15.130:0/3422461010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87cc0ff750 msgr2=0x7f87cc195130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d135c700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87cc0ff750 0x7f87cc195130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d135c700 1 -- 172.21.15.130:0/3422461010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87cc100110 msgr2=0x7f87cc195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.388+0000 7f87d135c700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87cc100110 0x7f87cc195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.389+0000 7f87d135c700 1 -- 172.21.15.130:0/3422461010 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80095f0 con 0x7f87cc100b00 2023-09-12T16:03:38.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.389+0000 7f87d135c700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc100b00 0x7f87cc19c6e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007d70 tx=0x7f87c400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:38.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.389+0000 7f87c9ffb700 1 -- 172.21.15.130:0/3422461010 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c400ad40 con 0x7f87cc100b00 2023-09-12T16:03:38.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.389+0000 7f87c9ffb700 1 -- 172.21.15.130:0/3422461010 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87c40128e0 con 0x7f87cc100b00 2023-09-12T16:03:38.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.389+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc19d260 con 0x7f87cc100b00 2023-09-12T16:03:38.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.389+0000 7f87c9ffb700 1 -- 172.21.15.130:0/3422461010 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c40116b0 con 0x7f87cc100b00 2023-09-12T16:03:38.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.389+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc19d720 con 0x7f87cc100b00 2023-09-12T16:03:38.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.390+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc10ad70 con 0x7f87cc100b00 2023-09-12T16:03:38.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.391+0000 7f87c9ffb700 1 -- 172.21.15.130:0/3422461010 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f87c4012420 con 0x7f87cc100b00 2023-09-12T16:03:38.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.391+0000 7f87c9ffb700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87bc0a0710 0x7f87bc0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.392+0000 7f87c9ffb700 1 -- 172.21.15.130:0/3422461010 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f87c40cc170 con 0x7f87cc100b00 2023-09-12T16:03:38.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.392+0000 7f87d0b5b700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87bc0a0710 0x7f87bc0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.392+0000 7f87d0b5b700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87bc0a0710 0x7f87bc0a2bd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87cc067540 tx=0x7f87c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:38.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.396+0000 7f87c9ffb700 1 -- 172.21.15.130:0/3422461010 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f87c4095940 con 0x7f87cc100b00 2023-09-12T16:03:38.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.630+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87cc10af90 con 0x7f87cc100b00 2023-09-12T16:03:38.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.631+0000 7f87c9ffb700 1 -- 172.21.15.130:0/3422461010 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11342 (secure 0 0 0) 0x7f87c4098f60 con 0x7f87cc100b00 2023-09-12T16:03:38.632 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:38.632 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:25.646573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:03:38.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87bc0a0710 msgr2=0x7f87bc0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87bc0a0710 0x7f87bc0a2bd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87cc067540 tx=0x7f87c000b410 comp rx=0 tx=0).stop 2023-09-12T16:03:38.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc100b00 msgr2=0x7f87cc19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc100b00 0x7f87cc19c6e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007d70 tx=0x7f87c400a340 comp rx=0 tx=0).stop 2023-09-12T16:03:38.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 shutdown_connections 2023-09-12T16:03:38.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87bc0a0710 0x7f87bc0a2bd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87cc100b00 0x7f87cc19c6e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87cc0ff750 0x7f87cc195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 --2- 172.21.15.130:0/3422461010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87cc100110 0x7f87cc195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 >> 172.21.15.130:0/3422461010 conn(0x7f87cc0fa1c0 msgr2=0x7f87cc0fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:38.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 shutdown_connections 2023-09-12T16:03:38.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.633+0000 7f87d2dbf700 1 -- 172.21.15.130:0/3422461010 wait complete. 2023-09-12T16:03:38.637 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 24 2023-09-12T16:03:38.646 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:38.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.768+0000 7fc70b521700 1 -- 172.21.15.130:0/2501207345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 msgr2=0x7fc704105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.768+0000 7fc70b521700 1 --2- 172.21.15.130:0/2501207345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 0x7fc704105e70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc700009990 tx=0x7fc700009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.768+0000 7fc70b521700 1 -- 172.21.15.130:0/2501207345 shutdown_connections 2023-09-12T16:03:38.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.768+0000 7fc70b521700 1 --2- 172.21.15.130:0/2501207345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7041063b0 0x7fc7041087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.768+0000 7fc70b521700 1 --2- 172.21.15.130:0/2501207345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc704101150 0x7fc704103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.768+0000 7fc70b521700 1 --2- 172.21.15.130:0/2501207345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 0x7fc704105e70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.768+0000 7fc70b521700 1 -- 172.21.15.130:0/2501207345 >> 172.21.15.130:0/2501207345 conn(0x7fc7040faa80 msgr2=0x7fc7040fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:38.773 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.768+0000 7fc70b521700 1 -- 172.21.15.130:0/2501207345 shutdown_connections 2023-09-12T16:03:38.773 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.768+0000 7fc70b521700 1 -- 172.21.15.130:0/2501207345 wait complete. 2023-09-12T16:03:38.773 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc70b521700 1 Processor -- start 2023-09-12T16:03:38.773 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc70b521700 1 -- start start 2023-09-12T16:03:38.773 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc70b521700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc704101150 0x7fc704196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc70b521700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 0x7fc7041967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc70b521700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7041063b0 0x7fc70419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc70b521700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70419ddf0 con 0x7fc704103a80 2023-09-12T16:03:38.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc70b521700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70419df30 con 0x7fc704101150 2023-09-12T16:03:38.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc70b521700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70419e0c0 con 0x7fc7041063b0 2023-09-12T16:03:38.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc708abc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 0x7fc7041967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc708abc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 0x7fc7041967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54882/0 (socket says 172.21.15.130:54882) 2023-09-12T16:03:38.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc708abc700 1 -- 172.21.15.130:0/465647740 learned_addr learned my addr 172.21.15.130:0/465647740 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:38.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc709abe700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7041063b0 0x7fc70419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.769+0000 7fc7092bd700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc704101150 0x7fc704196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc708abc700 1 -- 172.21.15.130:0/465647740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7041063b0 msgr2=0x7fc70419d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc708abc700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7041063b0 0x7fc70419d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc708abc700 1 -- 172.21.15.130:0/465647740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc704101150 msgr2=0x7fc704196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc708abc700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc704101150 0x7fc704196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc708abc700 1 -- 172.21.15.130:0/465647740 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7000095f0 con 0x7fc704103a80 2023-09-12T16:03:38.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc709abe700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7041063b0 0x7fc70419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:38.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc708abc700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 0x7fc7041967b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc7000059d0 tx=0x7fc700005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:38.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc6fa7fc700 1 -- 172.21.15.130:0/465647740 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70001d070 con 0x7fc704103a80 2023-09-12T16:03:38.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70419e340 con 0x7fc704103a80 2023-09-12T16:03:38.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc6fa7fc700 1 -- 172.21.15.130:0/465647740 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc70000fc40 con 0x7fc704103a80 2023-09-12T16:03:38.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc6fa7fc700 1 -- 172.21.15.130:0/465647740 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700017650 con 0x7fc704103a80 2023-09-12T16:03:38.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.770+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70419e7c0 con 0x7fc704103a80 2023-09-12T16:03:38.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.771+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7040fc600 con 0x7fc704103a80 2023-09-12T16:03:38.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.772+0000 7fc6fa7fc700 1 -- 172.21.15.130:0/465647740 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fc700022470 con 0x7fc704103a80 2023-09-12T16:03:38.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.773+0000 7fc6fa7fc700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc6f00a0710 0x7fc6f00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:38.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.773+0000 7fc7092bd700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc6f00a0710 0x7fc6f00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:38.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.773+0000 7fc6fa7fc700 1 -- 172.21.15.130:0/465647740 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc7000cdd40 con 0x7fc704103a80 2023-09-12T16:03:38.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.773+0000 7fc7092bd700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc6f00a0710 0x7fc6f00a2bd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009bf0 tx=0x7fc6f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:38.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.776+0000 7fc6fa7fc700 1 -- 172.21.15.130:0/465647740 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc700097490 con 0x7fc704103a80 2023-09-12T16:03:38.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.969+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc70410c090 con 0x7fc704103a80 2023-09-12T16:03:38.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.971+0000 7fc6fa7fc700 1 -- 172.21.15.130:0/465647740 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24473 (secure 0 0 0) 0x7fc70009aab0 con 0x7fc704103a80 2023-09-12T16:03:38.971 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:38.972 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":88,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:51.826597+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:38.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc6f00a0710 msgr2=0x7fc6f00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc6f00a0710 0x7fc6f00a2bd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009bf0 tx=0x7fc6f4009260 comp rx=0 tx=0).stop 2023-09-12T16:03:38.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 msgr2=0x7fc7041967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:38.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 0x7fc7041967b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc7000059d0 tx=0x7fc700005ef0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 shutdown_connections 2023-09-12T16:03:38.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc6f00a0710 0x7fc6f00a2bd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7041063b0 0x7fc70419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc704101150 0x7fc704196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 --2- 172.21.15.130:0/465647740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc704103a80 0x7fc7041967b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:38.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 >> 172.21.15.130:0/465647740 conn(0x7fc7040faa80 msgr2=0x7fc704101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:38.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 shutdown_connections 2023-09-12T16:03:38.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:38.973+0000 7fc70b521700 1 -- 172.21.15.130:0/465647740 wait complete. 2023-09-12T16:03:38.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:38 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2424883474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:38.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:38 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2622434232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:38.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:38 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3422461010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:38.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-12T16:02:04.422106+0000', 'modified': '2023-09-12T16:02:25.646573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24593, 'mds_1': 24592, 'mds_2': 24574}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24533': {'gid': 24533, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/3667735298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 3667735298}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3667735298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24557': {'gid': 24557, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6837/3413713845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 3413713845}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 3413713845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24562': {'gid': 24562, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/1801270748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 1801270748}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 1801270748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24574': {'gid': 24574, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.203:6837/3950241499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 3950241499}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 3950241499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24592': {'gid': 24592, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.203:6839/2671073294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 2671073294}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 2671073294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6839/3094783118', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 3094783118}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 3094783118}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-12T16:03:38.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-09-12T16:03:38.988 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-12T16:03:39.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.109+0000 7ff362bb7700 1 -- 172.21.15.130:0/1722448689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 msgr2=0x7ff35c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.109+0000 7ff362bb7700 1 --2- 172.21.15.130:0/1722448689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 0x7ff35c100bb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff3480099c0 tx=0x7ff348009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.109+0000 7ff362bb7700 1 -- 172.21.15.130:0/1722448689 shutdown_connections 2023-09-12T16:03:39.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.109+0000 7ff362bb7700 1 --2- 172.21.15.130:0/1722448689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff35c1010f0 0x7ff35c1079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.109+0000 7ff362bb7700 1 --2- 172.21.15.130:0/1722448689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff35c0ff560 0x7ff35c0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.109+0000 7ff362bb7700 1 --2- 172.21.15.130:0/1722448689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 0x7ff35c100bb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.109+0000 7ff362bb7700 1 -- 172.21.15.130:0/1722448689 >> 172.21.15.130:0/1722448689 conn(0x7ff35c0faad0 msgr2=0x7ff35c0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.109+0000 7ff362bb7700 1 -- 172.21.15.130:0/1722448689 shutdown_connections 2023-09-12T16:03:39.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.109+0000 7ff362bb7700 1 -- 172.21.15.130:0/1722448689 wait complete. 2023-09-12T16:03:39.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff362bb7700 1 Processor -- start 2023-09-12T16:03:39.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff362bb7700 1 -- start start 2023-09-12T16:03:39.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff362bb7700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff35c0ff560 0x7ff35c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff362bb7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 0x7ff35c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff362bb7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff35c1010f0 0x7ff35c19d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff362bb7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c19dea0 con 0x7ff35c100750 2023-09-12T16:03:39.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff362bb7700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c19dfe0 con 0x7ff35c0ff560 2023-09-12T16:03:39.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff362bb7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c19e170 con 0x7ff35c1010f0 2023-09-12T16:03:39.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff35bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 0x7ff35c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff35bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 0x7ff35c196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54896/0 (socket says 172.21.15.130:54896) 2023-09-12T16:03:39.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff35bfff700 1 -- 172.21.15.130:0/2753455768 learned_addr learned my addr 172.21.15.130:0/2753455768 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff360953700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff35c0ff560 0x7ff35c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.110+0000 7ff361154700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff35c1010f0 0x7ff35c19d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff35bfff700 1 -- 172.21.15.130:0/2753455768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff35c1010f0 msgr2=0x7ff35c19d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff35bfff700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff35c1010f0 0x7ff35c19d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff35bfff700 1 -- 172.21.15.130:0/2753455768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff35c0ff560 msgr2=0x7ff35c196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff35bfff700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff35c0ff560 0x7ff35c196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff35bfff700 1 -- 172.21.15.130:0/2753455768 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3480095f0 con 0x7ff35c100750 2023-09-12T16:03:39.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff35bfff700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 0x7ff35c196860 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff348009990 tx=0x7ff34800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff359ffb700 1 -- 172.21.15.130:0/2753455768 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34801d070 con 0x7ff35c100750 2023-09-12T16:03:39.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff359ffb700 1 -- 172.21.15.130:0/2753455768 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3480054f0 con 0x7ff35c100750 2023-09-12T16:03:39.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff359ffb700 1 -- 172.21.15.130:0/2753455768 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348004bd0 con 0x7ff35c100750 2023-09-12T16:03:39.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff362bb7700 1 -- 172.21.15.130:0/2753455768 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35c19e3f0 con 0x7ff35c100750 2023-09-12T16:03:39.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.111+0000 7ff362bb7700 1 -- 172.21.15.130:0/2753455768 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c19e810 con 0x7ff35c100750 2023-09-12T16:03:39.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.113+0000 7ff359ffb700 1 -- 172.21.15.130:0/2753455768 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7ff34800fb80 con 0x7ff35c100750 2023-09-12T16:03:39.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.114+0000 7ff359ffb700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff34c0a0750 0x7ff34c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.114+0000 7ff360953700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff34c0a0750 0x7ff34c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.114+0000 7ff359ffb700 1 -- 172.21.15.130:0/2753455768 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3480cce90 con 0x7ff35c100750 2023-09-12T16:03:39.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.114+0000 7ff360953700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff34c0a0750 0x7ff34c0a2c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff35c1005b0 tx=0x7ff35000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.114+0000 7ff36ed7db80 1 Processor -- start 2023-09-12T16:03:39.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.114+0000 7ff36ed7db80 1 -- start start 2023-09-12T16:03:39.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 0x55b0c01ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 0x55b0c01fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c01526e0 con 0x55b0c01fe1e0 2023-09-12T16:03:39.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c0150000 con 0x55b0c01ff020 2023-09-12T16:03:39.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c014b8f0 con 0x55b0c01ff980 2023-09-12T16:03:39.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff361154700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff361154700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54906/0 (socket says 172.21.15.130:54906) 2023-09-12T16:03:39.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff361154700 1 -- 172.21.15.130:0/122745090 learned_addr learned my addr 172.21.15.130:0/122745090 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff35bfff700 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 0x55b0c01fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff360953700 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 0x55b0c01ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff361154700 1 -- 172.21.15.130:0/122745090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 msgr2=0x55b0c01fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff361154700 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 0x55b0c01fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff361154700 1 -- 172.21.15.130:0/122745090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 msgr2=0x55b0c01ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff361154700 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 0x55b0c01ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff361154700 1 -- 172.21.15.130:0/122745090 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0bff77d90 con 0x55b0c01fe1e0 2023-09-12T16:03:39.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff360953700 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 0x55b0c01ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:39.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff361154700 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff354009440 tx=0x7ff354009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e84bcd35f014e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff346ffd700 1 -- 172.21.15.130:0/122745090 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35400e070 con 0x55b0c01fe1e0 2023-09-12T16:03:39.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff346ffd700 1 -- 172.21.15.130:0/122745090 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff35400f040 con 0x55b0c01fe1e0 2023-09-12T16:03:39.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.115+0000 7ff346ffd700 1 -- 172.21.15.130:0/122745090 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354015b00 con 0x55b0c01fe1e0 2023-09-12T16:03:39.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 -- 172.21.15.130:0/122745090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01fe1e0 msgr2=0x55b0c01fe660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff354009440 tx=0x7ff354009920 comp rx=0 tx=0).stop 2023-09-12T16:03:39.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 -- 172.21.15.130:0/122745090 shutdown_connections 2023-09-12T16:03:39.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe660 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 0x55b0c01ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/122745090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 0x55b0c01fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 -- 172.21.15.130:0/122745090 >> 172.21.15.130:0/122745090 conn(0x55b0c016e040 msgr2=0x55b0c01f46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 -- 172.21.15.130:0/122745090 shutdown_connections 2023-09-12T16:03:39.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 -- 172.21.15.130:0/122745090 wait complete. 2023-09-12T16:03:39.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.116+0000 7ff36ed7db80 1 -- start start 2023-09-12T16:03:39.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff36ed7db80 1 Processor -- start 2023-09-12T16:03:39.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01fe1e0 0x55b0c0207990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c0207ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 0x55b0c020ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c00e9b20 con 0x55b0c01ff020 2023-09-12T16:03:39.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c00e9820 con 0x55b0c01fe1e0 2023-09-12T16:03:39.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bffcd8d0 con 0x55b0c01ff980 2023-09-12T16:03:39.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff36ed7db80 10 client.0 _ll_register_callbacks cb 0x55b0c0033a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:39.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff35bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c0207ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff35bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c0207ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54910/0 (socket says 172.21.15.130:54910) 2023-09-12T16:03:39.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff35bfff700 1 -- 172.21.15.130:0/3648217903 learned_addr learned my addr 172.21.15.130:0/3648217903 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff360953700 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01fe1e0 0x55b0c0207990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff361154700 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 0x55b0c020ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff35bfff700 1 -- 172.21.15.130:0/3648217903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 msgr2=0x55b0c020ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff35bfff700 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 0x55b0c020ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff35bfff700 1 -- 172.21.15.130:0/3648217903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01fe1e0 msgr2=0x55b0c0207990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff35bfff700 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01fe1e0 0x55b0c0207990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff35bfff700 1 -- 172.21.15.130:0/3648217903 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3540090f0 con 0x55b0c01ff020 2023-09-12T16:03:39.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff360953700 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01fe1e0 0x55b0c0207990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:39.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.117+0000 7ff35bfff700 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c0207ed0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff34800bf40 tx=0x7ff3480a0390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8c4b0540f4f1c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff33bfff700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:39.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff33bfff700 1 -- 172.21.15.130:0/3648217903 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348099a50 con 0x55b0c01ff020 2023-09-12T16:03:39.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff33bfff700 1 -- 172.21.15.130:0/3648217903 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff34809a360 con 0x55b0c01ff020 2023-09-12T16:03:39.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3648217903 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0c00a34a0 con 0x55b0c01ff020 2023-09-12T16:03:39.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff33bfff700 1 -- 172.21.15.130:0/3648217903 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34809e190 con 0x55b0c01ff020 2023-09-12T16:03:39.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff36ed7db80 10 client.15546 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:39.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3648217903 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b0bff60c10 con 0x55b0c01ff020 2023-09-12T16:03:39.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff33bfff700 1 -- 172.21.15.130:0/3648217903 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0c00a34a0 con 0x55b0c01ff020 2023-09-12T16:03:39.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff33bfff700 1 -- 172.21.15.130:0/3648217903 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff3480b8140 con 0x55b0c01ff020 2023-09-12T16:03:39.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff36ed7db80 10 client.15546 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:39.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff36ed7db80 10 client.15546 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-12T16:03:39.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff36ed7db80 20 client.15546 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:39.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x55b0c0212c70 0x55b0c0211e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff36ed7db80 4 client.15546 mds_command: new command op to 24574 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:03:39.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.118+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3648217903 --> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0c0160850 con 0x55b0c0212c70 2023-09-12T16:03:39.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.119+0000 7ff360953700 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x55b0c0212c70 0x55b0c0211e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.119+0000 7ff360953700 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x55b0c0212c70 0x55b0c0211e40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7ad4edba775d722 server_cookie=d537459bea9c5e4a in_seq=0 out_seq=0 2023-09-12T16:03:39.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.119+0000 7ff33bfff700 10 client.15546 ms_handle_connect on v2:172.21.15.203:6836/3950241499 2023-09-12T16:03:39.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff33bfff700 1 -- 172.21.15.130:0/3648217903 <== mds.2 v2:172.21.15.203:6836/3950241499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b0c0160850 con 0x55b0c0212c70 2023-09-12T16:03:39.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff33bfff700 10 client.15546 handle_command_reply: tid=0 2023-09-12T16:03:39.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 client.15546 shutdown 2023-09-12T16:03:39.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3648217903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 msgr2=0x55b0c0207ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c0207ed0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff34800bf40 tx=0x7ff3480a0390 comp rx=0 tx=0).stop 2023-09-12T16:03:39.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3648217903 shutdown_connections 2023-09-12T16:03:39.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x55b0c0212c70 0x55b0c0211e40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01ff980 0x55b0c020ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01fe1e0 0x55b0c0207990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff33bfff700 0 client.15546 ms_handle_reset on v2:172.21.15.203:6836/3950241499 2023-09-12T16:03:39.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3648217903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c0207ed0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3648217903 >> 172.21.15.130:0/3648217903 conn(0x55b0c016e040 msgr2=0x55b0c01f3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3648217903 shutdown_connections 2023-09-12T16:03:39.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3648217903 wait complete. 2023-09-12T16:03:39.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.120+0000 7ff36ed7db80 20 client.15546 trim_cache size 0 max 16384 2023-09-12T16:03:39.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.137+0000 7ff36ed7db80 1 Processor -- start 2023-09-12T16:03:39.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.137+0000 7ff36ed7db80 1 -- start start 2023-09-12T16:03:39.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.137+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.137+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 0x55b0c020eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.137+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff980 0x55b0c01f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.137+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c01f5f90 con 0x55b0c01ff980 2023-09-12T16:03:39.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.137+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c00f34d0 con 0x55b0c01ff020 2023-09-12T16:03:39.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.137+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c0114f70 con 0x55b0c01fe1e0 2023-09-12T16:03:39.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.137+0000 7ff361154700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff980 0x55b0c01f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff361154700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff980 0x55b0c01f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54920/0 (socket says 172.21.15.130:54920) 2023-09-12T16:03:39.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff361154700 1 -- 172.21.15.130:0/2076057512 learned_addr learned my addr 172.21.15.130:0/2076057512 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff360953700 1 --2- 172.21.15.130:0/2076057512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff35bfff700 1 --2- 172.21.15.130:0/2076057512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 0x55b0c020eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff361154700 1 -- 172.21.15.130:0/2076057512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 msgr2=0x55b0c01fe5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff361154700 1 --2- 172.21.15.130:0/2076057512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff361154700 1 -- 172.21.15.130:0/2076057512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 msgr2=0x55b0c020eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff361154700 1 --2- 172.21.15.130:0/2076057512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 0x55b0c020eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff361154700 1 -- 172.21.15.130:0/2076057512 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0bff9e200 con 0x55b0c01ff980 2023-09-12T16:03:39.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff361154700 1 --2- 172.21.15.130:0/2076057512 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff980 0x55b0c01f4640 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff3540093e0 tx=0x7ff354006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977c2ff7d6742c9e server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff3477fe700 1 -- 172.21.15.130:0/2076057512 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35400e040 con 0x55b0c01ff980 2023-09-12T16:03:39.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff3477fe700 1 -- 172.21.15.130:0/2076057512 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3540095e0 con 0x55b0c01ff980 2023-09-12T16:03:39.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.138+0000 7ff3477fe700 1 -- 172.21.15.130:0/2076057512 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3540137b0 con 0x55b0c01ff980 2023-09-12T16:03:39.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 -- 172.21.15.130:0/2076057512 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff980 msgr2=0x55b0c01f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/2076057512 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff980 0x55b0c01f4640 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff3540093e0 tx=0x7ff354006040 comp rx=0 tx=0).stop 2023-09-12T16:03:39.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 -- 172.21.15.130:0/2076057512 shutdown_connections 2023-09-12T16:03:39.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/2076057512 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff980 0x55b0c01f4640 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/2076057512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 0x55b0c01fe5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/2076057512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff020 0x55b0c020eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 -- 172.21.15.130:0/2076057512 >> 172.21.15.130:0/2076057512 conn(0x55b0c016e040 msgr2=0x55b0c02066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 -- 172.21.15.130:0/2076057512 shutdown_connections 2023-09-12T16:03:39.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 -- 172.21.15.130:0/2076057512 wait complete. 2023-09-12T16:03:39.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 -- start start 2023-09-12T16:03:39.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 Processor -- start 2023-09-12T16:03:39.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 0x55b0c020a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c020a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff980 0x55b0c0266cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c020f000 con 0x55b0c01ff020 2023-09-12T16:03:39.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c020f170 con 0x55b0c01ff980 2023-09-12T16:03:39.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.139+0000 7ff36ed7db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c020f330 con 0x55b0c01fe1e0 2023-09-12T16:03:39.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff36ed7db80 10 client.0 _ll_register_callbacks cb 0x7ff348099ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:39.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff35bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c020a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff35bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c020a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54924/0 (socket says 172.21.15.130:54924) 2023-09-12T16:03:39.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff35bfff700 1 -- 172.21.15.130:0/3538302921 learned_addr learned my addr 172.21.15.130:0/3538302921 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff361154700 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff980 0x55b0c0266cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff360953700 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 0x55b0c020a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff35bfff700 1 -- 172.21.15.130:0/3538302921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 msgr2=0x55b0c020a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff35bfff700 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 0x55b0c020a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff35bfff700 1 -- 172.21.15.130:0/3538302921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff980 msgr2=0x55b0c0266cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff35bfff700 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff980 0x55b0c0266cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff35bfff700 1 -- 172.21.15.130:0/3538302921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0bff77d90 con 0x55b0c01ff020 2023-09-12T16:03:39.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff35bfff700 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c020a5e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff3480b8470 tx=0x7ff348099a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4018a25f2246bc server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff33bfff700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:39.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff33bfff700 1 -- 172.21.15.130:0/3538302921 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480966a0 con 0x55b0c01ff020 2023-09-12T16:03:39.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff33bfff700 1 -- 172.21.15.130:0/3538302921 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff34809a360 con 0x55b0c01ff020 2023-09-12T16:03:39.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3538302921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff35400f040 con 0x55b0c01ff020 2023-09-12T16:03:39.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff33bfff700 1 -- 172.21.15.130:0/3538302921 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34809e190 con 0x55b0c01ff020 2023-09-12T16:03:39.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff36ed7db80 10 client.15552 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:39.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3538302921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b0bff76260 con 0x55b0c01ff020 2023-09-12T16:03:39.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.140+0000 7ff33bfff700 1 -- 172.21.15.130:0/3538302921 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff35400f040 con 0x55b0c01ff020 2023-09-12T16:03:39.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff33bfff700 1 -- 172.21.15.130:0/3538302921 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff3480cacf0 con 0x55b0c01ff020 2023-09-12T16:03:39.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff36ed7db80 10 client.15552 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:39.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff36ed7db80 10 client.15552 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-12T16:03:39.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff36ed7db80 20 client.15552 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:39.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x55b0c026bc60 0x55b0c0268290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff36ed7db80 4 client.15552 mds_command: new command op to 24574 tid=0[{"prefix": "status"}] 2023-09-12T16:03:39.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3538302921 --> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b0c01448d0 con 0x55b0c026bc60 2023-09-12T16:03:39.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff360953700 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x55b0c026bc60 0x55b0c0268290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff33bfff700 10 client.15552 ms_handle_connect on v2:172.21.15.203:6836/3950241499 2023-09-12T16:03:39.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.141+0000 7ff360953700 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x55b0c026bc60 0x55b0c0268290 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4483d593c4678403 server_cookie=a5d49a9f3a96c640 in_seq=0 out_seq=0 2023-09-12T16:03:39.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff33bfff700 1 -- 172.21.15.130:0/3538302921 <== mds.2 v2:172.21.15.203:6836/3950241499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b0c01448d0 con 0x55b0c026bc60 2023-09-12T16:03:39.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff33bfff700 10 client.15552 handle_command_reply: tid=0 2023-09-12T16:03:39.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 client.15552 shutdown 2023-09-12T16:03:39.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3538302921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 msgr2=0x55b0c020a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c020a5e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff3480b8470 tx=0x7ff348099a80 comp rx=0 tx=0).stop 2023-09-12T16:03:39.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3538302921 shutdown_connections 2023-09-12T16:03:39.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x55b0c026bc60 0x55b0c0268290 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c01ff980 0x55b0c0266cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff33bfff700 0 client.15552 ms_handle_reset on v2:172.21.15.203:6836/3950241499 2023-09-12T16:03:39.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0c01fe1e0 0x55b0c020a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 --2- 172.21.15.130:0/3538302921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c01ff020 0x55b0c020a5e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3538302921 >> 172.21.15.130:0/3538302921 conn(0x55b0c016e040 msgr2=0x55b0c01f4620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3538302921 shutdown_connections 2023-09-12T16:03:39.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 1 -- 172.21.15.130:0/3538302921 wait complete. 2023-09-12T16:03:39.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff36ed7db80 20 client.15552 trim_cache size 0 max 16384 2023-09-12T16:03:39.167 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:03:39.167 INFO:teuthology.orchestra.run.smithi130.stdout: "cluster_fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T16:03:39.167 INFO:teuthology.orchestra.run.smithi130.stdout: "whoami": 2, 2023-09-12T16:03:39.167 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 24574, 2023-09-12T16:03:39.167 INFO:teuthology.orchestra.run.smithi130.stdout: "want_state": "up:active", 2023-09-12T16:03:39.168 INFO:teuthology.orchestra.run.smithi130.stdout: "state": "up:active", 2023-09-12T16:03:39.168 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs", 2023-09-12T16:03:39.168 INFO:teuthology.orchestra.run.smithi130.stdout: "rank_uptime": 74.502670393000002, 2023-09-12T16:03:39.168 INFO:teuthology.orchestra.run.smithi130.stdout: "mdsmap_epoch": 24, 2023-09-12T16:03:39.168 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch": 88, 2023-09-12T16:03:39.169 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch_barrier": 86, 2023-09-12T16:03:39.169 INFO:teuthology.orchestra.run.smithi130.stdout: "uptime": 110.835510994 2023-09-12T16:03:39.169 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:03:39.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff362bb7700 1 -- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff34c0a0750 msgr2=0x7ff34c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.142+0000 7ff362bb7700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff34c0a0750 0x7ff34c0a2c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff35c1005b0 tx=0x7ff35000b410 comp rx=0 tx=0).stop 2023-09-12T16:03:39.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 -- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 msgr2=0x7ff35c196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 0x7ff35c196860 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff348009990 tx=0x7ff34800f690 comp rx=0 tx=0).stop 2023-09-12T16:03:39.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 -- 172.21.15.130:0/2753455768 shutdown_connections 2023-09-12T16:03:39.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff34c0a0750 0x7ff34c0a2c10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff35c1010f0 0x7ff35c19d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff35c0ff560 0x7ff35c196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 --2- 172.21.15.130:0/2753455768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff35c100750 0x7ff35c196860 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 -- 172.21.15.130:0/2753455768 >> 172.21.15.130:0/2753455768 conn(0x7ff35c0faad0 msgr2=0x7ff35c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 -- 172.21.15.130:0/2753455768 shutdown_connections 2023-09-12T16:03:39.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.143+0000 7ff362bb7700 1 -- 172.21.15.130:0/2753455768 wait complete. 2023-09-12T16:03:39.173 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-12T16:03:39.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:38 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2424883474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:39.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:38 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2622434232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:39.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:38 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3422461010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:39.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:38 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2424883474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:39.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:38 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2622434232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:39.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:38 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3422461010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:39.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.294+0000 7fd569110700 1 -- 172.21.15.130:0/189738173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd564100750 msgr2=0x7fd564100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.294+0000 7fd569110700 1 --2- 172.21.15.130:0/189738173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd564100750 0x7fd564100bb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd5540099c0 tx=0x7fd554009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.294+0000 7fd569110700 1 -- 172.21.15.130:0/189738173 shutdown_connections 2023-09-12T16:03:39.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.294+0000 7fd569110700 1 --2- 172.21.15.130:0/189738173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641010f0 0x7fd5641079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.294+0000 7fd569110700 1 --2- 172.21.15.130:0/189738173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5640ff560 0x7fd5640ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.294+0000 7fd569110700 1 --2- 172.21.15.130:0/189738173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd564100750 0x7fd564100bb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.294+0000 7fd569110700 1 -- 172.21.15.130:0/189738173 >> 172.21.15.130:0/189738173 conn(0x7fd5640faad0 msgr2=0x7fd5640fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.294+0000 7fd569110700 1 -- 172.21.15.130:0/189738173 shutdown_connections 2023-09-12T16:03:39.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.294+0000 7fd569110700 1 -- 172.21.15.130:0/189738173 wait complete. 2023-09-12T16:03:39.298 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd569110700 1 Processor -- start 2023-09-12T16:03:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd569110700 1 -- start start 2023-09-12T16:03:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd569110700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5640ff560 0x7fd564195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd569110700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd564100750 0x7fd564195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd569110700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641010f0 0x7fd56419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd569110700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56419cd80 con 0x7fd5640ff560 2023-09-12T16:03:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd569110700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56419cec0 con 0x7fd5641010f0 2023-09-12T16:03:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd569110700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56419d050 con 0x7fd564100750 2023-09-12T16:03:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd562d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5640ff560 0x7fd564195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd562d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5640ff560 0x7fd564195200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54936/0 (socket says 172.21.15.130:54936) 2023-09-12T16:03:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.295+0000 7fd562d9d700 1 -- 172.21.15.130:0/3429544376 learned_addr learned my addr 172.21.15.130:0/3429544376 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd56259c700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd564100750 0x7fd564195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd56359e700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641010f0 0x7fd56419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd562d9d700 1 -- 172.21.15.130:0/3429544376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd564100750 msgr2=0x7fd564195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd562d9d700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd564100750 0x7fd564195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd562d9d700 1 -- 172.21.15.130:0/3429544376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641010f0 msgr2=0x7fd56419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd562d9d700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641010f0 0x7fd56419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd562d9d700 1 -- 172.21.15.130:0/3429544376 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540095f0 con 0x7fd5640ff560 2023-09-12T16:03:39.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd562d9d700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5640ff560 0x7fd564195200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd55800d980 tx=0x7fd55800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd54bfff700 1 -- 172.21.15.130:0/3429544376 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800de60 con 0x7fd5640ff560 2023-09-12T16:03:39.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd569110700 1 -- 172.21.15.130:0/3429544376 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56419d330 con 0x7fd5640ff560 2023-09-12T16:03:39.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd54bfff700 1 -- 172.21.15.130:0/3429544376 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5580044e0 con 0x7fd5640ff560 2023-09-12T16:03:39.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd54bfff700 1 -- 172.21.15.130:0/3429544376 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558003be0 con 0x7fd5640ff560 2023-09-12T16:03:39.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.296+0000 7fd569110700 1 -- 172.21.15.130:0/3429544376 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56419d890 con 0x7fd5640ff560 2023-09-12T16:03:39.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.298+0000 7fd54bfff700 1 -- 172.21.15.130:0/3429544376 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fd558009bb0 con 0x7fd5640ff560 2023-09-12T16:03:39.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.299+0000 7fd54bfff700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd54c0a06d0 0x7fd54c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.299+0000 7fd56259c700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd54c0a06d0 0x7fd54c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.299+0000 7fd54bfff700 1 -- 172.21.15.130:0/3429544376 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd5580cbdc0 con 0x7fd5640ff560 2023-09-12T16:03:39.311 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.299+0000 7fd56259c700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd54c0a06d0 0x7fd54c0a2b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd554000c00 tx=0x7fd554005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.311 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.299+0000 7fd5752d6b80 1 Processor -- start 2023-09-12T16:03:39.311 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.299+0000 7fd5752d6b80 1 -- start start 2023-09-12T16:03:39.311 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4ed0 0x556b21eb52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.311 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21eb3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4080 0x556b21ebfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21e0c6e0 con 0x556b21eb4080 2023-09-12T16:03:39.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21e0a000 con 0x556b21eb36c0 2023-09-12T16:03:39.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21e058f0 con 0x556b21eb4ed0 2023-09-12T16:03:39.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4080 0x556b21ebfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4080 0x556b21ebfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54948/0 (socket says 172.21.15.130:54948) 2023-09-12T16:03:39.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56359e700 1 -- 172.21.15.130:0/2290964775 learned_addr learned my addr 172.21.15.130:0/2290964775 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd562d9d700 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4ed0 0x556b21eb52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56259c700 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21eb3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56359e700 1 -- 172.21.15.130:0/2290964775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4ed0 msgr2=0x556b21eb52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56359e700 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4ed0 0x556b21eb52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.314 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56359e700 1 -- 172.21.15.130:0/2290964775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 msgr2=0x556b21eb3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.314 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56359e700 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21eb3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.314 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56359e700 1 -- 172.21.15.130:0/2290964775 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b21c31d90 con 0x556b21eb4080 2023-09-12T16:03:39.314 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56259c700 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21eb3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:39.314 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd56359e700 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4080 0x556b21ebfd10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd55c007460 tx=0x7fd55c007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227cabbf43ed08e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd5497fa700 1 -- 172.21.15.130:0/2290964775 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c013600 con 0x556b21eb4080 2023-09-12T16:03:39.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd5497fa700 1 -- 172.21.15.130:0/2290964775 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd55c013c20 con 0x556b21eb4080 2023-09-12T16:03:39.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.300+0000 7fd5497fa700 1 -- 172.21.15.130:0/2290964775 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c012c70 con 0x556b21eb4080 2023-09-12T16:03:39.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2290964775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4080 msgr2=0x556b21ebfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.315 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4080 0x556b21ebfd10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd55c007460 tx=0x7fd55c007770 comp rx=0 tx=0).stop 2023-09-12T16:03:39.316 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2290964775 shutdown_connections 2023-09-12T16:03:39.316 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4080 0x556b21ebfd10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.316 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4ed0 0x556b21eb52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.316 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/2290964775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21eb3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2290964775 >> 172.21.15.130:0/2290964775 conn(0x556b21e28040 msgr2=0x556b21eae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2290964775 shutdown_connections 2023-09-12T16:03:39.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2290964775 wait complete. 2023-09-12T16:03:39.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 -- start start 2023-09-12T16:03:39.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.301+0000 7fd5752d6b80 1 Processor -- start 2023-09-12T16:03:39.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21ec1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21ec8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21da3b20 con 0x556b21eb4ed0 2023-09-12T16:03:39.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21da3820 con 0x556b21eb36c0 2023-09-12T16:03:39.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21c878d0 con 0x556b21eb4080 2023-09-12T16:03:39.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd5752d6b80 10 client.0 _ll_register_callbacks cb 0x556b21ceda30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:39.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21ec8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56259c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21ec8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54954/0 (socket says 172.21.15.130:54954) 2023-09-12T16:03:39.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd562d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21ec1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56259c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:34564/0 (socket says 172.21.15.130:34564) 2023-09-12T16:03:39.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56359e700 1 -- 172.21.15.130:0/3617502439 learned_addr learned my addr 172.21.15.130:0/3617502439 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56359e700 1 -- 172.21.15.130:0/3617502439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 msgr2=0x556b21ec1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56359e700 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56359e700 1 -- 172.21.15.130:0/3617502439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 msgr2=0x556b21ec1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56359e700 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21ec1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56359e700 1 -- 172.21.15.130:0/3617502439 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55c008cd0 con 0x556b21eb4ed0 2023-09-12T16:03:39.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56259c700 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:03:39.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd562d9d700 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21ec1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:03:39.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd56359e700 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21ec8ec0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0086f0 tx=0x7fd55c008bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b4cc5912d65166 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd546ff5700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:39.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.302+0000 7fd546ff5700 1 -- 172.21.15.130:0/3617502439 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c013600 con 0x556b21eb4ed0 2023-09-12T16:03:39.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd546ff5700 1 -- 172.21.15.130:0/3617502439 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd55c012e40 con 0x556b21eb4ed0 2023-09-12T16:03:39.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd5752d6b80 1 -- 172.21.15.130:0/3617502439 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b21d5d4a0 con 0x556b21eb4ed0 2023-09-12T16:03:39.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd546ff5700 1 -- 172.21.15.130:0/3617502439 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd55c013760 con 0x556b21eb4ed0 2023-09-12T16:03:39.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd5752d6b80 10 client.15564 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:39.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd5752d6b80 1 -- 172.21.15.130:0/3617502439 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556b21c1ac10 con 0x556b21eb4ed0 2023-09-12T16:03:39.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd546ff5700 1 -- 172.21.15.130:0/3617502439 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b21d5d4a0 con 0x556b21eb4ed0 2023-09-12T16:03:39.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd546ff5700 1 -- 172.21.15.130:0/3617502439 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd55c007490 con 0x556b21eb4ed0 2023-09-12T16:03:39.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd5752d6b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:39.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd5752d6b80 10 client.15564 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-12T16:03:39.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd5752d6b80 20 client.15564 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:39.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556b21ecce30 0x556b21ecbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd5752d6b80 4 client.15564 mds_command: new command op to 24592 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:03:39.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd5752d6b80 1 -- 172.21.15.130:0/3617502439 --> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b21e1a850 con 0x556b21ecce30 2023-09-12T16:03:39.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.303+0000 7fd562d9d700 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556b21ecce30 0x556b21ecbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.304+0000 7fd562d9d700 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556b21ecce30 0x556b21ecbe20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a63920c2484975c5 server_cookie=beef67d333580eae in_seq=0 out_seq=0 2023-09-12T16:03:39.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.304+0000 7fd546ff5700 10 client.15564 ms_handle_connect on v2:172.21.15.203:6838/2671073294 2023-09-12T16:03:39.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.304+0000 7fd546ff5700 1 -- 172.21.15.130:0/3617502439 <== mds.1 v2:172.21.15.203:6838/2671073294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556b21e1a850 con 0x556b21ecce30 2023-09-12T16:03:39.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.304+0000 7fd546ff5700 10 client.15564 handle_command_reply: tid=0 2023-09-12T16:03:39.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.304+0000 7fd5752d6b80 1 client.15564 shutdown 2023-09-12T16:03:39.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.304+0000 7fd5752d6b80 1 -- 172.21.15.130:0/3617502439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 msgr2=0x556b21ec8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.304+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21ec8ec0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0086f0 tx=0x7fd55c008bb0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd5752d6b80 1 -- 172.21.15.130:0/3617502439 shutdown_connections 2023-09-12T16:03:39.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21ec8ec0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556b21ecce30 0x556b21ecbe20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21ec1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd546ff5700 0 client.15564 ms_handle_reset on v2:172.21.15.203:6838/2671073294 2023-09-12T16:03:39.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/3617502439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd5752d6b80 1 -- 172.21.15.130:0/3617502439 >> 172.21.15.130:0/3617502439 conn(0x556b21e28040 msgr2=0x556b21eadd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd5752d6b80 1 -- 172.21.15.130:0/3617502439 shutdown_connections 2023-09-12T16:03:39.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd5752d6b80 1 -- 172.21.15.130:0/3617502439 wait complete. 2023-09-12T16:03:39.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.305+0000 7fd5752d6b80 20 client.15564 trim_cache size 0 max 16384 2023-09-12T16:03:39.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd5752d6b80 1 Processor -- start 2023-09-12T16:03:39.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd5752d6b80 1 -- start start 2023-09-12T16:03:39.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb36c0 0x556b21ec96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb4ed0 0x556b21ebf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21ec9320 con 0x556b21eb36c0 2023-09-12T16:03:39.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21ea8de0 con 0x556b21eb4ed0 2023-09-12T16:03:39.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21e1a850 con 0x556b21eb4080 2023-09-12T16:03:39.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd562d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb36c0 0x556b21ec96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd562d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb36c0 0x556b21ec96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54958/0 (socket says 172.21.15.130:54958) 2023-09-12T16:03:39.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd562d9d700 1 -- 172.21.15.130:0/2970912688 learned_addr learned my addr 172.21.15.130:0/2970912688 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd56359e700 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb4ed0 0x556b21ebf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd56259c700 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd562d9d700 1 -- 172.21.15.130:0/2970912688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 msgr2=0x556b21ec8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd562d9d700 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd562d9d700 1 -- 172.21.15.130:0/2970912688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb4ed0 msgr2=0x556b21ebf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd562d9d700 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb4ed0 0x556b21ebf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd562d9d700 1 -- 172.21.15.130:0/2970912688 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b21c58200 con 0x556b21eb36c0 2023-09-12T16:03:39.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd56259c700 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:39.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd562d9d700 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb36c0 0x556b21ec96f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd558095640 tx=0x7fd5580cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cb4ba6ca7b0cba server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd549ffb700 1 -- 172.21.15.130:0/2970912688 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55809cf70 con 0x556b21eb36c0 2023-09-12T16:03:39.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd549ffb700 1 -- 172.21.15.130:0/2970912688 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5580b0070 con 0x556b21eb36c0 2023-09-12T16:03:39.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.322+0000 7fd549ffb700 1 -- 172.21.15.130:0/2970912688 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558098890 con 0x556b21eb36c0 2023-09-12T16:03:39.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2970912688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb36c0 msgr2=0x556b21ec96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb36c0 0x556b21ec96f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd558095640 tx=0x7fd5580cb040 comp rx=0 tx=0).stop 2023-09-12T16:03:39.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2970912688 shutdown_connections 2023-09-12T16:03:39.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb4ed0 0x556b21ebf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb36c0 0x556b21ec96f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/2970912688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2970912688 >> 172.21.15.130:0/2970912688 conn(0x556b21e28040 msgr2=0x556b21ecbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2970912688 shutdown_connections 2023-09-12T16:03:39.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 -- 172.21.15.130:0/2970912688 wait complete. 2023-09-12T16:03:39.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 -- start start 2023-09-12T16:03:39.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.323+0000 7fd5752d6b80 1 Processor -- start 2023-09-12T16:03:39.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21ec5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21f20e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21ec6560 con 0x556b21eb4ed0 2023-09-12T16:03:39.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21ec66d0 con 0x556b21eb36c0 2023-09-12T16:03:39.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b21f213c0 con 0x556b21eb4080 2023-09-12T16:03:39.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 10 client.0 _ll_register_callbacks cb 0x7fd558094e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:39.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21f20e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56359e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21f20e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54968/0 (socket says 172.21.15.130:54968) 2023-09-12T16:03:39.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56359e700 1 -- 172.21.15.130:0/1112573087 learned_addr learned my addr 172.21.15.130:0/1112573087 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56259c700 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd562d9d700 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21ec5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56359e700 1 -- 172.21.15.130:0/1112573087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 msgr2=0x556b21ec6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56359e700 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56359e700 1 -- 172.21.15.130:0/1112573087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 msgr2=0x556b21ec5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56359e700 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21ec5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56359e700 1 -- 172.21.15.130:0/1112573087 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56419d890 con 0x556b21eb4ed0 2023-09-12T16:03:39.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56259c700 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:39.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd56359e700 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21f20e80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd55c01fe30 tx=0x7fd55c01e390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8fc85389145c4a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd546ff5700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:39.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd546ff5700 1 -- 172.21.15.130:0/1112573087 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c0089c0 con 0x556b21eb4ed0 2023-09-12T16:03:39.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd546ff5700 1 -- 172.21.15.130:0/1112573087 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd55c004a90 con 0x556b21eb4ed0 2023-09-12T16:03:39.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 1 -- 172.21.15.130:0/1112573087 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd55c013c20 con 0x556b21eb4ed0 2023-09-12T16:03:39.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd546ff5700 1 -- 172.21.15.130:0/1112573087 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd55c004c50 con 0x556b21eb4ed0 2023-09-12T16:03:39.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 10 client.15570 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:39.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.324+0000 7fd5752d6b80 1 -- 172.21.15.130:0/1112573087 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556b21c30260 con 0x556b21eb4ed0 2023-09-12T16:03:39.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd546ff5700 1 -- 172.21.15.130:0/1112573087 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd55c013c20 con 0x556b21eb4ed0 2023-09-12T16:03:39.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd546ff5700 1 -- 172.21.15.130:0/1112573087 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd55c01e990 con 0x556b21eb4ed0 2023-09-12T16:03:39.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd5752d6b80 10 client.15570 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:39.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd5752d6b80 10 client.15570 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-12T16:03:39.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd5752d6b80 20 client.15570 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:39.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556b21f26280 0x556b21f22720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd5752d6b80 4 client.15570 mds_command: new command op to 24592 tid=0[{"prefix": "status"}] 2023-09-12T16:03:39.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd5752d6b80 1 -- 172.21.15.130:0/1112573087 --> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556b21dfe8d0 con 0x556b21f26280 2023-09-12T16:03:39.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd562d9d700 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556b21f26280 0x556b21f22720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd546ff5700 10 client.15570 ms_handle_connect on v2:172.21.15.203:6838/2671073294 2023-09-12T16:03:39.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.325+0000 7fd562d9d700 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556b21f26280 0x556b21f22720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14243c165d393851 server_cookie=8e05d9a58c12f790 in_seq=0 out_seq=0 2023-09-12T16:03:39.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd546ff5700 1 -- 172.21.15.130:0/1112573087 <== mds.1 v2:172.21.15.203:6838/2671073294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x556b21dfe8d0 con 0x556b21f26280 2023-09-12T16:03:39.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd546ff5700 10 client.15570 handle_command_reply: tid=0 2023-09-12T16:03:39.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 client.15570 shutdown 2023-09-12T16:03:39.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 -- 172.21.15.130:0/1112573087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 msgr2=0x556b21f20e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21f20e80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd55c01fe30 tx=0x7fd55c01e390 comp rx=0 tx=0).stop 2023-09-12T16:03:39.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 -- 172.21.15.130:0/1112573087 shutdown_connections 2023-09-12T16:03:39.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556b21eb4ed0 0x556b21f20e80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556b21f26280 0x556b21f22720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b21eb36c0 0x556b21ec5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd546ff5700 0 client.15570 ms_handle_reset on v2:172.21.15.203:6838/2671073294 2023-09-12T16:03:39.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 --2- 172.21.15.130:0/1112573087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556b21eb4080 0x556b21ec6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 -- 172.21.15.130:0/1112573087 >> 172.21.15.130:0/1112573087 conn(0x556b21e28040 msgr2=0x556b21ec96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 -- 172.21.15.130:0/1112573087 shutdown_connections 2023-09-12T16:03:39.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 1 -- 172.21.15.130:0/1112573087 wait complete. 2023-09-12T16:03:39.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd5752d6b80 20 client.15570 trim_cache size 0 max 16384 2023-09-12T16:03:39.350 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:03:39.350 INFO:teuthology.orchestra.run.smithi130.stdout: "cluster_fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T16:03:39.350 INFO:teuthology.orchestra.run.smithi130.stdout: "whoami": 1, 2023-09-12T16:03:39.350 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 24592, 2023-09-12T16:03:39.351 INFO:teuthology.orchestra.run.smithi130.stdout: "want_state": "up:active", 2023-09-12T16:03:39.351 INFO:teuthology.orchestra.run.smithi130.stdout: "state": "up:active", 2023-09-12T16:03:39.351 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs", 2023-09-12T16:03:39.351 INFO:teuthology.orchestra.run.smithi130.stdout: "rank_uptime": 75.692613034999994, 2023-09-12T16:03:39.351 INFO:teuthology.orchestra.run.smithi130.stdout: "mdsmap_epoch": 24, 2023-09-12T16:03:39.351 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch": 88, 2023-09-12T16:03:39.352 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch_barrier": 86, 2023-09-12T16:03:39.352 INFO:teuthology.orchestra.run.smithi130.stdout: "uptime": 106.252903461 2023-09-12T16:03:39.352 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:03:39.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 -- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd54c0a06d0 msgr2=0x7fd54c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd54c0a06d0 0x7fd54c0a2b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd554000c00 tx=0x7fd554005fb0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 -- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5640ff560 msgr2=0x7fd564195200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5640ff560 0x7fd564195200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd55800d980 tx=0x7fd55800dd40 comp rx=0 tx=0).stop 2023-09-12T16:03:39.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 -- 172.21.15.130:0/3429544376 shutdown_connections 2023-09-12T16:03:39.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd54c0a06d0 0x7fd54c0a2b90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641010f0 0x7fd56419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5640ff560 0x7fd564195200 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 --2- 172.21.15.130:0/3429544376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd564100750 0x7fd564195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 -- 172.21.15.130:0/3429544376 >> 172.21.15.130:0/3429544376 conn(0x7fd5640faad0 msgr2=0x7fd564103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.326+0000 7fd569110700 1 -- 172.21.15.130:0/3429544376 shutdown_connections 2023-09-12T16:03:39.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.327+0000 7fd569110700 1 -- 172.21.15.130:0/3429544376 wait complete. 2023-09-12T16:03:39.356 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-12T16:03:39.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.478+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1223378433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c1021a0 msgr2=0x7f749c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.478+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1223378433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c1021a0 0x7f749c108a50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f749400b0d0 tx=0x7f749400b3e0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.478+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1223378433 shutdown_connections 2023-09-12T16:03:39.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.478+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1223378433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c1021a0 0x7f749c108a50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.478+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1223378433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f749c100610 0x7f749c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.478+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1223378433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c101800 0x7f749c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.478+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1223378433 >> 172.21.15.130:0/1223378433 conn(0x7f749c0fbb80 msgr2=0x7f749c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.478+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1223378433 shutdown_connections 2023-09-12T16:03:39.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.478+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1223378433 wait complete. 2023-09-12T16:03:39.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f74a0fe8700 1 Processor -- start 2023-09-12T16:03:39.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f74a0fe8700 1 -- start start 2023-09-12T16:03:39.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f74a0fe8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c100610 0x7f749c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f74a0fe8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f749c101800 0x7f749c1968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f74a0fe8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c1021a0 0x7f749c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f74a0fe8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c19df20 con 0x7f749c100610 2023-09-12T16:03:39.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f74a0fe8700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c19e060 con 0x7f749c1021a0 2023-09-12T16:03:39.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f74a0fe8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c19e1f0 con 0x7f749c101800 2023-09-12T16:03:39.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f749a59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c100610 0x7f749c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f749a59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c100610 0x7f749c1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54978/0 (socket says 172.21.15.130:54978) 2023-09-12T16:03:39.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f749a59c700 1 -- 172.21.15.130:0/1121435781 learned_addr learned my addr 172.21.15.130:0/1121435781 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f749ad9d700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c1021a0 0x7f749c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.479+0000 7f7499d9b700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f749c101800 0x7f749c1968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f749a59c700 1 -- 172.21.15.130:0/1121435781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f749c101800 msgr2=0x7f749c1968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f749a59c700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f749c101800 0x7f749c1968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f749a59c700 1 -- 172.21.15.130:0/1121435781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c1021a0 msgr2=0x7f749c19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f749a59c700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c1021a0 0x7f749c19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f749a59c700 1 -- 172.21.15.130:0/1121435781 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494009d00 con 0x7f749c100610 2023-09-12T16:03:39.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f749ad9d700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c1021a0 0x7f749c19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:39.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f749a59c700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c100610 0x7f749c1963a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f749000e970 tx=0x7f749000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f748b7fe700 1 -- 172.21.15.130:0/1121435781 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490009790 con 0x7f749c100610 2023-09-12T16:03:39.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1121435781 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c19e4d0 con 0x7f749c100610 2023-09-12T16:03:39.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f748b7fe700 1 -- 172.21.15.130:0/1121435781 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74900044e0 con 0x7f749c100610 2023-09-12T16:03:39.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f748b7fe700 1 -- 172.21.15.130:0/1121435781 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490003e30 con 0x7f749c100610 2023-09-12T16:03:39.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.480+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1121435781 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c19ea30 con 0x7f749c100610 2023-09-12T16:03:39.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.482+0000 7f748b7fe700 1 -- 172.21.15.130:0/1121435781 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f7490003f90 con 0x7f749c100610 2023-09-12T16:03:39.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.483+0000 7f748b7fe700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74840a06d0 0x7f74840a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.483+0000 7f7499d9b700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74840a06d0 0x7f74840a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.483+0000 7f748b7fe700 1 -- 172.21.15.130:0/1121435781 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7490014070 con 0x7f749c100610 2023-09-12T16:03:39.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.483+0000 7f7499d9b700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74840a06d0 0x7f74840a2b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f748c009bf0 tx=0x7f748c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f74ad1aeb80 1 Processor -- start 2023-09-12T16:03:39.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f74ad1aeb80 1 -- start start 2023-09-12T16:03:39.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e45bb0 0x557b04e45fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 0x557b04e447b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 0x557b04e45260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04d9d6e0 con 0x557b04e45bb0 2023-09-12T16:03:39.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04d9b000 con 0x557b04e44db0 2023-09-12T16:03:39.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04d968f0 con 0x557b04e46550 2023-09-12T16:03:39.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f749a59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e45bb0 0x557b04e45fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f749a59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e45bb0 0x557b04e45fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54986/0 (socket says 172.21.15.130:54986) 2023-09-12T16:03:39.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f749a59c700 1 -- 172.21.15.130:0/1340378005 learned_addr learned my addr 172.21.15.130:0/1340378005 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f749ad9d700 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 0x557b04e45260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f7499d9b700 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 0x557b04e447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f749a59c700 1 -- 172.21.15.130:0/1340378005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 msgr2=0x557b04e447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f749a59c700 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 0x557b04e447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f749a59c700 1 -- 172.21.15.130:0/1340378005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 msgr2=0x557b04e45260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f749a59c700 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 0x557b04e45260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f749a59c700 1 -- 172.21.15.130:0/1340378005 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b04bc2d90 con 0x557b04e45bb0 2023-09-12T16:03:39.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.484+0000 7f7499d9b700 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 0x557b04e447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:39.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f749a59c700 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e45bb0 0x557b04e45fd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f749000c5b0 tx=0x7f74900992e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1952e87a3eee3e8a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f7488ff9700 1 -- 172.21.15.130:0/1340378005 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490099dd0 con 0x557b04e45bb0 2023-09-12T16:03:39.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f7488ff9700 1 -- 172.21.15.130:0/1340378005 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74900996d0 con 0x557b04e45bb0 2023-09-12T16:03:39.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f7488ff9700 1 -- 172.21.15.130:0/1340378005 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490016f00 con 0x557b04e45bb0 2023-09-12T16:03:39.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/1340378005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e45bb0 msgr2=0x557b04e45fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e45bb0 0x557b04e45fd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f749000c5b0 tx=0x7f74900992e0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/1340378005 shutdown_connections 2023-09-12T16:03:39.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 0x557b04e45260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e45bb0 0x557b04e45fd0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/1340378005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 0x557b04e447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.485+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/1340378005 >> 172.21.15.130:0/1340378005 conn(0x557b04db9040 msgr2=0x557b04e3f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/1340378005 shutdown_connections 2023-09-12T16:03:39.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/1340378005 wait complete. 2023-09-12T16:03:39.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 -- start start 2023-09-12T16:03:39.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 Processor -- start 2023-09-12T16:03:39.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e44db0 0x557b04e4e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e4ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04e55b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04d34b20 con 0x557b04e46550 2023-09-12T16:03:39.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04d34820 con 0x557b04e45bb0 2023-09-12T16:03:39.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04c188d0 con 0x557b04e44db0 2023-09-12T16:03:39.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f74ad1aeb80 10 client.0 _ll_register_callbacks cb 0x557b04c7ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:39.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f749ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04e55b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f749ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04e55b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54996/0 (socket says 172.21.15.130:54996) 2023-09-12T16:03:39.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f749ad9d700 1 -- 172.21.15.130:0/3383479763 learned_addr learned my addr 172.21.15.130:0/3383479763 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.486+0000 7f749a59c700 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e44db0 0x557b04e4e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f749ad9d700 1 -- 172.21.15.130:0/3383479763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e44db0 msgr2=0x557b04e4e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f749ad9d700 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e44db0 0x557b04e4e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f749ad9d700 1 -- 172.21.15.130:0/3383479763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 msgr2=0x557b04e4ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f7499d9b700 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e4ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f749ad9d700 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e4ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f749ad9d700 1 -- 172.21.15.130:0/3383479763 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c19ea30 con 0x557b04e46550 2023-09-12T16:03:39.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f749a59c700 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e44db0 0x557b04e4e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:39.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f7499d9b700 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e4ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:39.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f749ad9d700 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04e55b10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7494015000 tx=0x7f749400baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3203880443cdb93 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f7481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:39.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f7481ffb700 1 -- 172.21.15.130:0/3383479763 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400e040 con 0x557b04e46550 2023-09-12T16:03:39.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f7481ffb700 1 -- 172.21.15.130:0/3383479763 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7494004c50 con 0x557b04e46550 2023-09-12T16:03:39.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/3383479763 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b04cee4a0 con 0x557b04e46550 2023-09-12T16:03:39.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f7481ffb700 1 -- 172.21.15.130:0/3383479763 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7494004e50 con 0x557b04e46550 2023-09-12T16:03:39.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f74ad1aeb80 10 client.15582 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:39.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/3383479763 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557b04babc10 con 0x557b04e46550 2023-09-12T16:03:39.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f7481ffb700 1 -- 172.21.15.130:0/3383479763 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b04cee4a0 con 0x557b04e46550 2023-09-12T16:03:39.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f7481ffb700 1 -- 172.21.15.130:0/3383479763 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f749400b100 con 0x557b04e46550 2023-09-12T16:03:39.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.487+0000 7f74ad1aeb80 10 client.15582 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:39.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.488+0000 7f74ad1aeb80 10 client.15582 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-12T16:03:39.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.488+0000 7f74ad1aeb80 20 client.15582 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:39.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.488+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557b04e59c10 0x557b04e58b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.488+0000 7f74ad1aeb80 4 client.15582 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:03:39.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.488+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/3383479763 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b04dab850 con 0x557b04e59c10 2023-09-12T16:03:39.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.488+0000 7f749a59c700 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557b04e59c10 0x557b04e58b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.488+0000 7f749a59c700 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557b04e59c10 0x557b04e58b80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce8939536c03b0fb server_cookie=e3633be48258f8ee in_seq=0 out_seq=0 2023-09-12T16:03:39.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.488+0000 7f7481ffb700 10 client.15582 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:39.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f7481ffb700 1 -- 172.21.15.130:0/3383479763 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557b04dab850 con 0x557b04e59c10 2023-09-12T16:03:39.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f7481ffb700 10 client.15582 handle_command_reply: tid=0 2023-09-12T16:03:39.509 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 client.15582 shutdown 2023-09-12T16:03:39.509 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/3383479763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 msgr2=0x557b04e55b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.509 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04e55b10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7494015000 tx=0x7f749400baa0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.509 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/3383479763 shutdown_connections 2023-09-12T16:03:39.509 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557b04e59c10 0x557b04e58b80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04e55b10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f7481ffb700 0 client.15582 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:39.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e44db0 0x557b04e4e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/3383479763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e4ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/3383479763 >> 172.21.15.130:0/3383479763 conn(0x557b04db9040 msgr2=0x557b04e3ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/3383479763 shutdown_connections 2023-09-12T16:03:39.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/3383479763 wait complete. 2023-09-12T16:03:39.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.489+0000 7f74ad1aeb80 20 client.15582 trim_cache size 0 max 16384 2023-09-12T16:03:39.516 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.506+0000 7f74ad1aeb80 1 Processor -- start 2023-09-12T16:03:39.516 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.506+0000 7f74ad1aeb80 1 -- start start 2023-09-12T16:03:39.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e44db0 0x557b04e4c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e55a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 0x557b04e48ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04dab850 con 0x557b04e44db0 2023-09-12T16:03:39.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04cee4a0 con 0x557b04e45bb0 2023-09-12T16:03:39.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04c24de0 con 0x557b04e46550 2023-09-12T16:03:39.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749a59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e44db0 0x557b04e4c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749a59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e44db0 0x557b04e4c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55000/0 (socket says 172.21.15.130:55000) 2023-09-12T16:03:39.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749a59c700 1 -- 172.21.15.130:0/390804416 learned_addr learned my addr 172.21.15.130:0/390804416 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749ad9d700 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 0x557b04e48ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f7499d9b700 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e55a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749a59c700 1 -- 172.21.15.130:0/390804416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 msgr2=0x557b04e48ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749a59c700 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 0x557b04e48ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749a59c700 1 -- 172.21.15.130:0/390804416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 msgr2=0x557b04e55a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749a59c700 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e55a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749a59c700 1 -- 172.21.15.130:0/390804416 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b04be9200 con 0x557b04e44db0 2023-09-12T16:03:39.520 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f7499d9b700 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e55a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:39.520 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f749a59c700 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e44db0 0x557b04e4c390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f74900c9a70 tx=0x7f74900cc270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9563e1d94b9c7ea server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.520 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f74897fa700 1 -- 172.21.15.130:0/390804416 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74900b0070 con 0x557b04e44db0 2023-09-12T16:03:39.520 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f74897fa700 1 -- 172.21.15.130:0/390804416 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f749009d580 con 0x557b04e44db0 2023-09-12T16:03:39.520 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.507+0000 7f74897fa700 1 -- 172.21.15.130:0/390804416 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74900c9c40 con 0x557b04e44db0 2023-09-12T16:03:39.521 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/390804416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e44db0 msgr2=0x557b04e4c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.521 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e44db0 0x557b04e4c390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f74900c9a70 tx=0x7f74900cc270 comp rx=0 tx=0).stop 2023-09-12T16:03:39.521 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/390804416 shutdown_connections 2023-09-12T16:03:39.521 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e46550 0x557b04e48ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.521 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e44db0 0x557b04e4c390 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.522 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/390804416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e45bb0 0x557b04e55a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.522 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/390804416 >> 172.21.15.130:0/390804416 conn(0x557b04db9040 msgr2=0x557b04e563a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.522 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/390804416 shutdown_connections 2023-09-12T16:03:39.522 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/390804416 wait complete. 2023-09-12T16:03:39.522 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 -- start start 2023-09-12T16:03:39.522 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 Processor -- start 2023-09-12T16:03:39.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 0x557b04e507e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e45bb0 0x557b04e50d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04eadc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04e55d50 con 0x557b04e46550 2023-09-12T16:03:39.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04e55ec0 con 0x557b04e44db0 2023-09-12T16:03:39.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b04eae1a0 con 0x557b04e45bb0 2023-09-12T16:03:39.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f74ad1aeb80 10 client.0 _ll_register_callbacks cb 0x7f748c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:39.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f749ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04eadc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f749ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04eadc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55010/0 (socket says 172.21.15.130:55010) 2023-09-12T16:03:39.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.508+0000 7f749ad9d700 1 -- 172.21.15.130:0/334387163 learned_addr learned my addr 172.21.15.130:0/334387163 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:39.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f749a59c700 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 0x557b04e507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f749ad9d700 1 -- 172.21.15.130:0/334387163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e45bb0 msgr2=0x557b04e50d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f749ad9d700 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e45bb0 0x557b04e50d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f749ad9d700 1 -- 172.21.15.130:0/334387163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 msgr2=0x557b04e507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:39.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f749ad9d700 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 0x557b04e507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f749ad9d700 1 -- 172.21.15.130:0/334387163 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b04bc2d90 con 0x557b04e46550 2023-09-12T16:03:39.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f749a59c700 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 0x557b04e507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:39.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f749ad9d700 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04eadc60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7494015000 tx=0x7f74940097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a0a0899a976f05 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:39.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f7481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:39.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f7481ffb700 1 -- 172.21.15.130:0/334387163 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400e040 con 0x557b04e46550 2023-09-12T16:03:39.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/334387163 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74900996d0 con 0x557b04e46550 2023-09-12T16:03:39.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f7481ffb700 1 -- 172.21.15.130:0/334387163 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f74940059e0 con 0x557b04e46550 2023-09-12T16:03:39.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f7481ffb700 1 -- 172.21.15.130:0/334387163 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f749400b2c0 con 0x557b04e46550 2023-09-12T16:03:39.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f74ad1aeb80 10 client.15588 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:39.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/334387163 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557b04bc1260 con 0x557b04e46550 2023-09-12T16:03:39.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f7481ffb700 1 -- 172.21.15.130:0/334387163 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74900996d0 con 0x557b04e46550 2023-09-12T16:03:39.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.509+0000 7f7481ffb700 1 -- 172.21.15.130:0/334387163 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7494025990 con 0x557b04e46550 2023-09-12T16:03:39.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f74ad1aeb80 10 client.15588 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:39.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f74ad1aeb80 10 client.15588 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-12T16:03:39.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f74ad1aeb80 20 client.15588 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:39.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557b04eb30c0 0x557b04eaf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:39.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f74ad1aeb80 4 client.15588 mds_command: new command op to 24593 tid=0[{"prefix": "status"}] 2023-09-12T16:03:39.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/334387163 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557b04d8f8d0 con 0x557b04eb30c0 2023-09-12T16:03:39.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f749a59c700 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557b04eb30c0 0x557b04eaf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:39.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f7481ffb700 10 client.15588 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:39.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f749a59c700 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557b04eb30c0 0x557b04eaf510 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75bf58f3f3d6538e server_cookie=ab75b9344dcfe218 in_seq=0 out_seq=0 2023-09-12T16:03:39.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f7481ffb700 1 -- 172.21.15.130:0/334387163 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557b04d8f8d0 con 0x557b04eb30c0 2023-09-12T16:03:39.530 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f7481ffb700 10 client.15588 handle_command_reply: tid=0 2023-09-12T16:03:39.530 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f74ad1aeb80 1 client.15588 shutdown 2023-09-12T16:03:39.530 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/334387163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 msgr2=0x557b04eadc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.530 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.510+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04eadc60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7494015000 tx=0x7f74940097a0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.530 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/334387163 shutdown_connections 2023-09-12T16:03:39.531 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557b04eb30c0 0x557b04eaf510 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.531 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557b04e46550 0x557b04eadc60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.531 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f7481ffb700 0 client.15588 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:39.531 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b04e44db0 0x557b04e507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.531 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74ad1aeb80 1 --2- 172.21.15.130:0/334387163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557b04e45bb0 0x557b04e50d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.532 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/334387163 >> 172.21.15.130:0/334387163 conn(0x557b04db9040 msgr2=0x557b04e4c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.532 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/334387163 shutdown_connections 2023-09-12T16:03:39.532 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74ad1aeb80 1 -- 172.21.15.130:0/334387163 wait complete. 2023-09-12T16:03:39.532 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74ad1aeb80 20 client.15588 trim_cache size 0 max 16384 2023-09-12T16:03:39.533 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:03:39.533 INFO:teuthology.orchestra.run.smithi130.stdout: "cluster_fsid": "9ef22ea2-5184-11ee-9ab7-7b867c8bd7da", 2023-09-12T16:03:39.533 INFO:teuthology.orchestra.run.smithi130.stdout: "whoami": 0, 2023-09-12T16:03:39.533 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 24593, 2023-09-12T16:03:39.533 INFO:teuthology.orchestra.run.smithi130.stdout: "want_state": "up:active", 2023-09-12T16:03:39.534 INFO:teuthology.orchestra.run.smithi130.stdout: "state": "up:active", 2023-09-12T16:03:39.534 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs", 2023-09-12T16:03:39.534 INFO:teuthology.orchestra.run.smithi130.stdout: "rank_uptime": 94.486000752999999, 2023-09-12T16:03:39.534 INFO:teuthology.orchestra.run.smithi130.stdout: "mdsmap_epoch": 24, 2023-09-12T16:03:39.534 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch": 88, 2023-09-12T16:03:39.535 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_epoch_barrier": 86, 2023-09-12T16:03:39.535 INFO:teuthology.orchestra.run.smithi130.stdout: "uptime": 99.955357356999997 2023-09-12T16:03:39.535 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:03:39.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74840a06d0 msgr2=0x7f74840a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74840a06d0 0x7f74840a2b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f748c009bf0 tx=0x7f748c009260 comp rx=0 tx=0).stop 2023-09-12T16:03:39.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c100610 msgr2=0x7f749c1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:39.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c100610 0x7f749c1963a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f749000e970 tx=0x7f749000ed30 comp rx=0 tx=0).stop 2023-09-12T16:03:39.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1121435781 shutdown_connections 2023-09-12T16:03:39.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f74840a06d0 0x7f74840a2b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c1021a0 0x7f749c19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f749c100610 0x7f749c1963a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 --2- 172.21.15.130:0/1121435781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f749c101800 0x7f749c1968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:39.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1121435781 >> 172.21.15.130:0/1121435781 conn(0x7f749c0fbb80 msgr2=0x7f749c104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:39.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1121435781 shutdown_connections 2023-09-12T16:03:39.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:39.511+0000 7f74a0fe8700 1 -- 172.21.15.130:0/1121435781 wait complete. 2023-09-12T16:03:39.539 INFO:teuthology.run_tasks:Running task kclient... 2023-09-12T16:03:39.553 INFO:tasks.kclient:Mounting kernel clients... 2023-09-12T16:03:39.553 INFO:tasks.kclient:config is {'client.0': {}} 2023-09-12T16:03:39.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:39.554 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list 2023-09-12T16:03:39.571 INFO:teuthology.orchestra.run.smithi130.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-09-12T16:03:39.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:39.572 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-12T16:03:39.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:39.651 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link delete ceph-brx 2023-09-12T16:03:39.754 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-09-12T16:03:39.755 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-12T16:03:39.755 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-09-12T16:03:39.756 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-09-12T16:03:39.756 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph fs ls 2023-09-12T16:03:39.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:39 smithi130 ceph-mon[98670]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:03:39.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:39 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/465647740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:40.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:39 smithi159 ceph-mon[102522]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:03:40.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:39 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/465647740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:39 smithi203 ceph-mon[103431]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-12T16:03:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:39 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/465647740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:40.317 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:03:41.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.576+0000 7f1df584b700 1 -- 172.21.15.130:0/3186905429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df0101840 msgr2=0x7f1df010c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:41.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.576+0000 7f1df584b700 1 --2- 172.21.15.130:0/3186905429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df0101840 0x7f1df010c4d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1de800b0a0 tx=0x7f1de800b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.576+0000 7f1df584b700 1 -- 172.21.15.130:0/3186905429 shutdown_connections 2023-09-12T16:03:41.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.576+0000 7f1df584b700 1 --2- 172.21.15.130:0/3186905429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df0101840 0x7f1df010c4d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.576+0000 7f1df584b700 1 --2- 172.21.15.130:0/3186905429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df00ff880 0x7f1df00ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.576+0000 7f1df584b700 1 --2- 172.21.15.130:0/3186905429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df01001a0 0x7f1df0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.576+0000 7f1df584b700 1 -- 172.21.15.130:0/3186905429 >> 172.21.15.130:0/3186905429 conn(0x7f1df00765f0 msgr2=0x7f1df0076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:41.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.576+0000 7f1df584b700 1 -- 172.21.15.130:0/3186905429 shutdown_connections 2023-09-12T16:03:41.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.576+0000 7f1df584b700 1 -- 172.21.15.130:0/3186905429 wait complete. 2023-09-12T16:03:41.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1df584b700 1 Processor -- start 2023-09-12T16:03:41.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1df584b700 1 -- start start 2023-09-12T16:03:41.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1df584b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df00ff880 0x7f1df01022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:41.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1df584b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df01001a0 0x7f1df0102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:41.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1df584b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101840 0x7f1df0102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:41.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1df584b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df01a4cb0 con 0x7f1df00ff880 2023-09-12T16:03:41.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1df584b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df01a4e20 con 0x7f1df0101840 2023-09-12T16:03:41.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1df584b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df01a4fe0 con 0x7f1df01001a0 2023-09-12T16:03:41.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1deeffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df00ff880 0x7f1df01022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:41.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1deeffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df00ff880 0x7f1df01022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55020/0 (socket says 172.21.15.130:55020) 2023-09-12T16:03:41.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.577+0000 7f1deeffd700 1 -- 172.21.15.130:0/419636206 learned_addr learned my addr 172.21.15.130:0/419636206 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:41.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1def7fe700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101840 0x7f1df0102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:41.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1dee7fc700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df01001a0 0x7f1df0102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:41.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1deeffd700 1 -- 172.21.15.130:0/419636206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df01001a0 msgr2=0x7f1df0102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:41.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1deeffd700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df01001a0 0x7f1df0102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1deeffd700 1 -- 172.21.15.130:0/419636206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101840 msgr2=0x7f1df0102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:41.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1deeffd700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101840 0x7f1df0102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1deeffd700 1 -- 172.21.15.130:0/419636206 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de8009d00 con 0x7f1df00ff880 2023-09-12T16:03:41.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1def7fe700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101840 0x7f1df0102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:41.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1dee7fc700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df01001a0 0x7f1df0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:41.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1deeffd700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df00ff880 0x7f1df01022d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c9b0 tx=0x7f1de400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:41.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1df4849700 1 -- 172.21.15.130:0/419636206 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4007b00 con 0x7f1df00ff880 2023-09-12T16:03:41.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df01a52c0 con 0x7f1df00ff880 2023-09-12T16:03:41.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1df4849700 1 -- 172.21.15.130:0/419636206 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1de4007c60 con 0x7f1df00ff880 2023-09-12T16:03:41.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df01a5810 con 0x7f1df00ff880 2023-09-12T16:03:41.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.578+0000 7f1df4849700 1 -- 172.21.15.130:0/419636206 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4018660 con 0x7f1df00ff880 2023-09-12T16:03:41.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.579+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df004e300 con 0x7f1df00ff880 2023-09-12T16:03:41.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.580+0000 7f1df4849700 1 -- 172.21.15.130:0/419636206 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f1de4018880 con 0x7f1df00ff880 2023-09-12T16:03:41.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.581+0000 7f1df4849700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dd80a06d0 0x7f1dd80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:41.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.581+0000 7f1dee7fc700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dd80a06d0 0x7f1dd80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:41.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.581+0000 7f1df4849700 1 -- 172.21.15.130:0/419636206 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1de40cc350 con 0x7f1df00ff880 2023-09-12T16:03:41.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.581+0000 7f1dee7fc700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dd80a06d0 0x7f1dd80a2b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1de0005fd0 tx=0x7f1de0005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:41.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.584+0000 7f1df4849700 1 -- 172.21.15.130:0/419636206 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1de4095ae0 con 0x7f1df00ff880 2023-09-12T16:03:41.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.817+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1df010f5a0 con 0x7f1df00ff880 2023-09-12T16:03:41.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.817+0000 7f1df4849700 1 -- 172.21.15.130:0/419636206 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1de4099100 con 0x7f1df00ff880 2023-09-12T16:03:41.818 INFO:teuthology.orchestra.run.smithi130.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-12T16:03:41.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dd80a06d0 msgr2=0x7f1dd80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:41.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dd80a06d0 0x7f1dd80a2b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1de0005fd0 tx=0x7f1de0005ee0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df00ff880 msgr2=0x7f1df01022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:41.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df00ff880 0x7f1df01022d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c9b0 tx=0x7f1de400cd70 comp rx=0 tx=0).stop 2023-09-12T16:03:41.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 shutdown_connections 2023-09-12T16:03:41.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1dd80a06d0 0x7f1dd80a2b90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101840 0x7f1df0102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1df00ff880 0x7f1df01022d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 --2- 172.21.15.130:0/419636206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df01001a0 0x7f1df0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:41.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 >> 172.21.15.130:0/419636206 conn(0x7f1df00765f0 msgr2=0x7f1df0068950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:41.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 shutdown_connections 2023-09-12T16:03:41.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:41.820+0000 7f1df584b700 1 -- 172.21.15.130:0/419636206 wait complete. 2023-09-12T16:03:42.170 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:41 smithi130 ceph-mon[98670]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:42.171 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:41 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/419636206' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-12T16:03:42.180 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-12T16:03:42.180 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-12T16:03:42.180 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi130.front.sepia.ceph.com 2023-09-12T16:03:42.181 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-12T16:03:42.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-09-12T16:03:42.181 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-12T16:03:42.181 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-12T16:03:42.182 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-12T16:03:42.182 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-09-12T16:03:42.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.183 DEBUG:teuthology.orchestra.run.smithi130:> ip addr 2023-09-12T16:03:42.201 INFO:teuthology.orchestra.run.smithi130.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T16:03:42.202 INFO:teuthology.orchestra.run.smithi130.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T16:03:42.202 INFO:teuthology.orchestra.run.smithi130.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T16:03:42.202 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft forever preferred_lft forever 2023-09-12T16:03:42.202 INFO:teuthology.orchestra.run.smithi130.stdout: inet6 ::1/128 scope host 2023-09-12T16:03:42.203 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft forever preferred_lft forever 2023-09-12T16:03:42.203 INFO:teuthology.orchestra.run.smithi130.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-12T16:03:42.203 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:d9:37:96 brd ff:ff:ff:ff:ff:ff 2023-09-12T16:03:42.203 INFO:teuthology.orchestra.run.smithi130.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-12T16:03:42.203 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:d9:37:97 brd ff:ff:ff:ff:ff:ff 2023-09-12T16:03:42.204 INFO:teuthology.orchestra.run.smithi130.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-12T16:03:42.204 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:88:7f:36 brd ff:ff:ff:ff:ff:ff 2023-09-12T16:03:42.204 INFO:teuthology.orchestra.run.smithi130.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-12T16:03:42.204 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:88:7f:37 brd ff:ff:ff:ff:ff:ff 2023-09-12T16:03:42.205 INFO:teuthology.orchestra.run.smithi130.stdout: inet 172.21.15.130/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-12T16:03:42.205 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft 41858sec preferred_lft 41858sec 2023-09-12T16:03:42.205 INFO:teuthology.orchestra.run.smithi130.stdout: inet6 fe80::13c4:484c:d749:6616/64 scope link noprefixroute 2023-09-12T16:03:42.205 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft forever preferred_lft forever 2023-09-12T16:03:42.206 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-12T16:03:42.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.206 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:03:42.207 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:03:42.207 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link add name ceph-brx type bridge 2023-09-12T16:03:42.207 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip addr flush dev ceph-brx 2023-09-12T16:03:42.208 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link set ceph-brx up 2023-09-12T16:03:42.208 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-12T16:03:42.208 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:03:42.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:41 smithi159 ceph-mon[102522]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:42.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:41 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/419636206' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-12T16:03:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:41 smithi203 ceph-mon[103431]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-09-12T16:03:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:41 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/419636206' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-12T16:03:42.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.324 DEBUG:teuthology.orchestra.run.smithi130:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-12T16:03:42.350 INFO:teuthology.orchestra.run.smithi130.stdout:1 2023-09-12T16:03:42.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.352 DEBUG:teuthology.orchestra.run.smithi130:> route 2023-09-12T16:03:42.411 INFO:teuthology.orchestra.run.smithi130.stdout:Kernel IP routing table 2023-09-12T16:03:42.411 INFO:teuthology.orchestra.run.smithi130.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-12T16:03:42.411 INFO:teuthology.orchestra.run.smithi130.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-12T16:03:42.411 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-12T16:03:42.412 INFO:teuthology.orchestra.run.smithi130.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-12T16:03:42.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.412 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:03:42.413 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:03:42.413 DEBUG:teuthology.orchestra.run.smithi130:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-12T16:03:42.413 DEBUG:teuthology.orchestra.run.smithi130:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-12T16:03:42.414 DEBUG:teuthology.orchestra.run.smithi130:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-12T16:03:42.414 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:03:42.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.517 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list 2023-09-12T16:03:42.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.537 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list-id 2023-09-12T16:03:42.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.592 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:03:42.592 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:03:42.592 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns add ceph-ns--tmp-cephfs 2023-09-12T16:03:42.592 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-09-12T16:03:42.593 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:03:42.685 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-09-12T16:03:42.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.686 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:03:42.687 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:03:42.687 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-09-12T16:03:42.687 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-12T16:03:42.688 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-09-12T16:03:42.688 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-09-12T16:03:42.688 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-09-12T16:03:42.689 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:03:42.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:42.899 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec bash -c ' 2023-09-12T16:03:42.900 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2023-09-12T16:03:42.900 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link set brx.0 up 2023-09-12T16:03:42.900 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link set dev brx.0 master ceph-brx 2023-09-12T16:03:42.900 DEBUG:teuthology.orchestra.run.smithi130:> ') 2023-09-12T16:03:43.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T16:03:43.009 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -v /tmp/cephfs 2023-09-12T16:03:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:mkdir: created directory '/tmp/cephfs' 2023-09-12T16:03:43.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T16:03:43.047 DEBUG:teuthology.orchestra.run.smithi130:> chmod 0000 /tmp/cephfs 2023-09-12T16:03:43.105 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-09-12T16:03:43.106 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-09-12T16:03:43.106 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/47d66d2e-a22e-4bfa-ac90-69257256aa66 2023-09-12T16:03:43.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:43.106 DEBUG:teuthology.orchestra.run.smithi130:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/47d66d2e-a22e-4bfa-ac90-69257256aa66 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-09-12T16:03:43.190 INFO:teuthology.orchestra.run.smithi130.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-09-12T16:03:43.191 INFO:teuthology.orchestra.run.smithi130.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-09-12T16:03:43.407 INFO:teuthology.orchestra.run.smithi130.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-09-12T16:03:43.407 INFO:teuthology.orchestra.run.smithi130.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-09-12T16:03:43.408 INFO:teuthology.orchestra.run.smithi130.stdout:mount.ceph: resolved to: "172.21.15.130:3300,172.21.15.159:3300,172.21.15.203:3300" 2023-09-12T16:03:43.408 INFO:teuthology.orchestra.run.smithi130.stdout:mount.ceph: trying mount with new device syntax: 0@9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.cephfs=/volumes/_nogroup/sv_1/47d66d2e-a22e-4bfa-ac90-69257256aa66 2023-09-12T16:03:43.408 INFO:teuthology.orchestra.run.smithi130.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.130:3300/172.21.15.159:3300/172.21.15.203:3300" will pass to kernel 2023-09-12T16:03:43.408 INFO:teuthology.orchestra.run.smithi130.stdout:mount.ceph: v2 mount support check returned 2 2023-09-12T16:03:43.409 INFO:teuthology.orchestra.run.smithi130.stdout:mount.ceph: kernel does not support v2 syntax 2023-09-12T16:03:43.409 INFO:teuthology.orchestra.run.smithi130.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.130:3300,172.21.15.159:3300,172.21.15.203:3300:/volumes/_nogroup/sv_1/47d66d2e-a22e-4bfa-ac90-69257256aa66 2023-09-12T16:03:43.409 INFO:teuthology.orchestra.run.smithi130.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mds_namespace=cephfs,fsid=9ef22ea2-5184-11ee-9ab7-7b867c8bd7da" will pass to kernel 2023-09-12T16:03:43.410 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-09-12T16:03:43.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:43.411 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 1777 /tmp/cephfs 2023-09-12T16:03:43.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:43.448 DEBUG:teuthology.orchestra.run.smithi130:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-12T16:03:43.542 INFO:teuthology.orchestra.run.smithi130.stdout:client15597 2023-09-12T16:03:43.543 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T16:03:43.543 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T16:03:43.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:43.564 DEBUG:teuthology.orchestra.run.smithi130:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-12T16:03:43.659 INFO:teuthology.orchestra.run.smithi130.stdout:client15597 2023-09-12T16:03:43.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:43.660 DEBUG:teuthology.orchestra.run.smithi130:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-12T16:03:43.712 INFO:teuthology.orchestra.run.smithi130.stdout:client15597 2023-09-12T16:03:43.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:03:43.713 DEBUG:teuthology.orchestra.run.smithi130:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da.client15597/status') 2023-09-12T16:03:43.769 INFO:teuthology.orchestra.run.smithi130.stdout:instance: client.15597 (3)192.168.0.1:0/2802457390 2023-09-12T16:03:43.769 INFO:teuthology.orchestra.run.smithi130.stdout:blocklisted: false 2023-09-12T16:03:43.771 INFO:teuthology.orchestra.run.smithi130.stderr:0+1 records in 2023-09-12T16:03:43.771 INFO:teuthology.orchestra.run.smithi130.stderr:0+1 records out 2023-09-12T16:03:43.771 INFO:teuthology.orchestra.run.smithi130.stderr:70 bytes copied, 2.6589e-05 s, 2.6 MB/s 2023-09-12T16:03:43.772 INFO:teuthology.run_tasks:Running task check-counter... 2023-09-12T16:03:43.783 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-09-12T16:03:43.835 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:43.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.958+0000 7fda1955d700 1 -- 172.21.15.130:0/2538892782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 msgr2=0x7fda14108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:43.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.958+0000 7fda1955d700 1 --2- 172.21.15.130:0/2538892782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 0x7fda14108a40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00b0d0 tx=0x7fda0c00b3e0 comp rx=0 tx=0).stop 2023-09-12T16:03:43.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.958+0000 7fda1955d700 1 -- 172.21.15.130:0/2538892782 shutdown_connections 2023-09-12T16:03:43.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.958+0000 7fda1955d700 1 --2- 172.21.15.130:0/2538892782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 0x7fda14108a40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:43.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.958+0000 7fda1955d700 1 --2- 172.21.15.130:0/2538892782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda14100600 0x7fda14100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:43.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.958+0000 7fda1955d700 1 --2- 172.21.15.130:0/2538892782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda141017f0 0x7fda14101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:43.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.958+0000 7fda1955d700 1 -- 172.21.15.130:0/2538892782 >> 172.21.15.130:0/2538892782 conn(0x7fda140fbb50 msgr2=0x7fda140fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:43.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.958+0000 7fda1955d700 1 -- 172.21.15.130:0/2538892782 shutdown_connections 2023-09-12T16:03:43.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.958+0000 7fda1955d700 1 -- 172.21.15.130:0/2538892782 wait complete. 2023-09-12T16:03:43.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda1955d700 1 Processor -- start 2023-09-12T16:03:43.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda1955d700 1 -- start start 2023-09-12T16:03:43.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda1955d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda14100600 0x7fda14196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:43.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda1955d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda141017f0 0x7fda14196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:43.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda1955d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 0x7fda1419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:43.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda1955d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1419dfc0 con 0x7fda14102190 2023-09-12T16:03:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda1955d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1419e100 con 0x7fda141017f0 2023-09-12T16:03:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda1955d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1419e290 con 0x7fda14100600 2023-09-12T16:03:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda137fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 0x7fda1419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda137fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 0x7fda1419d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55040/0 (socket says 172.21.15.130:55040) 2023-09-12T16:03:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda137fe700 1 -- 172.21.15.130:0/2780858805 learned_addr learned my addr 172.21.15.130:0/2780858805 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda12ffd700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda14100600 0x7fda14196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda127fc700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda141017f0 0x7fda14196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda137fe700 1 -- 172.21.15.130:0/2780858805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda14100600 msgr2=0x7fda14196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda137fe700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda14100600 0x7fda14196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda137fe700 1 -- 172.21.15.130:0/2780858805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda141017f0 msgr2=0x7fda14196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda137fe700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda141017f0 0x7fda14196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda137fe700 1 -- 172.21.15.130:0/2780858805 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c009d00 con 0x7fda14102190 2023-09-12T16:03:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.959+0000 7fda127fc700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda141017f0 0x7fda14196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.960+0000 7fda137fe700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 0x7fda1419d9f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda0c007a00 tx=0x7fda0c0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.960+0000 7fd9fbfff700 1 -- 172.21.15.130:0/2780858805 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00e040 con 0x7fda14102190 2023-09-12T16:03:43.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.960+0000 7fd9fbfff700 1 -- 172.21.15.130:0/2780858805 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda0c012dc0 con 0x7fda14102190 2023-09-12T16:03:43.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.960+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1419e510 con 0x7fda14102190 2023-09-12T16:03:43.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.960+0000 7fd9fbfff700 1 -- 172.21.15.130:0/2780858805 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c01b450 con 0x7fda14102190 2023-09-12T16:03:43.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.960+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1419e990 con 0x7fda14102190 2023-09-12T16:03:43.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.961+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda14107b50 con 0x7fda14102190 2023-09-12T16:03:43.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.962+0000 7fd9fbfff700 1 -- 172.21.15.130:0/2780858805 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fda0c01d610 con 0x7fda14102190 2023-09-12T16:03:43.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.962+0000 7fd9fbfff700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd9fc0a0710 0x7fd9fc0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:43.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.962+0000 7fd9fbfff700 1 -- 172.21.15.130:0/2780858805 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fda0c0ce620 con 0x7fda14102190 2023-09-12T16:03:43.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.962+0000 7fda12ffd700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd9fc0a0710 0x7fd9fc0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:43.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.963+0000 7fda12ffd700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd9fc0a0710 0x7fd9fc0a2bd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fda08005950 tx=0x7fda080058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:43.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:43.966+0000 7fd9fbfff700 1 -- 172.21.15.130:0/2780858805 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fda0c097d70 con 0x7fda14102190 2023-09-12T16:03:44.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.158+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda1410c0e0 con 0x7fda14102190 2023-09-12T16:03:44.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.160+0000 7fd9fbfff700 1 -- 172.21.15.130:0/2780858805 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24473 (secure 0 0 0) 0x7fda0c09b390 con 0x7fda14102190 2023-09-12T16:03:44.161 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:44.161 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":88,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:51.826597+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:44.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd9fc0a0710 msgr2=0x7fd9fc0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd9fc0a0710 0x7fd9fc0a2bd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fda08005950 tx=0x7fda080058e0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 msgr2=0x7fda1419d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 0x7fda1419d9f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda0c007a00 tx=0x7fda0c0094f0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 shutdown_connections 2023-09-12T16:03:44.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd9fc0a0710 0x7fd9fc0a2bd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda14102190 0x7fda1419d9f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda14100600 0x7fda14196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 --2- 172.21.15.130:0/2780858805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda141017f0 0x7fda14196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 >> 172.21.15.130:0/2780858805 conn(0x7fda140fbb50 msgr2=0x7fda14104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:44.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 shutdown_connections 2023-09-12T16:03:44.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.162+0000 7fda1955d700 1 -- 172.21.15.130:0/2780858805 wait complete. 2023-09-12T16:03:44.174 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:43 smithi130 ceph-mon[98670]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:44.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:03:44.178 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:03:44.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:43 smithi159 ceph-mon[102522]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:44.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:43 smithi203 ceph-mon[103431]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-09-12T16:03:44.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.302+0000 7f79049d7700 1 -- 172.21.15.130:0/2731246107 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79000fe8a0 msgr2=0x7f79000fecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.302+0000 7f79049d7700 1 --2- 172.21.15.130:0/2731246107 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79000fe8a0 0x7f79000fecc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009990 tx=0x7f78f4009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.302+0000 7f79049d7700 1 -- 172.21.15.130:0/2731246107 shutdown_connections 2023-09-12T16:03:44.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.302+0000 7f79049d7700 1 --2- 172.21.15.130:0/2731246107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7900102210 0x7f7900108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.302+0000 7f79049d7700 1 --2- 172.21.15.130:0/2731246107 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79000fe8a0 0x7f79000fecc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.302+0000 7f79049d7700 1 --2- 172.21.15.130:0/2731246107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7900101870 0x7f7900101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.302+0000 7f79049d7700 1 -- 172.21.15.130:0/2731246107 >> 172.21.15.130:0/2731246107 conn(0x7f79000fa1d0 msgr2=0x7f79000fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:44.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.302+0000 7f79049d7700 1 -- 172.21.15.130:0/2731246107 shutdown_connections 2023-09-12T16:03:44.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.302+0000 7f79049d7700 1 -- 172.21.15.130:0/2731246107 wait complete. 2023-09-12T16:03:44.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f79049d7700 1 Processor -- start 2023-09-12T16:03:44.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f79049d7700 1 -- start start 2023-09-12T16:03:44.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f79049d7700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79000fe8a0 0x7f7900196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f79049d7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7900101870 0x7f79001968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f79049d7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7900102210 0x7f790019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f79049d7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790019df00 con 0x7f7900101870 2023-09-12T16:03:44.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f79049d7700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790019e040 con 0x7f79000fe8a0 2023-09-12T16:03:44.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f79049d7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790019e1d0 con 0x7f7900102210 2023-09-12T16:03:44.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fdd9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7900101870 0x7f79001968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fdd9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7900101870 0x7f79001968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55052/0 (socket says 172.21.15.130:55052) 2023-09-12T16:03:44.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fdd9b700 1 -- 172.21.15.130:0/2351904098 learned_addr learned my addr 172.21.15.130:0/2351904098 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:44.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fe59c700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79000fe8a0 0x7f7900196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fed9d700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7900102210 0x7f790019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fdd9b700 1 -- 172.21.15.130:0/2351904098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7900102210 msgr2=0x7f790019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fdd9b700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7900102210 0x7f790019d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fdd9b700 1 -- 172.21.15.130:0/2351904098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79000fe8a0 msgr2=0x7f7900196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fdd9b700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79000fe8a0 0x7f7900196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fdd9b700 1 -- 172.21.15.130:0/2351904098 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f40095f0 con 0x7f7900101870 2023-09-12T16:03:44.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.303+0000 7f78fdd9b700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7900101870 0x7f79001968c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f78f000d950 tx=0x7f78f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:44.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.304+0000 7f78ef7fe700 1 -- 172.21.15.130:0/2351904098 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0009940 con 0x7f7900101870 2023-09-12T16:03:44.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.304+0000 7f78ef7fe700 1 -- 172.21.15.130:0/2351904098 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78f0010460 con 0x7f7900101870 2023-09-12T16:03:44.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.304+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790019e4b0 con 0x7f7900101870 2023-09-12T16:03:44.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.304+0000 7f78ef7fe700 1 -- 172.21.15.130:0/2351904098 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000f620 con 0x7f7900101870 2023-09-12T16:03:44.311 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.304+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790019e9c0 con 0x7f7900101870 2023-09-12T16:03:44.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.304+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790010be00 con 0x7f7900101870 2023-09-12T16:03:44.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.306+0000 7f78ef7fe700 1 -- 172.21.15.130:0/2351904098 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f78f00108d0 con 0x7f7900101870 2023-09-12T16:03:44.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.306+0000 7f78ef7fe700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e80a06d0 0x7f78e80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.306+0000 7f78fe59c700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e80a06d0 0x7f78e80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.307+0000 7f78ef7fe700 1 -- 172.21.15.130:0/2351904098 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f78f00cbf10 con 0x7f7900101870 2023-09-12T16:03:44.313 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.307+0000 7f78fe59c700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e80a06d0 0x7f78e80a2b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f78f400b5c0 tx=0x7f78f4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:44.314 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.310+0000 7f78ef7fe700 1 -- 172.21.15.130:0/2351904098 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f78f0095570 con 0x7f7900101870 2023-09-12T16:03:44.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.497+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7900107690 con 0x7f7900101870 2023-09-12T16:03:44.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.498+0000 7f78ef7fe700 1 -- 172.21.15.130:0/2351904098 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78f00160a0 con 0x7f7900101870 2023-09-12T16:03:44.499 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:03:44.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e80a06d0 msgr2=0x7f78e80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e80a06d0 0x7f78e80a2b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f78f400b5c0 tx=0x7f78f4005fd0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7900101870 msgr2=0x7f79001968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7900101870 0x7f79001968c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f78f000d950 tx=0x7f78f000dc60 comp rx=0 tx=0).stop 2023-09-12T16:03:44.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 shutdown_connections 2023-09-12T16:03:44.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f78e80a06d0 0x7f78e80a2b90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7900102210 0x7f790019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79000fe8a0 0x7f7900196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 --2- 172.21.15.130:0/2351904098 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7900101870 0x7f79001968c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 >> 172.21.15.130:0/2351904098 conn(0x7f79000fa1d0 msgr2=0x7f79000fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:44.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 shutdown_connections 2023-09-12T16:03:44.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.500+0000 7f79049d7700 1 -- 172.21.15.130:0/2351904098 wait complete. 2023-09-12T16:03:44.512 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:03:44.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.638+0000 7f36e3e43700 1 -- 172.21.15.130:0/1160599735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc101030 msgr2=0x7f36dc1078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.638+0000 7f36e3e43700 1 --2- 172.21.15.130:0/1160599735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc101030 0x7f36dc1078e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f36d400b0d0 tx=0x7f36d400b3e0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.638+0000 7f36e3e43700 1 -- 172.21.15.130:0/1160599735 shutdown_connections 2023-09-12T16:03:44.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.638+0000 7f36e3e43700 1 --2- 172.21.15.130:0/1160599735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc101030 0x7f36dc1078e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.638+0000 7f36e3e43700 1 --2- 172.21.15.130:0/1160599735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36dc0ff4a0 0x7f36dc0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.638+0000 7f36e3e43700 1 --2- 172.21.15.130:0/1160599735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36dc100690 0x7f36dc100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.638+0000 7f36e3e43700 1 -- 172.21.15.130:0/1160599735 >> 172.21.15.130:0/1160599735 conn(0x7f36dc0faa10 msgr2=0x7f36dc0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:44.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.638+0000 7f36e3e43700 1 -- 172.21.15.130:0/1160599735 shutdown_connections 2023-09-12T16:03:44.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.638+0000 7f36e3e43700 1 -- 172.21.15.130:0/1160599735 wait complete. 2023-09-12T16:03:44.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e3e43700 1 Processor -- start 2023-09-12T16:03:44.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e3e43700 1 -- start start 2023-09-12T16:03:44.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e3e43700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36dc0ff4a0 0x7f36dc10b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e3e43700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc100690 0x7f36dc109a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e3e43700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36dc101030 0x7f36dc109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e3e43700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc10a750 con 0x7f36dc100690 2023-09-12T16:03:44.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e3e43700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc10a8c0 con 0x7f36dc101030 2023-09-12T16:03:44.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e3e43700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc1a0340 con 0x7f36dc0ff4a0 2023-09-12T16:03:44.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e13de700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc100690 0x7f36dc109a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e13de700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc100690 0x7f36dc109a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55066/0 (socket says 172.21.15.130:55066) 2023-09-12T16:03:44.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e13de700 1 -- 172.21.15.130:0/794709287 learned_addr learned my addr 172.21.15.130:0/794709287 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:44.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e1bdf700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36dc0ff4a0 0x7f36dc10b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e13de700 1 -- 172.21.15.130:0/794709287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36dc0ff4a0 msgr2=0x7f36dc10b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e13de700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36dc0ff4a0 0x7f36dc10b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e13de700 1 -- 172.21.15.130:0/794709287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36dc101030 msgr2=0x7f36dc109fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e23e0700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36dc101030 0x7f36dc109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e13de700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36dc101030 0x7f36dc109fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.639+0000 7f36e13de700 1 -- 172.21.15.130:0/794709287 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4009d00 con 0x7f36dc100690 2023-09-12T16:03:44.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.640+0000 7f36e13de700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc100690 0x7f36dc109a70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e9a0 tx=0x7f36d800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:44.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.640+0000 7f36d2ffd700 1 -- 172.21.15.130:0/794709287 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8009790 con 0x7f36dc100690 2023-09-12T16:03:44.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.640+0000 7f36d2ffd700 1 -- 172.21.15.130:0/794709287 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36d800cbd0 con 0x7f36dc100690 2023-09-12T16:03:44.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.640+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36dc1a0620 con 0x7f36dc100690 2023-09-12T16:03:44.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.640+0000 7f36d2ffd700 1 -- 172.21.15.130:0/794709287 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8018680 con 0x7f36dc100690 2023-09-12T16:03:44.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.640+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc1a0b30 con 0x7f36dc100690 2023-09-12T16:03:44.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.641+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36dc10bdf0 con 0x7f36dc100690 2023-09-12T16:03:44.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.642+0000 7f36d2ffd700 1 -- 172.21.15.130:0/794709287 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f36d800cd40 con 0x7f36dc100690 2023-09-12T16:03:44.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.642+0000 7f36d2ffd700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f36c80a0750 0x7f36c80a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.642+0000 7f36d2ffd700 1 -- 172.21.15.130:0/794709287 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f36d8014070 con 0x7f36dc100690 2023-09-12T16:03:44.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.643+0000 7f36e1bdf700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f36c80a0750 0x7f36c80a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.643+0000 7f36e1bdf700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f36c80a0750 0x7f36c80a2c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f36dc1004f0 tx=0x7f36cc005d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:44.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.646+0000 7f36d2ffd700 1 -- 172.21.15.130:0/794709287 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f36d8095f30 con 0x7f36dc100690 2023-09-12T16:03:44.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.837+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36dc1062b0 con 0x7f36dc100690 2023-09-12T16:03:44.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.837+0000 7f36d2ffd700 1 -- 172.21.15.130:0/794709287 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36d8099550 con 0x7f36dc100690 2023-09-12T16:03:44.839 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:44.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f36c80a0750 msgr2=0x7f36c80a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f36c80a0750 0x7f36c80a2c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f36dc1004f0 tx=0x7f36cc005d70 comp rx=0 tx=0).stop 2023-09-12T16:03:44.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc100690 msgr2=0x7f36dc109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc100690 0x7f36dc109a70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e9a0 tx=0x7f36d800ed60 comp rx=0 tx=0).stop 2023-09-12T16:03:44.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 shutdown_connections 2023-09-12T16:03:44.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f36c80a0750 0x7f36c80a2c10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36dc101030 0x7f36dc109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36dc0ff4a0 0x7f36dc10b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 --2- 172.21.15.130:0/794709287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36dc100690 0x7f36dc109a70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 >> 172.21.15.130:0/794709287 conn(0x7f36dc0faa10 msgr2=0x7f36dc0fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:44.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 shutdown_connections 2023-09-12T16:03:44.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.840+0000 7f36e3e43700 1 -- 172.21.15.130:0/794709287 wait complete. 2023-09-12T16:03:44.853 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:03:44.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.977+0000 7f4d57cb2700 1 -- 172.21.15.130:0/2638543912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50103a00 msgr2=0x7f4d50105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.977+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/2638543912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50103a00 0x7f4d50105df0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009990 tx=0x7f4d4c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.977+0000 7f4d57cb2700 1 -- 172.21.15.130:0/2638543912 shutdown_connections 2023-09-12T16:03:44.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.977+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/2638543912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50106330 0x7f4d50108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.977+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/2638543912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d501010d0 0x7f4d501034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.977+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/2638543912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50103a00 0x7f4d50105df0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.977+0000 7f4d57cb2700 1 -- 172.21.15.130:0/2638543912 >> 172.21.15.130:0/2638543912 conn(0x7f4d500faa60 msgr2=0x7f4d500fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:44.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.977+0000 7f4d57cb2700 1 -- 172.21.15.130:0/2638543912 shutdown_connections 2023-09-12T16:03:44.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.977+0000 7f4d57cb2700 1 -- 172.21.15.130:0/2638543912 wait complete. 2023-09-12T16:03:44.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d57cb2700 1 Processor -- start 2023-09-12T16:03:44.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d57cb2700 1 -- start start 2023-09-12T16:03:44.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d57cb2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d501010d0 0x7f4d501962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d57cb2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103a00 0x7f4d501967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d57cb2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50106330 0x7f4d5019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d57cb2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5019de30 con 0x7f4d50106330 2023-09-12T16:03:44.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d57cb2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5019df70 con 0x7f4d501010d0 2023-09-12T16:03:44.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d57cb2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5019e100 con 0x7f4d50103a00 2023-09-12T16:03:44.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5624f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50106330 0x7f4d5019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5624f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50106330 0x7f4d5019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55082/0 (socket says 172.21.15.130:55082) 2023-09-12T16:03:44.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5624f700 1 -- 172.21.15.130:0/3133748499 learned_addr learned my addr 172.21.15.130:0/3133748499 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:44.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d55a4e700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d501010d0 0x7f4d501962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5524d700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103a00 0x7f4d501967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5624f700 1 -- 172.21.15.130:0/3133748499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103a00 msgr2=0x7f4d501967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5624f700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103a00 0x7f4d501967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5624f700 1 -- 172.21.15.130:0/3133748499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d501010d0 msgr2=0x7f4d501962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:44.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5624f700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d501010d0 0x7f4d501962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:44.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5624f700 1 -- 172.21.15.130:0/3133748499 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c0095f0 con 0x7f4d50106330 2023-09-12T16:03:44.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d5624f700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50106330 0x7f4d5019d860 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4d48007ee0 tx=0x7f4d4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:44.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.978+0000 7f4d46ffd700 1 -- 172.21.15.130:0/3133748499 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48017070 con 0x7f4d50106330 2023-09-12T16:03:44.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.979+0000 7f4d46ffd700 1 -- 172.21.15.130:0/3133748499 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d4800f040 con 0x7f4d50106330 2023-09-12T16:03:44.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.979+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5019e3e0 con 0x7f4d50106330 2023-09-12T16:03:44.986 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.979+0000 7f4d46ffd700 1 -- 172.21.15.130:0/3133748499 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48005580 con 0x7f4d50106330 2023-09-12T16:03:44.986 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.979+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5019e8a0 con 0x7f4d50106330 2023-09-12T16:03:44.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.980+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d500fc5e0 con 0x7f4d50106330 2023-09-12T16:03:44.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.981+0000 7f4d46ffd700 1 -- 172.21.15.130:0/3133748499 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f4d4800db20 con 0x7f4d50106330 2023-09-12T16:03:44.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.981+0000 7f4d46ffd700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4d3c0a06d0 0x7f4d3c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:44.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.981+0000 7f4d55a4e700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4d3c0a06d0 0x7f4d3c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:44.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.981+0000 7f4d46ffd700 1 -- 172.21.15.130:0/3133748499 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4d480cb7c0 con 0x7f4d50106330 2023-09-12T16:03:44.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.982+0000 7f4d55a4e700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4d3c0a06d0 0x7f4d3c0a2b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d40005950 tx=0x7f4d4000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:44.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:44.985+0000 7f4d46ffd700 1 -- 172.21.15.130:0/3133748499 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4d48094e80 con 0x7f4d50106330 2023-09-12T16:03:45.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.182+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d5010c060 con 0x7f4d50106330 2023-09-12T16:03:45.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.182+0000 7f4d46ffd700 1 -- 172.21.15.130:0/3133748499 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d480984a0 con 0x7f4d50106330 2023-09-12T16:03:45.183 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:03:45.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4d3c0a06d0 msgr2=0x7f4d3c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4d3c0a06d0 0x7f4d3c0a2b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d40005950 tx=0x7f4d4000b410 comp rx=0 tx=0).stop 2023-09-12T16:03:45.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50106330 msgr2=0x7f4d5019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50106330 0x7f4d5019d860 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4d48007ee0 tx=0x7f4d4800d3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 shutdown_connections 2023-09-12T16:03:45.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4d3c0a06d0 0x7f4d3c0a2b90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d50106330 0x7f4d5019d860 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d501010d0 0x7f4d501962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 --2- 172.21.15.130:0/3133748499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103a00 0x7f4d501967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 >> 172.21.15.130:0/3133748499 conn(0x7f4d500faa60 msgr2=0x7f4d50101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:45.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.184+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 shutdown_connections 2023-09-12T16:03:45.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.185+0000 7f4d57cb2700 1 -- 172.21.15.130:0/3133748499 wait complete. 2023-09-12T16:03:45.196 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:44 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2780858805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:45.196 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:44 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2351904098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:45.197 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:44 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/794709287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:45.198 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:03:45.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:44 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2780858805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:45.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:44 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2351904098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:45.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:44 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/794709287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:45.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:44 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2780858805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:45.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:44 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2351904098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:45.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:44 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/794709287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:45.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.324+0000 7f94c763e700 1 -- 172.21.15.130:0/3266755448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01039a0 msgr2=0x7f94c0105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.324+0000 7f94c763e700 1 --2- 172.21.15.130:0/3266755448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01039a0 0x7f94c0105d90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009990 tx=0x7f94bc009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.325+0000 7f94c763e700 1 -- 172.21.15.130:0/3266755448 shutdown_connections 2023-09-12T16:03:45.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.325+0000 7f94c763e700 1 --2- 172.21.15.130:0/3266755448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c01062d0 0x7f94c0108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.325+0000 7f94c763e700 1 --2- 172.21.15.130:0/3266755448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94c0101070 0x7f94c0103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.325+0000 7f94c763e700 1 --2- 172.21.15.130:0/3266755448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01039a0 0x7f94c0105d90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.325+0000 7f94c763e700 1 -- 172.21.15.130:0/3266755448 >> 172.21.15.130:0/3266755448 conn(0x7f94c00faa60 msgr2=0x7f94c00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:45.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.325+0000 7f94c763e700 1 -- 172.21.15.130:0/3266755448 shutdown_connections 2023-09-12T16:03:45.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.325+0000 7f94c763e700 1 -- 172.21.15.130:0/3266755448 wait complete. 2023-09-12T16:03:45.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.325+0000 7f94c763e700 1 Processor -- start 2023-09-12T16:03:45.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.325+0000 7f94c763e700 1 -- start start 2023-09-12T16:03:45.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c763e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c0101070 0x7f94c01962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:45.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c763e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94c01039a0 0x7f94c01967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:45.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c763e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01062d0 0x7f94c019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:45.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c763e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c019de20 con 0x7f94c01062d0 2023-09-12T16:03:45.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c763e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c019df60 con 0x7f94c01039a0 2023-09-12T16:03:45.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c763e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c019e0f0 con 0x7f94c0101070 2023-09-12T16:03:45.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c5bdb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01062d0 0x7f94c019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:45.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c5bdb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01062d0 0x7f94c019d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55096/0 (socket says 172.21.15.130:55096) 2023-09-12T16:03:45.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c5bdb700 1 -- 172.21.15.130:0/3332044536 learned_addr learned my addr 172.21.15.130:0/3332044536 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:45.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c4bd9700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94c01039a0 0x7f94c01967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:45.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c53da700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c0101070 0x7f94c01962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:45.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c5bdb700 1 -- 172.21.15.130:0/3332044536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c0101070 msgr2=0x7f94c01962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c5bdb700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c0101070 0x7f94c01962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c5bdb700 1 -- 172.21.15.130:0/3332044536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94c01039a0 msgr2=0x7f94c01967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c5bdb700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94c01039a0 0x7f94c01967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c5bdb700 1 -- 172.21.15.130:0/3332044536 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc0095f0 con 0x7f94c01062d0 2023-09-12T16:03:45.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94c5bdb700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01062d0 0x7f94c019d850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f94b8007ee0 tx=0x7f94b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:45.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.326+0000 7f94b67fc700 1 -- 172.21.15.130:0/3332044536 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b800dd20 con 0x7f94c01062d0 2023-09-12T16:03:45.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.327+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c019e3d0 con 0x7f94c01062d0 2023-09-12T16:03:45.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.327+0000 7f94b67fc700 1 -- 172.21.15.130:0/3332044536 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94b800f040 con 0x7f94c01062d0 2023-09-12T16:03:45.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.327+0000 7f94b67fc700 1 -- 172.21.15.130:0/3332044536 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b80048a0 con 0x7f94c01062d0 2023-09-12T16:03:45.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.327+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c019e890 con 0x7f94c01062d0 2023-09-12T16:03:45.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.327+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c00fc5e0 con 0x7f94c01062d0 2023-09-12T16:03:45.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.328+0000 7f94b67fc700 1 -- 172.21.15.130:0/3332044536 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f94b8003a00 con 0x7f94c01062d0 2023-09-12T16:03:45.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.329+0000 7f94b67fc700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94ac0a06d0 0x7f94ac0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:45.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.329+0000 7f94c53da700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94ac0a06d0 0x7f94ac0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:45.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.329+0000 7f94b67fc700 1 -- 172.21.15.130:0/3332044536 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f94b80cb890 con 0x7f94c01062d0 2023-09-12T16:03:45.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.330+0000 7f94c53da700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94ac0a06d0 0x7f94ac0a2b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f94b000ac50 tx=0x7f94b000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:45.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.334+0000 7f94b67fc700 1 -- 172.21.15.130:0/3332044536 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f94b804ee10 con 0x7f94c01062d0 2023-09-12T16:03:45.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.524+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f94c002ce40 con 0x7f94c01062d0 2023-09-12T16:03:45.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.525+0000 7f94b67fc700 1 -- 172.21.15.130:0/3332044536 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f94b80b6e60 con 0x7f94c01062d0 2023-09-12T16:03:45.526 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:45.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94ac0a06d0 msgr2=0x7f94ac0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94ac0a06d0 0x7f94ac0a2b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f94b000ac50 tx=0x7f94b000a380 comp rx=0 tx=0).stop 2023-09-12T16:03:45.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01062d0 msgr2=0x7f94c019d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01062d0 0x7f94c019d850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f94b8007ee0 tx=0x7f94b800d3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 shutdown_connections 2023-09-12T16:03:45.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f94ac0a06d0 0x7f94ac0a2b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94c01062d0 0x7f94c019d850 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c0101070 0x7f94c01962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.529 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 --2- 172.21.15.130:0/3332044536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94c01039a0 0x7f94c01967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.530 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 >> 172.21.15.130:0/3332044536 conn(0x7f94c00faa60 msgr2=0x7f94c0101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:45.530 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 shutdown_connections 2023-09-12T16:03:45.530 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.527+0000 7f94c763e700 1 -- 172.21.15.130:0/3332044536 wait complete. 2023-09-12T16:03:45.541 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:45.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.662+0000 7fa4f6aba700 1 -- 172.21.15.130:0/1430667357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f00ff4e0 msgr2=0x7fa4f00ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.662+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/1430667357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f00ff4e0 0x7fa4f00ff900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009990 tx=0x7fa4e4009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.662+0000 7fa4f6aba700 1 -- 172.21.15.130:0/1430667357 shutdown_connections 2023-09-12T16:03:45.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.662+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/1430667357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4f0101070 0x7fa4f0107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.662+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/1430667357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f00ff4e0 0x7fa4f00ff900 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.662+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/1430667357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f01006d0 0x7fa4f0100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.662+0000 7fa4f6aba700 1 -- 172.21.15.130:0/1430667357 >> 172.21.15.130:0/1430667357 conn(0x7fa4f00faa30 msgr2=0x7fa4f00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:45.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.662+0000 7fa4f6aba700 1 -- 172.21.15.130:0/1430667357 shutdown_connections 2023-09-12T16:03:45.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.662+0000 7fa4f6aba700 1 -- 172.21.15.130:0/1430667357 wait complete. 2023-09-12T16:03:45.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f6aba700 1 Processor -- start 2023-09-12T16:03:45.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f6aba700 1 -- start start 2023-09-12T16:03:45.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f6aba700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f00ff4e0 0x7fa4f00682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:45.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f6aba700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4f01006d0 0x7fa4f019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:45.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f6aba700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f0101070 0x7fa4f019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:45.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f6aba700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f019ddd0 con 0x7fa4f01006d0 2023-09-12T16:03:45.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f6aba700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f019df40 con 0x7fa4f00ff4e0 2023-09-12T16:03:45.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f6aba700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f019e100 con 0x7fa4f0101070 2023-09-12T16:03:45.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4effff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4f01006d0 0x7fa4f019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:45.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4effff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4f01006d0 0x7fa4f019afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55108/0 (socket says 172.21.15.130:55108) 2023-09-12T16:03:45.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f4856700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f00ff4e0 0x7fa4f00682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:45.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4effff700 1 -- 172.21.15.130:0/2961648789 learned_addr learned my addr 172.21.15.130:0/2961648789 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:45.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4f5057700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f0101070 0x7fa4f019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:45.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4effff700 1 -- 172.21.15.130:0/2961648789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f0101070 msgr2=0x7fa4f019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4effff700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f0101070 0x7fa4f019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4effff700 1 -- 172.21.15.130:0/2961648789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f00ff4e0 msgr2=0x7fa4f00682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4effff700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f00ff4e0 0x7fa4f00682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.663+0000 7fa4effff700 1 -- 172.21.15.130:0/2961648789 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40095f0 con 0x7fa4f01006d0 2023-09-12T16:03:45.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.664+0000 7fa4effff700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4f01006d0 0x7fa4f019afb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00e9a0 tx=0x7fa4dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:45.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.664+0000 7fa4edffb700 1 -- 172.21.15.130:0/2961648789 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc009790 con 0x7fa4f01006d0 2023-09-12T16:03:45.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.664+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f019e3e0 con 0x7fa4f01006d0 2023-09-12T16:03:45.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.664+0000 7fa4edffb700 1 -- 172.21.15.130:0/2961648789 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4dc00cbd0 con 0x7fa4f01006d0 2023-09-12T16:03:45.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.664+0000 7fa4edffb700 1 -- 172.21.15.130:0/2961648789 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc018680 con 0x7fa4f01006d0 2023-09-12T16:03:45.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.664+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f019e940 con 0x7fa4f01006d0 2023-09-12T16:03:45.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.665+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f010acf0 con 0x7fa4f01006d0 2023-09-12T16:03:45.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.666+0000 7fa4edffb700 1 -- 172.21.15.130:0/2961648789 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fa4dc00cd40 con 0x7fa4f01006d0 2023-09-12T16:03:45.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.666+0000 7fa4edffb700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa4e00a0710 0x7fa4e00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:45.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.667+0000 7fa4f4856700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa4e00a0710 0x7fa4e00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:45.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.667+0000 7fa4edffb700 1 -- 172.21.15.130:0/2961648789 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa4dc014070 con 0x7fa4f01006d0 2023-09-12T16:03:45.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.667+0000 7fa4f4856700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa4e00a0710 0x7fa4e00a2bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400b2e0 tx=0x7fa4e4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:45.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.670+0000 7fa4edffb700 1 -- 172.21.15.130:0/2961648789 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa4dc095c20 con 0x7fa4f01006d0 2023-09-12T16:03:45.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.866+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4f019eb60 con 0x7fa4f01006d0 2023-09-12T16:03:45.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.868+0000 7fa4edffb700 1 -- 172.21.15.130:0/2961648789 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24473 (secure 0 0 0) 0x7fa4dc099240 con 0x7fa4f01006d0 2023-09-12T16:03:45.869 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:45.870 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":88,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:51.826597+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:45.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa4e00a0710 msgr2=0x7fa4e00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa4e00a0710 0x7fa4e00a2bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400b2e0 tx=0x7fa4e4005fb0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4f01006d0 msgr2=0x7fa4f019afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:45.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4f01006d0 0x7fa4f019afb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00e9a0 tx=0x7fa4dc00ecb0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 shutdown_connections 2023-09-12T16:03:45.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa4e00a0710 0x7fa4e00a2bd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f0101070 0x7fa4f019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f00ff4e0 0x7fa4f00682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 --2- 172.21.15.130:0/2961648789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4f01006d0 0x7fa4f019afb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:45.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 >> 172.21.15.130:0/2961648789 conn(0x7fa4f00faa30 msgr2=0x7fa4f00fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:45.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 shutdown_connections 2023-09-12T16:03:45.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:45.870+0000 7fa4f6aba700 1 -- 172.21.15.130:0/2961648789 wait complete. 2023-09-12T16:03:45.886 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:03:45.887 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:03:46.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 -- 172.21.15.130:0/3221053863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df01005f0 msgr2=0x7f5df0100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 --2- 172.21.15.130:0/3221053863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df01005f0 0x7f5df0100a10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5de00099c0 tx=0x7f5de0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 -- 172.21.15.130:0/3221053863 shutdown_connections 2023-09-12T16:03:46.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 --2- 172.21.15.130:0/3221053863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df0102180 0x7f5df0108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 --2- 172.21.15.130:0/3221053863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df01005f0 0x7f5df0100a10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 --2- 172.21.15.130:0/3221053863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df01017e0 0x7f5df0101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 -- 172.21.15.130:0/3221053863 >> 172.21.15.130:0/3221053863 conn(0x7f5df00fbb80 msgr2=0x7f5df00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:46.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 -- 172.21.15.130:0/3221053863 shutdown_connections 2023-09-12T16:03:46.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 -- 172.21.15.130:0/3221053863 wait complete. 2023-09-12T16:03:46.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.012+0000 7f5df5dae700 1 Processor -- start 2023-09-12T16:03:46.016 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5df5dae700 1 -- start start 2023-09-12T16:03:46.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5df5dae700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df01005f0 0x7f5df0196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5df5dae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df01017e0 0x7f5df01967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5df5dae700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0102180 0x7f5df019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5df5dae700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df019dde0 con 0x7f5df0102180 2023-09-12T16:03:46.017 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5df5dae700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df019df20 con 0x7f5df01005f0 2023-09-12T16:03:46.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5df5dae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df019e0b0 con 0x7f5df01017e0 2023-09-12T16:03:46.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0102180 0x7f5df019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0102180 0x7f5df019d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55122/0 (socket says 172.21.15.130:55122) 2023-09-12T16:03:46.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deffff700 1 -- 172.21.15.130:0/914912823 learned_addr learned my addr 172.21.15.130:0/914912823 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:46.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5def7fe700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df01005f0 0x7f5df0196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deeffd700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df01017e0 0x7f5df01967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deffff700 1 -- 172.21.15.130:0/914912823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df01017e0 msgr2=0x7f5df01967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deffff700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df01017e0 0x7f5df01967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deffff700 1 -- 172.21.15.130:0/914912823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df01005f0 msgr2=0x7f5df0196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deffff700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df01005f0 0x7f5df0196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deffff700 1 -- 172.21.15.130:0/914912823 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de00095f0 con 0x7f5df0102180 2023-09-12T16:03:46.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5def7fe700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df01005f0 0x7f5df0196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:46.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5deffff700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0102180 0x7f5df019d810 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800e940 tx=0x7f5dd800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:46.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.013+0000 7f5decff9700 1 -- 172.21.15.130:0/914912823 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80097b0 con 0x7f5df0102180 2023-09-12T16:03:46.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.014+0000 7f5decff9700 1 -- 172.21.15.130:0/914912823 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd800ee90 con 0x7f5df0102180 2023-09-12T16:03:46.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.014+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df019e390 con 0x7f5df0102180 2023-09-12T16:03:46.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.014+0000 7f5decff9700 1 -- 172.21.15.130:0/914912823 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8005520 con 0x7f5df0102180 2023-09-12T16:03:46.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.014+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df019e850 con 0x7f5df0102180 2023-09-12T16:03:46.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.014+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df010be20 con 0x7f5df0102180 2023-09-12T16:03:46.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.017+0000 7f5decff9700 1 -- 172.21.15.130:0/914912823 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f5dd8004750 con 0x7f5df0102180 2023-09-12T16:03:46.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.017+0000 7f5decff9700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ddc0a06d0 0x7f5ddc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.017+0000 7f5def7fe700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ddc0a06d0 0x7f5ddc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.017+0000 7f5decff9700 1 -- 172.21.15.130:0/914912823 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5dd8014070 con 0x7f5df0102180 2023-09-12T16:03:46.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.018+0000 7f5def7fe700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ddc0a06d0 0x7f5ddc0a2b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009990 tx=0x7f5de001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:46.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.020+0000 7f5decff9700 1 -- 172.21.15.130:0/914912823 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5dd8095a00 con 0x7f5df0102180 2023-09-12T16:03:46.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.211+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5df0107480 con 0x7f5df0102180 2023-09-12T16:03:46.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.212+0000 7f5decff9700 1 -- 172.21.15.130:0/914912823 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5dd8099020 con 0x7f5df0102180 2023-09-12T16:03:46.213 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:03:46.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.213+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ddc0a06d0 msgr2=0x7f5ddc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ddc0a06d0 0x7f5ddc0a2b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009990 tx=0x7f5de001a040 comp rx=0 tx=0).stop 2023-09-12T16:03:46.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0102180 msgr2=0x7f5df019d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0102180 0x7f5df019d810 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800e940 tx=0x7f5dd800ec50 comp rx=0 tx=0).stop 2023-09-12T16:03:46.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 shutdown_connections 2023-09-12T16:03:46.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5ddc0a06d0 0x7f5ddc0a2b90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0102180 0x7f5df019d810 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df01005f0 0x7f5df0196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 --2- 172.21.15.130:0/914912823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df01017e0 0x7f5df01967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 >> 172.21.15.130:0/914912823 conn(0x7f5df00fbb80 msgr2=0x7f5df00fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:46.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 shutdown_connections 2023-09-12T16:03:46.217 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.214+0000 7f5df5dae700 1 -- 172.21.15.130:0/914912823 wait complete. 2023-09-12T16:03:46.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:45 smithi130 ceph-mon[98670]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-12T16:03:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:45 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3133748499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:45 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3332044536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:45 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2961648789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:46.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:45 smithi159 ceph-mon[102522]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-12T16:03:46.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:45 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3133748499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:46.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:45 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3332044536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:46.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:45 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2961648789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:46.228 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:03:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:45 smithi203 ceph-mon[103431]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-12T16:03:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:45 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3133748499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:46.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:45 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3332044536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:46.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:45 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2961648789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:46.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.353+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3848066032 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 msgr2=0x7fa6b00ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.353+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3848066032 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 0x7fa6b00ff910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40099c0 tx=0x7fa6a4009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.353+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3848066032 shutdown_connections 2023-09-12T16:03:46.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.353+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3848066032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b0101080 0x7fa6b0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.353+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3848066032 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 0x7fa6b00ff910 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.353+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3848066032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b01006e0 0x7fa6b0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.353+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3848066032 >> 172.21.15.130:0/3848066032 conn(0x7fa6b00faa60 msgr2=0x7fa6b00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:46.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.353+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3848066032 shutdown_connections 2023-09-12T16:03:46.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.353+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3848066032 wait complete. 2023-09-12T16:03:46.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6b5d33700 1 Processor -- start 2023-09-12T16:03:46.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6b5d33700 1 -- start start 2023-09-12T16:03:46.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6b5d33700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 0x7fa6b0195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6b5d33700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b01006e0 0x7fa6b0195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6b5d33700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b0101080 0x7fa6b019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6b5d33700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b019cd80 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6b5d33700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b019cec0 con 0x7fa6b0101080 2023-09-12T16:03:46.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6b5d33700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b019d050 con 0x7fa6b01006e0 2023-09-12T16:03:46.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6af7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 0x7fa6b0195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6af7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 0x7fa6b0195200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55132/0 (socket says 172.21.15.130:55132) 2023-09-12T16:03:46.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.354+0000 7fa6af7fe700 1 -- 172.21.15.130:0/3586957134 learned_addr learned my addr 172.21.15.130:0/3586957134 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:46.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6aeffd700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b01006e0 0x7fa6b0195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6affff700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b0101080 0x7fa6b019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6af7fe700 1 -- 172.21.15.130:0/3586957134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b01006e0 msgr2=0x7fa6b0195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6af7fe700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b01006e0 0x7fa6b0195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6af7fe700 1 -- 172.21.15.130:0/3586957134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b0101080 msgr2=0x7fa6b019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6af7fe700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b0101080 0x7fa6b019c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6af7fe700 1 -- 172.21.15.130:0/3586957134 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40095f0 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6af7fe700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 0x7fa6b0195200 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009fd0 tx=0x7fa6a400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:46.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6acff9700 1 -- 172.21.15.130:0/3586957134 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a401d070 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b019d2d0 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6acff9700 1 -- 172.21.15.130:0/3586957134 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6a40054f0 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6acff9700 1 -- 172.21.15.130:0/3586957134 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4004bd0 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.355+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b019d750 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.356+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b0106ad0 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.357+0000 7fa6acff9700 1 -- 172.21.15.130:0/3586957134 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fa6a400fb80 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.358+0000 7fa6acff9700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa6980a06d0 0x7fa6980a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.358+0000 7fa6aeffd700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa6980a06d0 0x7fa6980a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.358+0000 7fa6acff9700 1 -- 172.21.15.130:0/3586957134 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa6a40cd2d0 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.358+0000 7fa6aeffd700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa6980a06d0 0x7fa6980a2b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009c60 tx=0x7fa6a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:46.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.361+0000 7fa6acff9700 1 -- 172.21.15.130:0/3586957134 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa6a40969c0 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.553 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.552+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6b010ae90 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.554 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.553+0000 7fa6acff9700 1 -- 172.21.15.130:0/3586957134 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6a4099fe0 con 0x7fa6b00ff4f0 2023-09-12T16:03:46.554 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:46.556 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa6980a06d0 msgr2=0x7fa6980a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.556 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa6980a06d0 0x7fa6980a2b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009c60 tx=0x7fa6a0009260 comp rx=0 tx=0).stop 2023-09-12T16:03:46.556 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 msgr2=0x7fa6b0195200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.556 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 0x7fa6b0195200 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009fd0 tx=0x7fa6a400f690 comp rx=0 tx=0).stop 2023-09-12T16:03:46.557 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 shutdown_connections 2023-09-12T16:03:46.557 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa6980a06d0 0x7fa6980a2b90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.557 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b0101080 0x7fa6b019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.557 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b00ff4f0 0x7fa6b0195200 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.557 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 --2- 172.21.15.130:0/3586957134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b01006e0 0x7fa6b0195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.558 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 >> 172.21.15.130:0/3586957134 conn(0x7fa6b00faa60 msgr2=0x7fa6b0103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:46.558 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 shutdown_connections 2023-09-12T16:03:46.558 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.555+0000 7fa6b5d33700 1 -- 172.21.15.130:0/3586957134 wait complete. 2023-09-12T16:03:46.569 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:03:46.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.694+0000 7f70a1607700 1 -- 172.21.15.130:0/3399079484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 msgr2=0x7f709c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.694+0000 7f70a1607700 1 --2- 172.21.15.130:0/3399079484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 0x7f709c100b40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f708c0099c0 tx=0x7f708c009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.694+0000 7f70a1607700 1 -- 172.21.15.130:0/3399079484 shutdown_connections 2023-09-12T16:03:46.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.694+0000 7f70a1607700 1 --2- 172.21.15.130:0/3399079484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f709c101080 0x7f709c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.694+0000 7f70a1607700 1 --2- 172.21.15.130:0/3399079484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f709c0ff4f0 0x7f709c0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.694+0000 7f70a1607700 1 --2- 172.21.15.130:0/3399079484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 0x7f709c100b40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.694+0000 7f70a1607700 1 -- 172.21.15.130:0/3399079484 >> 172.21.15.130:0/3399079484 conn(0x7f709c0faa60 msgr2=0x7f709c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:46.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.694+0000 7f70a1607700 1 -- 172.21.15.130:0/3399079484 shutdown_connections 2023-09-12T16:03:46.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.694+0000 7f70a1607700 1 -- 172.21.15.130:0/3399079484 wait complete. 2023-09-12T16:03:46.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f70a1607700 1 Processor -- start 2023-09-12T16:03:46.699 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f70a1607700 1 -- start start 2023-09-12T16:03:46.700 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f70a1607700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f709c0ff4f0 0x7f709c1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.700 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f70a1607700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 0x7f709c196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.700 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f70a1607700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f709c101080 0x7f709c19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.700 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f70a1607700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19de60 con 0x7f709c1006e0 2023-09-12T16:03:46.700 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f70a1607700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19dfa0 con 0x7f709c0ff4f0 2023-09-12T16:03:46.700 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f70a1607700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19e130 con 0x7f709c101080 2023-09-12T16:03:46.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f709a7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 0x7f709c196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f709a7fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 0x7f709c196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55148/0 (socket says 172.21.15.130:55148) 2023-09-12T16:03:46.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f709a7fc700 1 -- 172.21.15.130:0/2926351807 learned_addr learned my addr 172.21.15.130:0/2926351807 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:46.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.695+0000 7f709affd700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f709c0ff4f0 0x7f709c1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f709b7fe700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f709c101080 0x7f709c19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f709a7fc700 1 -- 172.21.15.130:0/2926351807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f709c101080 msgr2=0x7f709c19d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f709a7fc700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f709c101080 0x7f709c19d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f709a7fc700 1 -- 172.21.15.130:0/2926351807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f709c0ff4f0 msgr2=0x7f709c1962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f709a7fc700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f709c0ff4f0 0x7f709c1962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f709a7fc700 1 -- 172.21.15.130:0/2926351807 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0095f0 con 0x7f709c1006e0 2023-09-12T16:03:46.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f709a7fc700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 0x7f709c196820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f708c009fd0 tx=0x7f708c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:46.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f7093fff700 1 -- 172.21.15.130:0/2926351807 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c01d070 con 0x7f709c1006e0 2023-09-12T16:03:46.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c19e3b0 con 0x7f709c1006e0 2023-09-12T16:03:46.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f7093fff700 1 -- 172.21.15.130:0/2926351807 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f708c005030 con 0x7f709c1006e0 2023-09-12T16:03:46.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f7093fff700 1 -- 172.21.15.130:0/2926351807 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c004b30 con 0x7f709c1006e0 2023-09-12T16:03:46.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.696+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c19e830 con 0x7f709c1006e0 2023-09-12T16:03:46.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.697+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c10ad10 con 0x7f709c1006e0 2023-09-12T16:03:46.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.698+0000 7f7093fff700 1 -- 172.21.15.130:0/2926351807 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f708c0051a0 con 0x7f709c1006e0 2023-09-12T16:03:46.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.698+0000 7f7093fff700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f70880a06d0 0x7f70880a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:46.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.698+0000 7f7093fff700 1 -- 172.21.15.130:0/2926351807 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f708c0cce40 con 0x7f709c1006e0 2023-09-12T16:03:46.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.699+0000 7f709affd700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f70880a06d0 0x7f70880a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:46.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.699+0000 7f709affd700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f70880a06d0 0x7f70880a2b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f709c100540 tx=0x7f7084009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:46.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.703+0000 7f7093fff700 1 -- 172.21.15.130:0/2926351807 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f708c096590 con 0x7f709c1006e0 2023-09-12T16:03:46.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.895+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f709c10aed0 con 0x7f709c1006e0 2023-09-12T16:03:46.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.896+0000 7f7093fff700 1 -- 172.21.15.130:0/2926351807 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f709c10aed0 con 0x7f709c1006e0 2023-09-12T16:03:46.897 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:03:46.898 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f70880a06d0 msgr2=0x7f70880a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f70880a06d0 0x7f70880a2b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f709c100540 tx=0x7f7084009260 comp rx=0 tx=0).stop 2023-09-12T16:03:46.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 msgr2=0x7f709c196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:46.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 0x7f709c196820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f708c009fd0 tx=0x7f708c009580 comp rx=0 tx=0).stop 2023-09-12T16:03:46.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 shutdown_connections 2023-09-12T16:03:46.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f70880a06d0 0x7f70880a2b90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f709c101080 0x7f709c19d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f709c0ff4f0 0x7f709c1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 --2- 172.21.15.130:0/2926351807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f709c1006e0 0x7f709c196820 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:46.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 >> 172.21.15.130:0/2926351807 conn(0x7f709c0faa60 msgr2=0x7f709c068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:46.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 shutdown_connections 2023-09-12T16:03:46.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:46.898+0000 7f70a1607700 1 -- 172.21.15.130:0/2926351807 wait complete. 2023-09-12T16:03:46.911 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:03:47.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.035+0000 7f7b11c5e700 1 -- 172.21.15.130:0/2228487458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c100620 msgr2=0x7f7b0c100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.035+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/2228487458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c100620 0x7f7b0c100a40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0099c0 tx=0x7f7afc009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.035+0000 7f7b11c5e700 1 -- 172.21.15.130:0/2228487458 shutdown_connections 2023-09-12T16:03:47.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.035+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/2228487458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0c1021b0 0x7f7b0c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.035+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/2228487458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c100620 0x7f7b0c100a40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.035+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/2228487458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b0c101810 0x7f7b0c101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.035+0000 7f7b11c5e700 1 -- 172.21.15.130:0/2228487458 >> 172.21.15.130:0/2228487458 conn(0x7f7b0c0fbb70 msgr2=0x7f7b0c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:47.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.035+0000 7f7b11c5e700 1 -- 172.21.15.130:0/2228487458 shutdown_connections 2023-09-12T16:03:47.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.035+0000 7f7b11c5e700 1 -- 172.21.15.130:0/2228487458 wait complete. 2023-09-12T16:03:47.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b11c5e700 1 Processor -- start 2023-09-12T16:03:47.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b11c5e700 1 -- start start 2023-09-12T16:03:47.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b11c5e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0c100620 0x7f7b0c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b11c5e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b0c101810 0x7f7b0c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b11c5e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c1021b0 0x7f7b0c19d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b11c5e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c19dea0 con 0x7f7b0c1021b0 2023-09-12T16:03:47.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b11c5e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c19dfe0 con 0x7f7b0c101810 2023-09-12T16:03:47.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b11c5e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c19e170 con 0x7f7b0c100620 2023-09-12T16:03:47.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b0bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c1021b0 0x7f7b0c19d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b0bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c1021b0 0x7f7b0c19d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55164/0 (socket says 172.21.15.130:55164) 2023-09-12T16:03:47.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.036+0000 7f7b0bfff700 1 -- 172.21.15.130:0/378061592 learned_addr learned my addr 172.21.15.130:0/378061592 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:47.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0b7fe700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0c100620 0x7f7b0c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0affd700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b0c101810 0x7f7b0c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0bfff700 1 -- 172.21.15.130:0/378061592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0c100620 msgr2=0x7f7b0c196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0bfff700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0c100620 0x7f7b0c196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0bfff700 1 -- 172.21.15.130:0/378061592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b0c101810 msgr2=0x7f7b0c196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0bfff700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b0c101810 0x7f7b0c196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0bfff700 1 -- 172.21.15.130:0/378061592 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0095f0 con 0x7f7b0c1021b0 2023-09-12T16:03:47.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0affd700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b0c101810 0x7f7b0c196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:47.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0b7fe700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0c100620 0x7f7b0c196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:47.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b0bfff700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c1021b0 0x7f7b0c19d8d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400ef30 tx=0x7f7b0400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:47.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b08ff9700 1 -- 172.21.15.130:0/378061592 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04018070 con 0x7f7b0c1021b0 2023-09-12T16:03:47.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c19e3f0 con 0x7f7b0c1021b0 2023-09-12T16:03:47.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b08ff9700 1 -- 172.21.15.130:0/378061592 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b0400f040 con 0x7f7b0c1021b0 2023-09-12T16:03:47.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b08ff9700 1 -- 172.21.15.130:0/378061592 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04003dc0 con 0x7f7b0c1021b0 2023-09-12T16:03:47.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.037+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c19e8b0 con 0x7f7b0c1021b0 2023-09-12T16:03:47.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.038+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c107c00 con 0x7f7b0c1021b0 2023-09-12T16:03:47.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.039+0000 7f7b08ff9700 1 -- 172.21.15.130:0/378061592 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f7b0400caa0 con 0x7f7b0c1021b0 2023-09-12T16:03:47.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.040+0000 7f7b08ff9700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a06d0 0x7f7af40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.040+0000 7f7b0b7fe700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a06d0 0x7f7af40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.040+0000 7f7b08ff9700 1 -- 172.21.15.130:0/378061592 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7b040cc6c0 con 0x7f7b0c1021b0 2023-09-12T16:03:47.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.040+0000 7f7b0b7fe700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a06d0 0x7f7af40a2b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009990 tx=0x7f7afc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:47.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.043+0000 7f7b08ff9700 1 -- 172.21.15.130:0/378061592 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7b04095e10 con 0x7f7b0c1021b0 2023-09-12T16:03:47.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:46 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/914912823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:47.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:46 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3586957134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:47.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:46 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/914912823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:47.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:46 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3586957134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:47.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.237+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7b0c10c020 con 0x7f7b0c1021b0 2023-09-12T16:03:47.276 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:46 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/914912823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:47.277 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:46 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3586957134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:47.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.238+0000 7f7b08ff9700 1 -- 172.21.15.130:0/378061592 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7b04099430 con 0x7f7b0c1021b0 2023-09-12T16:03:47.277 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:47.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a06d0 msgr2=0x7f7af40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a06d0 0x7f7af40a2b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009990 tx=0x7f7afc01a040 comp rx=0 tx=0).stop 2023-09-12T16:03:47.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c1021b0 msgr2=0x7f7b0c19d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c1021b0 0x7f7b0c19d8d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400ef30 tx=0x7f7b0400c450 comp rx=0 tx=0).stop 2023-09-12T16:03:47.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 shutdown_connections 2023-09-12T16:03:47.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7af40a06d0 0x7f7af40a2b90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b0c1021b0 0x7f7b0c19d8d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0c100620 0x7f7b0c196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 --2- 172.21.15.130:0/378061592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b0c101810 0x7f7b0c196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 >> 172.21.15.130:0/378061592 conn(0x7f7b0c0fbb70 msgr2=0x7f7b0c104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:47.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 shutdown_connections 2023-09-12T16:03:47.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.240+0000 7f7b11c5e700 1 -- 172.21.15.130:0/378061592 wait complete. 2023-09-12T16:03:47.282 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-09-12T16:03:47.283 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:03:47.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.404+0000 7fc39c846700 1 -- 172.21.15.130:0/3973246600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3981006e0 msgr2=0x7fc398100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.404+0000 7fc39c846700 1 --2- 172.21.15.130:0/3973246600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3981006e0 0x7fc398100b40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc3880099c0 tx=0x7fc388009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.404+0000 7fc39c846700 1 -- 172.21.15.130:0/3973246600 shutdown_connections 2023-09-12T16:03:47.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.404+0000 7fc39c846700 1 --2- 172.21.15.130:0/3973246600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc398101080 0x7fc398107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.404+0000 7fc39c846700 1 --2- 172.21.15.130:0/3973246600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3980ff4f0 0x7fc3980ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.404+0000 7fc39c846700 1 --2- 172.21.15.130:0/3973246600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc3981006e0 0x7fc398100b40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.404+0000 7fc39c846700 1 -- 172.21.15.130:0/3973246600 >> 172.21.15.130:0/3973246600 conn(0x7fc3980faa20 msgr2=0x7fc3980fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:47.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.404+0000 7fc39c846700 1 -- 172.21.15.130:0/3973246600 shutdown_connections 2023-09-12T16:03:47.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.404+0000 7fc39c846700 1 -- 172.21.15.130:0/3973246600 wait complete. 2023-09-12T16:03:47.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc39c846700 1 Processor -- start 2023-09-12T16:03:47.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc39c846700 1 -- start start 2023-09-12T16:03:47.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc39c846700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3980ff4f0 0x7fc398195170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc39c846700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3981006e0 0x7fc3981956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc39c846700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc398101080 0x7fc39819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc39c846700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39819ccf0 con 0x7fc398101080 2023-09-12T16:03:47.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc39c846700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39819ce30 con 0x7fc3981006e0 2023-09-12T16:03:47.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc39c846700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39819cfc0 con 0x7fc3980ff4f0 2023-09-12T16:03:47.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc396d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc398101080 0x7fc39819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc396d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc398101080 0x7fc39819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55178/0 (socket says 172.21.15.130:55178) 2023-09-12T16:03:47.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc396d9d700 1 -- 172.21.15.130:0/2333250305 learned_addr learned my addr 172.21.15.130:0/2333250305 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:47.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc39659c700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3980ff4f0 0x7fc398195170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.405+0000 7fc395d9b700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3981006e0 0x7fc3981956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc396d9d700 1 -- 172.21.15.130:0/2333250305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3980ff4f0 msgr2=0x7fc398195170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc396d9d700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3980ff4f0 0x7fc398195170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc396d9d700 1 -- 172.21.15.130:0/2333250305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3981006e0 msgr2=0x7fc3981956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc396d9d700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3981006e0 0x7fc3981956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc396d9d700 1 -- 172.21.15.130:0/2333250305 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3880095f0 con 0x7fc398101080 2023-09-12T16:03:47.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc396d9d700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc398101080 0x7fc39819c720 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc390007ee0 tx=0x7fc39000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:47.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc3877fe700 1 -- 172.21.15.130:0/2333250305 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390017070 con 0x7fc398101080 2023-09-12T16:03:47.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc3877fe700 1 -- 172.21.15.130:0/2333250305 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc39000f040 con 0x7fc398101080 2023-09-12T16:03:47.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39819d2a0 con 0x7fc398101080 2023-09-12T16:03:47.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc3877fe700 1 -- 172.21.15.130:0/2333250305 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390005580 con 0x7fc398101080 2023-09-12T16:03:47.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.406+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39819d7b0 con 0x7fc398101080 2023-09-12T16:03:47.416 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.407+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc398106a30 con 0x7fc398101080 2023-09-12T16:03:47.416 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.408+0000 7fc3877fe700 1 -- 172.21.15.130:0/2333250305 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fc39000db20 con 0x7fc398101080 2023-09-12T16:03:47.416 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.409+0000 7fc3877fe700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3800a0710 0x7fc3800a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.416 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.409+0000 7fc39659c700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3800a0710 0x7fc3800a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.417 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.409+0000 7fc3877fe700 1 -- 172.21.15.130:0/2333250305 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc3900cb7d0 con 0x7fc398101080 2023-09-12T16:03:47.417 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.409+0000 7fc39659c700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3800a0710 0x7fc3800a2bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc398100540 tx=0x7fc38c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:47.417 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.413+0000 7fc3877fe700 1 -- 172.21.15.130:0/2333250305 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc390094e30 con 0x7fc398101080 2023-09-12T16:03:47.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.646+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc39810ad10 con 0x7fc398101080 2023-09-12T16:03:47.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.647+0000 7fc3877fe700 1 -- 172.21.15.130:0/2333250305 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11342 (secure 0 0 0) 0x7fc390098450 con 0x7fc398101080 2023-09-12T16:03:47.648 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:47.648 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:25.646573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:03:47.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3800a0710 msgr2=0x7fc3800a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3800a0710 0x7fc3800a2bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc398100540 tx=0x7fc38c00b410 comp rx=0 tx=0).stop 2023-09-12T16:03:47.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc398101080 msgr2=0x7fc39819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc398101080 0x7fc39819c720 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc390007ee0 tx=0x7fc39000d3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 shutdown_connections 2023-09-12T16:03:47.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc3800a0710 0x7fc3800a2bd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc398101080 0x7fc39819c720 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3980ff4f0 0x7fc398195170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 --2- 172.21.15.130:0/2333250305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3981006e0 0x7fc3981956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.649+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 >> 172.21.15.130:0/2333250305 conn(0x7fc3980faa20 msgr2=0x7fc398103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:47.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.650+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 shutdown_connections 2023-09-12T16:03:47.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.650+0000 7fc39c846700 1 -- 172.21.15.130:0/2333250305 wait complete. 2023-09-12T16:03:47.653 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 24 2023-09-12T16:03:47.663 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-09-12T16:03:47.663 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-09-12T16:03:47.663 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T16:03:47.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 -- 172.21.15.130:0/1622023020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c1006d0 msgr2=0x7fa85c100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 --2- 172.21.15.130:0/1622023020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c1006d0 0x7fa85c100b30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa8480099c0 tx=0x7fa848009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 -- 172.21.15.130:0/1622023020 shutdown_connections 2023-09-12T16:03:47.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 --2- 172.21.15.130:0/1622023020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa85c101070 0x7fa85c107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 --2- 172.21.15.130:0/1622023020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c0ff4e0 0x7fa85c0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 --2- 172.21.15.130:0/1622023020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c1006d0 0x7fa85c100b30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 -- 172.21.15.130:0/1622023020 >> 172.21.15.130:0/1622023020 conn(0x7fa85c0faa30 msgr2=0x7fa85c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:47.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 -- 172.21.15.130:0/1622023020 shutdown_connections 2023-09-12T16:03:47.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 -- 172.21.15.130:0/1622023020 wait complete. 2023-09-12T16:03:47.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 Processor -- start 2023-09-12T16:03:47.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.785+0000 7fa862d59700 1 -- start start 2023-09-12T16:03:47.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa862d59700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c0ff4e0 0x7fa85c068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa862d59700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c1006d0 0x7fa85c19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa862d59700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa85c101070 0x7fa85c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa862d59700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c19de10 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa862d59700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c19df80 con 0x7fa85c1006d0 2023-09-12T16:03:47.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa862d59700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c19e140 con 0x7fa85c101070 2023-09-12T16:03:47.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa860af5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c0ff4e0 0x7fa85c068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa860af5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c0ff4e0 0x7fa85c068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55188/0 (socket says 172.21.15.130:55188) 2023-09-12T16:03:47.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa860af5700 1 -- 172.21.15.130:0/2937853963 learned_addr learned my addr 172.21.15.130:0/2937853963 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:47.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa85bfff700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c1006d0 0x7fa85c19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa8612f6700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa85c101070 0x7fa85c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa860af5700 1 -- 172.21.15.130:0/2937853963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa85c101070 msgr2=0x7fa85c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa860af5700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa85c101070 0x7fa85c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa860af5700 1 -- 172.21.15.130:0/2937853963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c1006d0 msgr2=0x7fa85c19afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:47.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa860af5700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c1006d0 0x7fa85c19afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:47.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa860af5700 1 -- 172.21.15.130:0/2937853963 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8480095f0 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa8612f6700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa85c101070 0x7fa85c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:47.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa85bfff700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c1006d0 0x7fa85c19afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:47.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa860af5700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c0ff4e0 0x7fa85c068330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa85000d980 tx=0x7fa85000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:47.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa859ffb700 1 -- 172.21.15.130:0/2937853963 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85000bb60 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa859ffb700 1 -- 172.21.15.130:0/2937853963 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa850007620 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c19e420 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.786+0000 7fa859ffb700 1 -- 172.21.15.130:0/2937853963 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85000f620 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.787+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c19e930 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.788+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85c106a70 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.788+0000 7fa859ffb700 1 -- 172.21.15.130:0/2937853963 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fa850010460 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.789+0000 7fa859ffb700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa84c0a0710 0x7fa84c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:47.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.789+0000 7fa85bfff700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa84c0a0710 0x7fa84c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:47.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.789+0000 7fa859ffb700 1 -- 172.21.15.130:0/2937853963 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa8500cc080 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.790+0000 7fa85bfff700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa84c0a0710 0x7fa84c0a2bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa848006010 tx=0x7fa848000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:47.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.793+0000 7fa859ffb700 1 -- 172.21.15.130:0/2937853963 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa8500fee10 con 0x7fa85c0ff4e0 2023-09-12T16:03:47.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:47 smithi130 ceph-mon[98670]: pgmap v316: 129 pgs: 129 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-12T16:03:47.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:47 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2926351807' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:47.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:47 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/378061592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:47.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:47 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2333250305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:47.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.983+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa85c02cfd0 con 0x7fa84c0a0710 2023-09-12T16:03:47.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:47.996+0000 7fa859ffb700 1 -- 172.21.15.130:0/2937853963 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427369 (secure 0 0 0) 0x7fa85c02cfd0 con 0x7fa84c0a0710 2023-09-12T16:03:48.003 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:48.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa84c0a0710 msgr2=0x7fa84c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa84c0a0710 0x7fa84c0a2bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa848006010 tx=0x7fa848000bc0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c0ff4e0 msgr2=0x7fa85c068330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c0ff4e0 0x7fa85c068330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa85000d980 tx=0x7fa85000dd40 comp rx=0 tx=0).stop 2023-09-12T16:03:48.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 shutdown_connections 2023-09-12T16:03:48.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa84c0a0710 0x7fa84c0a2bd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa85c101070 0x7fa85c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa85c0ff4e0 0x7fa85c068330 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 --2- 172.21.15.130:0/2937853963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c1006d0 0x7fa85c19afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 >> 172.21.15.130:0/2937853963 conn(0x7fa85c0faa30 msgr2=0x7fa85c103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:48.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 shutdown_connections 2023-09-12T16:03:48.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.000+0000 7fa862d59700 1 -- 172.21.15.130:0/2937853963 wait complete. 2023-09-12T16:03:48.009 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2023-09-12T16:03:48.013 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2023-09-12T16:03:46.782696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660527,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":615,"num_read_kb":427,"num_write":341,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1240551,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335964,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605860,"statfs":{"total":1151940427776,"available":1151596400640,"internally_reserved":0,"allocated":11980800,"data_stored":7819538,"data_compressed":363759,"data_compressed_allocated":491520,"data_compressed_original":2514944,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4638,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":46,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003026"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025907+0000","last_change":"2023-09-12T16:02:05.039617+0000","last_active":"2023-09-12T16:02:51.025907+0000","last_peered":"2023-09-12T16:02:51.025907+0000","last_clean":"2023-09-12T16:02:51.025907+0000","last_became_active":"2023-09-12T16:02:05.039264+0000","last_became_peered":"2023-09-12T16:02:05.039264+0000","last_unstale":"2023-09-12T16:02:51.025907+0000","last_undegraded":"2023-09-12T16:02:51.025907+0000","last_fullsized":"2023-09-12T16:02:51.025907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:37:01.381406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829291+0000","last_change":"2023-09-12T16:02:05.033151+0000","last_active":"2023-09-12T16:02:50.829291+0000","last_peered":"2023-09-12T16:02:50.829291+0000","last_clean":"2023-09-12T16:02:50.829291+0000","last_became_active":"2023-09-12T16:02:05.032866+0000","last_became_peered":"2023-09-12T16:02:05.032866+0000","last_unstale":"2023-09-12T16:02:50.829291+0000","last_undegraded":"2023-09-12T16:02:50.829291+0000","last_fullsized":"2023-09-12T16:02:50.829291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:35.921428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.032478+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.032124+0000","last_became_peered":"2023-09-12T16:02:05.032124+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:58:54.112577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835786+0000","last_change":"2023-09-12T16:02:05.033088+0000","last_active":"2023-09-12T16:02:50.835786+0000","last_peered":"2023-09-12T16:02:50.835786+0000","last_clean":"2023-09-12T16:02:50.835786+0000","last_became_active":"2023-09-12T16:02:05.032332+0000","last_became_peered":"2023-09-12T16:02:05.032332+0000","last_unstale":"2023-09-12T16:02:50.835786+0000","last_undegraded":"2023-09-12T16:02:50.835786+0000","last_fullsized":"2023-09-12T16:02:50.835786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:25:38.751961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:05.036612+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:05.036215+0000","last_became_peered":"2023-09-12T16:02:05.036215+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:15:04.561751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832126+0000","last_change":"2023-09-12T16:02:05.043816+0000","last_active":"2023-09-12T16:02:50.832126+0000","last_peered":"2023-09-12T16:02:50.832126+0000","last_clean":"2023-09-12T16:02:50.832126+0000","last_became_active":"2023-09-12T16:02:05.043599+0000","last_became_peered":"2023-09-12T16:02:05.043599+0000","last_unstale":"2023-09-12T16:02:50.832126+0000","last_undegraded":"2023-09-12T16:02:50.832126+0000","last_fullsized":"2023-09-12T16:02:50.832126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:32:00.286196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829345+0000","last_change":"2023-09-12T16:02:05.031476+0000","last_active":"2023-09-12T16:02:50.829345+0000","last_peered":"2023-09-12T16:02:50.829345+0000","last_clean":"2023-09-12T16:02:50.829345+0000","last_became_active":"2023-09-12T16:02:05.031150+0000","last_became_peered":"2023-09-12T16:02:05.031150+0000","last_unstale":"2023-09-12T16:02:50.829345+0000","last_undegraded":"2023-09-12T16:02:50.829345+0000","last_fullsized":"2023-09-12T16:02:50.829345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:22:27.405505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831825+0000","last_change":"2023-09-12T16:02:05.031981+0000","last_active":"2023-09-12T16:02:50.831825+0000","last_peered":"2023-09-12T16:02:50.831825+0000","last_clean":"2023-09-12T16:02:50.831825+0000","last_became_active":"2023-09-12T16:02:05.031692+0000","last_became_peered":"2023-09-12T16:02:05.031692+0000","last_unstale":"2023-09-12T16:02:50.831825+0000","last_undegraded":"2023-09-12T16:02:50.831825+0000","last_fullsized":"2023-09-12T16:02:50.831825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:51:00.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838117+0000","last_change":"2023-09-12T16:02:05.029898+0000","last_active":"2023-09-12T16:02:50.838117+0000","last_peered":"2023-09-12T16:02:50.838117+0000","last_clean":"2023-09-12T16:02:50.838117+0000","last_became_active":"2023-09-12T16:02:05.029731+0000","last_became_peered":"2023-09-12T16:02:05.029731+0000","last_unstale":"2023-09-12T16:02:50.838117+0000","last_undegraded":"2023-09-12T16:02:50.838117+0000","last_fullsized":"2023-09-12T16:02:50.838117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:16:31.194561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833228+0000","last_change":"2023-09-12T16:02:05.044846+0000","last_active":"2023-09-12T16:02:50.833228+0000","last_peered":"2023-09-12T16:02:50.833228+0000","last_clean":"2023-09-12T16:02:50.833228+0000","last_became_active":"2023-09-12T16:02:05.044430+0000","last_became_peered":"2023-09-12T16:02:05.044430+0000","last_unstale":"2023-09-12T16:02:50.833228+0000","last_undegraded":"2023-09-12T16:02:50.833228+0000","last_fullsized":"2023-09-12T16:02:50.833228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:15.999080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830392+0000","last_change":"2023-09-12T16:02:05.030285+0000","last_active":"2023-09-12T16:02:50.830392+0000","last_peered":"2023-09-12T16:02:50.830392+0000","last_clean":"2023-09-12T16:02:50.830392+0000","last_became_active":"2023-09-12T16:02:05.029585+0000","last_became_peered":"2023-09-12T16:02:05.029585+0000","last_unstale":"2023-09-12T16:02:50.830392+0000","last_undegraded":"2023-09-12T16:02:50.830392+0000","last_fullsized":"2023-09-12T16:02:50.830392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:52:27.471412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835453+0000","last_change":"2023-09-12T16:02:05.037258+0000","last_active":"2023-09-12T16:02:50.835453+0000","last_peered":"2023-09-12T16:02:50.835453+0000","last_clean":"2023-09-12T16:02:50.835453+0000","last_became_active":"2023-09-12T16:02:05.036941+0000","last_became_peered":"2023-09-12T16:02:05.036941+0000","last_unstale":"2023-09-12T16:02:50.835453+0000","last_undegraded":"2023-09-12T16:02:50.835453+0000","last_fullsized":"2023-09-12T16:02:50.835453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:04:17.998927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835219+0000","last_change":"2023-09-12T16:02:05.041986+0000","last_active":"2023-09-12T16:02:50.835219+0000","last_peered":"2023-09-12T16:02:50.835219+0000","last_clean":"2023-09-12T16:02:50.835219+0000","last_became_active":"2023-09-12T16:02:05.041584+0000","last_became_peered":"2023-09-12T16:02:05.041584+0000","last_unstale":"2023-09-12T16:02:50.835219+0000","last_undegraded":"2023-09-12T16:02:50.835219+0000","last_fullsized":"2023-09-12T16:02:50.835219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:39.065525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828785+0000","last_change":"2023-09-12T16:02:05.030405+0000","last_active":"2023-09-12T16:02:50.828785+0000","last_peered":"2023-09-12T16:02:50.828785+0000","last_clean":"2023-09-12T16:02:50.828785+0000","last_became_active":"2023-09-12T16:02:05.029488+0000","last_became_peered":"2023-09-12T16:02:05.029488+0000","last_unstale":"2023-09-12T16:02:50.828785+0000","last_undegraded":"2023-09-12T16:02:50.828785+0000","last_fullsized":"2023-09-12T16:02:50.828785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.646069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829401+0000","last_change":"2023-09-12T16:02:05.032576+0000","last_active":"2023-09-12T16:02:50.829401+0000","last_peered":"2023-09-12T16:02:50.829401+0000","last_clean":"2023-09-12T16:02:50.829401+0000","last_became_active":"2023-09-12T16:02:05.032277+0000","last_became_peered":"2023-09-12T16:02:05.032277+0000","last_unstale":"2023-09-12T16:02:50.829401+0000","last_undegraded":"2023-09-12T16:02:50.829401+0000","last_fullsized":"2023-09-12T16:02:50.829401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:47:32.734830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836039+0000","last_change":"2023-09-12T16:02:05.023468+0000","last_active":"2023-09-12T16:02:50.836039+0000","last_peered":"2023-09-12T16:02:50.836039+0000","last_clean":"2023-09-12T16:02:50.836039+0000","last_became_active":"2023-09-12T16:02:05.023132+0000","last_became_peered":"2023-09-12T16:02:05.023132+0000","last_unstale":"2023-09-12T16:02:50.836039+0000","last_undegraded":"2023-09-12T16:02:50.836039+0000","last_fullsized":"2023-09-12T16:02:50.836039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:21:49.378779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025810+0000","last_change":"2023-09-12T16:02:05.039910+0000","last_active":"2023-09-12T16:02:51.025810+0000","last_peered":"2023-09-12T16:02:51.025810+0000","last_clean":"2023-09-12T16:02:51.025810+0000","last_became_active":"2023-09-12T16:02:05.039605+0000","last_became_peered":"2023-09-12T16:02:05.039605+0000","last_unstale":"2023-09-12T16:02:51.025810+0000","last_undegraded":"2023-09-12T16:02:51.025810+0000","last_fullsized":"2023-09-12T16:02:51.025810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:19:58.823471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833500+0000","last_change":"2023-09-12T16:02:05.031667+0000","last_active":"2023-09-12T16:02:50.833500+0000","last_peered":"2023-09-12T16:02:50.833500+0000","last_clean":"2023-09-12T16:02:50.833500+0000","last_became_active":"2023-09-12T16:02:05.030886+0000","last_became_peered":"2023-09-12T16:02:05.030886+0000","last_unstale":"2023-09-12T16:02:50.833500+0000","last_undegraded":"2023-09-12T16:02:50.833500+0000","last_fullsized":"2023-09-12T16:02:50.833500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:24:02.312902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836131+0000","last_change":"2023-09-12T16:02:05.037607+0000","last_active":"2023-09-12T16:02:50.836131+0000","last_peered":"2023-09-12T16:02:50.836131+0000","last_clean":"2023-09-12T16:02:50.836131+0000","last_became_active":"2023-09-12T16:02:05.037095+0000","last_became_peered":"2023-09-12T16:02:05.037095+0000","last_unstale":"2023-09-12T16:02:50.836131+0000","last_undegraded":"2023-09-12T16:02:50.836131+0000","last_fullsized":"2023-09-12T16:02:50.836131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:17:04.886737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834805+0000","last_change":"2023-09-12T16:02:09.292093+0000","last_active":"2023-09-12T16:02:50.834805+0000","last_peered":"2023-09-12T16:02:50.834805+0000","last_clean":"2023-09-12T16:02:50.834805+0000","last_became_active":"2023-09-12T16:02:09.291602+0000","last_became_peered":"2023-09-12T16:02:09.291602+0000","last_unstale":"2023-09-12T16:02:50.834805+0000","last_undegraded":"2023-09-12T16:02:50.834805+0000","last_fullsized":"2023-09-12T16:02:50.834805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:36:23.089498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.857296+0000","last_change":"2023-09-12T16:02:51.857296+0000","last_active":"2023-09-12T16:02:51.857296+0000","last_peered":"2023-09-12T16:02:51.857296+0000","last_clean":"2023-09-12T16:02:51.857296+0000","last_became_active":"2023-09-12T16:02:51.843961+0000","last_became_peered":"2023-09-12T16:02:51.843961+0000","last_unstale":"2023-09-12T16:02:51.857296+0000","last_undegraded":"2023-09-12T16:02:51.857296+0000","last_fullsized":"2023-09-12T16:02:51.857296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:55:57.938037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832962+0000","last_change":"2023-09-12T16:02:05.032191+0000","last_active":"2023-09-12T16:02:50.832962+0000","last_peered":"2023-09-12T16:02:50.832962+0000","last_clean":"2023-09-12T16:02:50.832962+0000","last_became_active":"2023-09-12T16:02:05.031888+0000","last_became_peered":"2023-09-12T16:02:05.031888+0000","last_unstale":"2023-09-12T16:02:50.832962+0000","last_undegraded":"2023-09-12T16:02:50.832962+0000","last_fullsized":"2023-09-12T16:02:50.832962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:50:15.775078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834369+0000","last_change":"2023-09-12T16:02:09.290650+0000","last_active":"2023-09-12T16:02:50.834369+0000","last_peered":"2023-09-12T16:02:50.834369+0000","last_clean":"2023-09-12T16:02:50.834369+0000","last_became_active":"2023-09-12T16:02:09.290205+0000","last_became_peered":"2023-09-12T16:02:09.290205+0000","last_unstale":"2023-09-12T16:02:50.834369+0000","last_undegraded":"2023-09-12T16:02:50.834369+0000","last_fullsized":"2023-09-12T16:02:50.834369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:44:04.930063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833124+0000","last_change":"2023-09-12T16:02:04.032506+0000","last_active":"2023-09-12T16:02:50.833124+0000","last_peered":"2023-09-12T16:02:50.833124+0000","last_clean":"2023-09-12T16:02:50.833124+0000","last_became_active":"2023-09-12T16:02:04.032047+0000","last_became_peered":"2023-09-12T16:02:04.032047+0000","last_unstale":"2023-09-12T16:02:50.833124+0000","last_undegraded":"2023-09-12T16:02:50.833124+0000","last_fullsized":"2023-09-12T16:02:50.833124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:30:22.664465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.025692+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.025370+0000","last_became_peered":"2023-09-12T16:02:05.025370+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.305375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.068789+0000","last_change":"2023-09-12T16:02:52.068789+0000","last_active":"2023-09-12T16:02:52.068789+0000","last_peered":"2023-09-12T16:02:52.068789+0000","last_clean":"2023-09-12T16:02:52.068789+0000","last_became_active":"2023-09-12T16:02:52.068276+0000","last_became_peered":"2023-09-12T16:02:52.068276+0000","last_unstale":"2023-09-12T16:02:52.068789+0000","last_undegraded":"2023-09-12T16:02:52.068789+0000","last_fullsized":"2023-09-12T16:02:52.068789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:19:19.009173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831481+0000","last_change":"2023-09-12T16:02:04.032079+0000","last_active":"2023-09-12T16:02:50.831481+0000","last_peered":"2023-09-12T16:02:50.831481+0000","last_clean":"2023-09-12T16:02:50.831481+0000","last_became_active":"2023-09-12T16:02:04.031835+0000","last_became_peered":"2023-09-12T16:02:04.031835+0000","last_unstale":"2023-09-12T16:02:50.831481+0000","last_undegraded":"2023-09-12T16:02:50.831481+0000","last_fullsized":"2023-09-12T16:02:50.831481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:48:33.935012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837888+0000","last_change":"2023-09-12T16:02:05.037671+0000","last_active":"2023-09-12T16:02:50.837888+0000","last_peered":"2023-09-12T16:02:50.837888+0000","last_clean":"2023-09-12T16:02:50.837888+0000","last_became_active":"2023-09-12T16:02:05.037307+0000","last_became_peered":"2023-09-12T16:02:05.037307+0000","last_unstale":"2023-09-12T16:02:50.837888+0000","last_undegraded":"2023-09-12T16:02:50.837888+0000","last_fullsized":"2023-09-12T16:02:50.837888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:47:23.836612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834100+0000","last_change":"2023-09-12T16:02:09.289165+0000","last_active":"2023-09-12T16:02:50.834100+0000","last_peered":"2023-09-12T16:02:50.834100+0000","last_clean":"2023-09-12T16:02:50.834100+0000","last_became_active":"2023-09-12T16:02:09.288903+0000","last_became_peered":"2023-09-12T16:02:09.288903+0000","last_unstale":"2023-09-12T16:02:50.834100+0000","last_undegraded":"2023-09-12T16:02:50.834100+0000","last_fullsized":"2023-09-12T16:02:50.834100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:27:09.160822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831262+0000","last_change":"2023-09-12T16:02:04.027279+0000","last_active":"2023-09-12T16:02:50.831262+0000","last_peered":"2023-09-12T16:02:50.831262+0000","last_clean":"2023-09-12T16:02:50.831262+0000","last_became_active":"2023-09-12T16:02:04.026975+0000","last_became_peered":"2023-09-12T16:02:04.026975+0000","last_unstale":"2023-09-12T16:02:50.831262+0000","last_undegraded":"2023-09-12T16:02:50.831262+0000","last_fullsized":"2023-09-12T16:02:50.831262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:45.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832885+0000","last_change":"2023-09-12T16:02:05.027582+0000","last_active":"2023-09-12T16:02:50.832885+0000","last_peered":"2023-09-12T16:02:50.832885+0000","last_clean":"2023-09-12T16:02:50.832885+0000","last_became_active":"2023-09-12T16:02:05.027236+0000","last_became_peered":"2023-09-12T16:02:05.027236+0000","last_unstale":"2023-09-12T16:02:50.832885+0000","last_undegraded":"2023-09-12T16:02:50.832885+0000","last_fullsized":"2023-09-12T16:02:50.832885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:06:24.812731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836055+0000","last_change":"2023-09-12T16:02:09.284381+0000","last_active":"2023-09-12T16:02:50.836055+0000","last_peered":"2023-09-12T16:02:50.836055+0000","last_clean":"2023-09-12T16:02:50.836055+0000","last_became_active":"2023-09-12T16:02:09.283947+0000","last_became_peered":"2023-09-12T16:02:09.283947+0000","last_unstale":"2023-09-12T16:02:50.836055+0000","last_undegraded":"2023-09-12T16:02:50.836055+0000","last_fullsized":"2023-09-12T16:02:50.836055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:05:24.803781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:40.770486+0000","last_change":"2023-09-12T16:02:04.027693+0000","last_active":"2023-09-12T16:03:40.770486+0000","last_peered":"2023-09-12T16:03:40.770486+0000","last_clean":"2023-09-12T16:03:40.770486+0000","last_became_active":"2023-09-12T16:02:04.027315+0000","last_became_peered":"2023-09-12T16:02:04.027315+0000","last_unstale":"2023-09-12T16:03:40.770486+0000","last_undegraded":"2023-09-12T16:03:40.770486+0000","last_fullsized":"2023-09-12T16:03:40.770486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:32:21.756802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831054+0000","last_change":"2023-09-12T16:02:05.031673+0000","last_active":"2023-09-12T16:02:50.831054+0000","last_peered":"2023-09-12T16:02:50.831054+0000","last_clean":"2023-09-12T16:02:50.831054+0000","last_became_active":"2023-09-12T16:02:05.031364+0000","last_became_peered":"2023-09-12T16:02:05.031364+0000","last_unstale":"2023-09-12T16:02:50.831054+0000","last_undegraded":"2023-09-12T16:02:50.831054+0000","last_fullsized":"2023-09-12T16:02:50.831054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:48:05.222530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835641+0000","last_change":"2023-09-12T16:02:09.288536+0000","last_active":"2023-09-12T16:02:50.835641+0000","last_peered":"2023-09-12T16:02:50.835641+0000","last_clean":"2023-09-12T16:02:50.835641+0000","last_became_active":"2023-09-12T16:02:09.288062+0000","last_became_peered":"2023-09-12T16:02:09.288062+0000","last_unstale":"2023-09-12T16:02:50.835641+0000","last_undegraded":"2023-09-12T16:02:50.835641+0000","last_fullsized":"2023-09-12T16:02:50.835641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:39:28.935563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832024+0000","last_change":"2023-09-12T16:02:04.026298+0000","last_active":"2023-09-12T16:02:50.832024+0000","last_peered":"2023-09-12T16:02:50.832024+0000","last_clean":"2023-09-12T16:02:50.832024+0000","last_became_active":"2023-09-12T16:02:04.025975+0000","last_became_peered":"2023-09-12T16:02:04.025975+0000","last_unstale":"2023-09-12T16:02:50.832024+0000","last_undegraded":"2023-09-12T16:02:50.832024+0000","last_fullsized":"2023-09-12T16:02:50.832024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:12:04.338497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832502+0000","last_change":"2023-09-12T16:02:05.028337+0000","last_active":"2023-09-12T16:02:50.832502+0000","last_peered":"2023-09-12T16:02:50.832502+0000","last_clean":"2023-09-12T16:02:50.832502+0000","last_became_active":"2023-09-12T16:02:05.028036+0000","last_became_peered":"2023-09-12T16:02:05.028036+0000","last_unstale":"2023-09-12T16:02:50.832502+0000","last_undegraded":"2023-09-12T16:02:50.832502+0000","last_fullsized":"2023-09-12T16:02:50.832502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:39:09.387191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025809+0000","last_change":"2023-09-12T16:02:09.290950+0000","last_active":"2023-09-12T16:02:51.025809+0000","last_peered":"2023-09-12T16:02:51.025809+0000","last_clean":"2023-09-12T16:02:51.025809+0000","last_became_active":"2023-09-12T16:02:09.290477+0000","last_became_peered":"2023-09-12T16:02:09.290477+0000","last_unstale":"2023-09-12T16:02:51.025809+0000","last_undegraded":"2023-09-12T16:02:51.025809+0000","last_fullsized":"2023-09-12T16:02:51.025809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:53:45.907575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834845+0000","last_change":"2023-09-12T16:02:04.026025+0000","last_active":"2023-09-12T16:02:50.834845+0000","last_peered":"2023-09-12T16:02:50.834845+0000","last_clean":"2023-09-12T16:02:50.834845+0000","last_became_active":"2023-09-12T16:02:04.025722+0000","last_became_peered":"2023-09-12T16:02:04.025722+0000","last_unstale":"2023-09-12T16:02:50.834845+0000","last_undegraded":"2023-09-12T16:02:50.834845+0000","last_fullsized":"2023-09-12T16:02:50.834845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:29:58.357960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831920+0000","last_change":"2023-09-12T16:02:05.028461+0000","last_active":"2023-09-12T16:02:50.831920+0000","last_peered":"2023-09-12T16:02:50.831920+0000","last_clean":"2023-09-12T16:02:50.831920+0000","last_became_active":"2023-09-12T16:02:05.028176+0000","last_became_peered":"2023-09-12T16:02:05.028176+0000","last_unstale":"2023-09-12T16:02:50.831920+0000","last_undegraded":"2023-09-12T16:02:50.831920+0000","last_fullsized":"2023-09-12T16:02:50.831920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:16:48.652513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829138+0000","last_change":"2023-09-12T16:02:09.295245+0000","last_active":"2023-09-12T16:02:50.829138+0000","last_peered":"2023-09-12T16:02:50.829138+0000","last_clean":"2023-09-12T16:02:50.829138+0000","last_became_active":"2023-09-12T16:02:09.294872+0000","last_became_peered":"2023-09-12T16:02:09.294872+0000","last_unstale":"2023-09-12T16:02:50.829138+0000","last_undegraded":"2023-09-12T16:02:50.829138+0000","last_fullsized":"2023-09-12T16:02:50.829138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:43:49.462317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832140+0000","last_change":"2023-09-12T16:02:04.030815+0000","last_active":"2023-09-12T16:02:50.832140+0000","last_peered":"2023-09-12T16:02:50.832140+0000","last_clean":"2023-09-12T16:02:50.832140+0000","last_became_active":"2023-09-12T16:02:04.030435+0000","last_became_peered":"2023-09-12T16:02:04.030435+0000","last_unstale":"2023-09-12T16:02:50.832140+0000","last_undegraded":"2023-09-12T16:02:50.832140+0000","last_fullsized":"2023-09-12T16:02:50.832140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:39:59.661143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832565+0000","last_change":"2023-09-12T16:02:05.031155+0000","last_active":"2023-09-12T16:02:50.832565+0000","last_peered":"2023-09-12T16:02:50.832565+0000","last_clean":"2023-09-12T16:02:50.832565+0000","last_became_active":"2023-09-12T16:02:05.030803+0000","last_became_peered":"2023-09-12T16:02:05.030803+0000","last_unstale":"2023-09-12T16:02:50.832565+0000","last_undegraded":"2023-09-12T16:02:50.832565+0000","last_fullsized":"2023-09-12T16:02:50.832565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:55:21.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832473+0000","last_change":"2023-09-12T16:02:09.286445+0000","last_active":"2023-09-12T16:02:50.832473+0000","last_peered":"2023-09-12T16:02:50.832473+0000","last_clean":"2023-09-12T16:02:50.832473+0000","last_became_active":"2023-09-12T16:02:09.286049+0000","last_became_peered":"2023-09-12T16:02:09.286049+0000","last_unstale":"2023-09-12T16:02:50.832473+0000","last_undegraded":"2023-09-12T16:02:50.832473+0000","last_fullsized":"2023-09-12T16:02:50.832473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:43:13.943984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:44.775459+0000","last_change":"2023-09-12T16:02:04.035341+0000","last_active":"2023-09-12T16:03:44.775459+0000","last_peered":"2023-09-12T16:03:44.775459+0000","last_clean":"2023-09-12T16:03:44.775459+0000","last_became_active":"2023-09-12T16:02:04.035092+0000","last_became_peered":"2023-09-12T16:02:04.035092+0000","last_unstale":"2023-09-12T16:03:44.775459+0000","last_undegraded":"2023-09-12T16:03:44.775459+0000","last_fullsized":"2023-09-12T16:03:44.775459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:33:00.784401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838363+0000","last_change":"2023-09-12T16:02:05.025229+0000","last_active":"2023-09-12T16:02:50.838363+0000","last_peered":"2023-09-12T16:02:50.838363+0000","last_clean":"2023-09-12T16:02:50.838363+0000","last_became_active":"2023-09-12T16:02:05.024963+0000","last_became_peered":"2023-09-12T16:02:05.024963+0000","last_unstale":"2023-09-12T16:02:50.838363+0000","last_undegraded":"2023-09-12T16:02:50.838363+0000","last_fullsized":"2023-09-12T16:02:50.838363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:46:04.318082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832595+0000","last_change":"2023-09-12T16:02:09.295011+0000","last_active":"2023-09-12T16:02:50.832595+0000","last_peered":"2023-09-12T16:02:50.832595+0000","last_clean":"2023-09-12T16:02:50.832595+0000","last_became_active":"2023-09-12T16:02:09.294617+0000","last_became_peered":"2023-09-12T16:02:09.294617+0000","last_unstale":"2023-09-12T16:02:50.832595+0000","last_undegraded":"2023-09-12T16:02:50.832595+0000","last_fullsized":"2023-09-12T16:02:50.832595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:57:53.902946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830613+0000","last_change":"2023-09-12T16:02:04.028086+0000","last_active":"2023-09-12T16:02:50.830613+0000","last_peered":"2023-09-12T16:02:50.830613+0000","last_clean":"2023-09-12T16:02:50.830613+0000","last_became_active":"2023-09-12T16:02:04.027798+0000","last_became_peered":"2023-09-12T16:02:04.027798+0000","last_unstale":"2023-09-12T16:02:50.830613+0000","last_undegraded":"2023-09-12T16:02:50.830613+0000","last_fullsized":"2023-09-12T16:02:50.830613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:11:16.921824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829002+0000","last_change":"2023-09-12T16:02:05.030691+0000","last_active":"2023-09-12T16:02:50.829002+0000","last_peered":"2023-09-12T16:02:50.829002+0000","last_clean":"2023-09-12T16:02:50.829002+0000","last_became_active":"2023-09-12T16:02:05.030379+0000","last_became_peered":"2023-09-12T16:02:05.030379+0000","last_unstale":"2023-09-12T16:02:50.829002+0000","last_undegraded":"2023-09-12T16:02:50.829002+0000","last_fullsized":"2023-09-12T16:02:50.829002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:15:05.221078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835660+0000","last_change":"2023-09-12T16:02:09.290703+0000","last_active":"2023-09-12T16:02:50.835660+0000","last_peered":"2023-09-12T16:02:50.835660+0000","last_clean":"2023-09-12T16:02:50.835660+0000","last_became_active":"2023-09-12T16:02:09.290227+0000","last_became_peered":"2023-09-12T16:02:09.290227+0000","last_unstale":"2023-09-12T16:02:50.835660+0000","last_undegraded":"2023-09-12T16:02:50.835660+0000","last_fullsized":"2023-09-12T16:02:50.835660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:07:06.182295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832824+0000","last_change":"2023-09-12T16:02:04.028630+0000","last_active":"2023-09-12T16:02:50.832824+0000","last_peered":"2023-09-12T16:02:50.832824+0000","last_clean":"2023-09-12T16:02:50.832824+0000","last_became_active":"2023-09-12T16:02:04.028319+0000","last_became_peered":"2023-09-12T16:02:04.028319+0000","last_unstale":"2023-09-12T16:02:50.832824+0000","last_undegraded":"2023-09-12T16:02:50.832824+0000","last_fullsized":"2023-09-12T16:02:50.832824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:13:43.349425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838071+0000","last_change":"2023-09-12T16:02:05.032392+0000","last_active":"2023-09-12T16:02:50.838071+0000","last_peered":"2023-09-12T16:02:50.838071+0000","last_clean":"2023-09-12T16:02:50.838071+0000","last_became_active":"2023-09-12T16:02:05.032099+0000","last_became_peered":"2023-09-12T16:02:05.032099+0000","last_unstale":"2023-09-12T16:02:50.838071+0000","last_undegraded":"2023-09-12T16:02:50.838071+0000","last_fullsized":"2023-09-12T16:02:50.838071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:46:03.108459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832745+0000","last_change":"2023-09-12T16:02:09.289573+0000","last_active":"2023-09-12T16:02:50.832745+0000","last_peered":"2023-09-12T16:02:50.832745+0000","last_clean":"2023-09-12T16:02:50.832745+0000","last_became_active":"2023-09-12T16:02:09.289179+0000","last_became_peered":"2023-09-12T16:02:09.289179+0000","last_unstale":"2023-09-12T16:02:50.832745+0000","last_undegraded":"2023-09-12T16:02:50.832745+0000","last_fullsized":"2023-09-12T16:02:50.832745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:52:35.905346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:44.931972+0000","last_change":"2023-09-12T16:02:51.851180+0000","last_active":"2023-09-12T16:03:44.931972+0000","last_peered":"2023-09-12T16:03:44.931972+0000","last_clean":"2023-09-12T16:03:44.931972+0000","last_became_active":"2023-09-12T16:02:51.841452+0000","last_became_peered":"2023-09-12T16:02:51.841452+0000","last_unstale":"2023-09-12T16:03:44.931972+0000","last_undegraded":"2023-09-12T16:03:44.931972+0000","last_fullsized":"2023-09-12T16:03:44.931972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:20:20.903637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65523,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":138,"num_write":35,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":59233,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831192+0000","last_change":"2023-09-12T16:02:05.031409+0000","last_active":"2023-09-12T16:02:50.831192+0000","last_peered":"2023-09-12T16:02:50.831192+0000","last_clean":"2023-09-12T16:02:50.831192+0000","last_became_active":"2023-09-12T16:02:05.031110+0000","last_became_peered":"2023-09-12T16:02:05.031110+0000","last_unstale":"2023-09-12T16:02:50.831192+0000","last_undegraded":"2023-09-12T16:02:50.831192+0000","last_fullsized":"2023-09-12T16:02:50.831192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:10:54.482078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833098+0000","last_change":"2023-09-12T16:02:09.291493+0000","last_active":"2023-09-12T16:02:50.833098+0000","last_peered":"2023-09-12T16:02:50.833098+0000","last_clean":"2023-09-12T16:02:50.833098+0000","last_became_active":"2023-09-12T16:02:09.291044+0000","last_became_peered":"2023-09-12T16:02:09.291044+0000","last_unstale":"2023-09-12T16:02:50.833098+0000","last_undegraded":"2023-09-12T16:02:50.833098+0000","last_fullsized":"2023-09-12T16:02:50.833098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:34:30.269720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835682+0000","last_change":"2023-09-12T16:02:04.029283+0000","last_active":"2023-09-12T16:02:50.835682+0000","last_peered":"2023-09-12T16:02:50.835682+0000","last_clean":"2023-09-12T16:02:50.835682+0000","last_became_active":"2023-09-12T16:02:04.028959+0000","last_became_peered":"2023-09-12T16:02:04.028959+0000","last_unstale":"2023-09-12T16:02:50.835682+0000","last_undegraded":"2023-09-12T16:02:50.835682+0000","last_fullsized":"2023-09-12T16:02:50.835682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:28:52.750278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834981+0000","last_change":"2023-09-12T16:02:05.034140+0000","last_active":"2023-09-12T16:02:50.834981+0000","last_peered":"2023-09-12T16:02:50.834981+0000","last_clean":"2023-09-12T16:02:50.834981+0000","last_became_active":"2023-09-12T16:02:05.033840+0000","last_became_peered":"2023-09-12T16:02:05.033840+0000","last_unstale":"2023-09-12T16:02:50.834981+0000","last_undegraded":"2023-09-12T16:02:50.834981+0000","last_fullsized":"2023-09-12T16:02:50.834981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:59:55.522608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025577+0000","last_change":"2023-09-12T16:02:09.289998+0000","last_active":"2023-09-12T16:02:51.025577+0000","last_peered":"2023-09-12T16:02:51.025577+0000","last_clean":"2023-09-12T16:02:51.025577+0000","last_became_active":"2023-09-12T16:02:09.289561+0000","last_became_peered":"2023-09-12T16:02:09.289561+0000","last_unstale":"2023-09-12T16:02:51.025577+0000","last_undegraded":"2023-09-12T16:02:51.025577+0000","last_fullsized":"2023-09-12T16:02:51.025577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:09:47.327213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832456+0000","last_change":"2023-09-12T16:02:04.029883+0000","last_active":"2023-09-12T16:02:50.832456+0000","last_peered":"2023-09-12T16:02:50.832456+0000","last_clean":"2023-09-12T16:02:50.832456+0000","last_became_active":"2023-09-12T16:02:04.029547+0000","last_became_peered":"2023-09-12T16:02:04.029547+0000","last_unstale":"2023-09-12T16:02:50.832456+0000","last_undegraded":"2023-09-12T16:02:50.832456+0000","last_fullsized":"2023-09-12T16:02:50.832456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:47:46.478325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846042+0000","last_change":"2023-09-12T16:02:51.846042+0000","last_active":"2023-09-12T16:02:51.846042+0000","last_peered":"2023-09-12T16:02:51.846042+0000","last_clean":"2023-09-12T16:02:51.846042+0000","last_became_active":"2023-09-12T16:02:51.845572+0000","last_became_peered":"2023-09-12T16:02:51.845572+0000","last_unstale":"2023-09-12T16:02:51.846042+0000","last_undegraded":"2023-09-12T16:02:51.846042+0000","last_fullsized":"2023-09-12T16:02:51.846042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:55.335485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069758+0000","last_change":"2023-09-12T16:02:52.069157+0000","last_active":"2023-09-12T16:02:52.069758+0000","last_peered":"2023-09-12T16:02:52.069758+0000","last_clean":"2023-09-12T16:02:52.069758+0000","last_became_active":"2023-09-12T16:02:52.068600+0000","last_became_peered":"2023-09-12T16:02:52.068600+0000","last_unstale":"2023-09-12T16:02:52.069758+0000","last_undegraded":"2023-09-12T16:02:52.069758+0000","last_fullsized":"2023-09-12T16:02:52.069758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:28:44.160612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:40.769628+0000","last_change":"2023-09-12T16:02:04.025723+0000","last_active":"2023-09-12T16:03:40.769628+0000","last_peered":"2023-09-12T16:03:40.769628+0000","last_clean":"2023-09-12T16:03:40.769628+0000","last_became_active":"2023-09-12T16:02:04.025432+0000","last_became_peered":"2023-09-12T16:02:04.025432+0000","last_unstale":"2023-09-12T16:03:40.769628+0000","last_undegraded":"2023-09-12T16:03:40.769628+0000","last_fullsized":"2023-09-12T16:03:40.769628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:56.279910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.839291+0000","last_change":"2023-09-12T16:02:51.839291+0000","last_active":"2023-09-12T16:02:51.839291+0000","last_peered":"2023-09-12T16:02:51.839291+0000","last_clean":"2023-09-12T16:02:51.839291+0000","last_became_active":"2023-09-12T16:02:51.838968+0000","last_became_peered":"2023-09-12T16:02:51.838968+0000","last_unstale":"2023-09-12T16:02:51.839291+0000","last_undegraded":"2023-09-12T16:02:51.839291+0000","last_fullsized":"2023-09-12T16:02:51.839291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:21:12.542019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835124+0000","last_change":"2023-09-12T16:02:09.286447+0000","last_active":"2023-09-12T16:02:50.835124+0000","last_peered":"2023-09-12T16:02:50.835124+0000","last_clean":"2023-09-12T16:02:50.835124+0000","last_became_active":"2023-09-12T16:02:09.285805+0000","last_became_peered":"2023-09-12T16:02:09.285805+0000","last_unstale":"2023-09-12T16:02:50.835124+0000","last_undegraded":"2023-09-12T16:02:50.835124+0000","last_fullsized":"2023-09-12T16:02:50.835124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:35:05.518872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830710+0000","last_change":"2023-09-12T16:02:04.032527+0000","last_active":"2023-09-12T16:02:50.830710+0000","last_peered":"2023-09-12T16:02:50.830710+0000","last_clean":"2023-09-12T16:02:50.830710+0000","last_became_active":"2023-09-12T16:02:04.032264+0000","last_became_peered":"2023-09-12T16:02:04.032264+0000","last_unstale":"2023-09-12T16:02:50.830710+0000","last_undegraded":"2023-09-12T16:02:50.830710+0000","last_fullsized":"2023-09-12T16:02:50.830710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:53:52.433549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833597+0000","last_change":"2023-09-12T16:02:05.029281+0000","last_active":"2023-09-12T16:02:50.833597+0000","last_peered":"2023-09-12T16:02:50.833597+0000","last_clean":"2023-09-12T16:02:50.833597+0000","last_became_active":"2023-09-12T16:02:05.028919+0000","last_became_peered":"2023-09-12T16:02:05.028919+0000","last_unstale":"2023-09-12T16:02:50.833597+0000","last_undegraded":"2023-09-12T16:02:50.833597+0000","last_fullsized":"2023-09-12T16:02:50.833597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:26.667060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069744+0000","last_change":"2023-09-12T16:02:52.069336+0000","last_active":"2023-09-12T16:02:52.069744+0000","last_peered":"2023-09-12T16:02:52.069744+0000","last_clean":"2023-09-12T16:02:52.069744+0000","last_became_active":"2023-09-12T16:02:52.068805+0000","last_became_peered":"2023-09-12T16:02:52.068805+0000","last_unstale":"2023-09-12T16:02:52.069744+0000","last_undegraded":"2023-09-12T16:02:52.069744+0000","last_fullsized":"2023-09-12T16:02:52.069744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:09.095952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831444+0000","last_change":"2023-09-12T16:02:04.026373+0000","last_active":"2023-09-12T16:02:50.831444+0000","last_peered":"2023-09-12T16:02:50.831444+0000","last_clean":"2023-09-12T16:02:50.831444+0000","last_became_active":"2023-09-12T16:02:04.025907+0000","last_became_peered":"2023-09-12T16:02:04.025907+0000","last_unstale":"2023-09-12T16:02:50.831444+0000","last_undegraded":"2023-09-12T16:02:50.831444+0000","last_fullsized":"2023-09-12T16:02:50.831444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:33:15.152128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025705+0000","last_change":"2023-09-12T16:02:05.040143+0000","last_active":"2023-09-12T16:02:51.025705+0000","last_peered":"2023-09-12T16:02:51.025705+0000","last_clean":"2023-09-12T16:02:51.025705+0000","last_became_active":"2023-09-12T16:02:05.039749+0000","last_became_peered":"2023-09-12T16:02:05.039749+0000","last_unstale":"2023-09-12T16:02:51.025705+0000","last_undegraded":"2023-09-12T16:02:51.025705+0000","last_fullsized":"2023-09-12T16:02:51.025705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:49.561252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:09.841670+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:09.841203+0000","last_became_peered":"2023-09-12T16:02:09.841203+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:55:46.128996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833178+0000","last_change":"2023-09-12T16:02:04.027953+0000","last_active":"2023-09-12T16:02:50.833178+0000","last_peered":"2023-09-12T16:02:50.833178+0000","last_clean":"2023-09-12T16:02:50.833178+0000","last_became_active":"2023-09-12T16:02:04.027624+0000","last_became_peered":"2023-09-12T16:02:04.027624+0000","last_unstale":"2023-09-12T16:02:50.833178+0000","last_undegraded":"2023-09-12T16:02:50.833178+0000","last_fullsized":"2023-09-12T16:02:50.833178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:33:58.356358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832388+0000","last_change":"2023-09-12T16:02:05.036593+0000","last_active":"2023-09-12T16:02:50.832388+0000","last_peered":"2023-09-12T16:02:50.832388+0000","last_clean":"2023-09-12T16:02:50.832388+0000","last_became_active":"2023-09-12T16:02:05.036204+0000","last_became_peered":"2023-09-12T16:02:05.036204+0000","last_unstale":"2023-09-12T16:02:50.832388+0000","last_undegraded":"2023-09-12T16:02:50.832388+0000","last_fullsized":"2023-09-12T16:02:50.832388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:12:31.068837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835475+0000","last_change":"2023-09-12T16:02:09.292405+0000","last_active":"2023-09-12T16:02:50.835475+0000","last_peered":"2023-09-12T16:02:50.835475+0000","last_clean":"2023-09-12T16:02:50.835475+0000","last_became_active":"2023-09-12T16:02:09.291877+0000","last_became_peered":"2023-09-12T16:02:09.291877+0000","last_unstale":"2023-09-12T16:02:50.835475+0000","last_undegraded":"2023-09-12T16:02:50.835475+0000","last_fullsized":"2023-09-12T16:02:50.835475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:53:04.847310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829465+0000","last_change":"2023-09-12T16:02:04.034738+0000","last_active":"2023-09-12T16:02:50.829465+0000","last_peered":"2023-09-12T16:02:50.829465+0000","last_clean":"2023-09-12T16:02:50.829465+0000","last_became_active":"2023-09-12T16:02:04.034408+0000","last_became_peered":"2023-09-12T16:02:04.034408+0000","last_unstale":"2023-09-12T16:02:50.829465+0000","last_undegraded":"2023-09-12T16:02:50.829465+0000","last_fullsized":"2023-09-12T16:02:50.829465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:27.845770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832464+0000","last_change":"2023-09-12T16:02:05.044936+0000","last_active":"2023-09-12T16:02:50.832464+0000","last_peered":"2023-09-12T16:02:50.832464+0000","last_clean":"2023-09-12T16:02:50.832464+0000","last_became_active":"2023-09-12T16:02:05.044617+0000","last_became_peered":"2023-09-12T16:02:05.044617+0000","last_unstale":"2023-09-12T16:02:50.832464+0000","last_undegraded":"2023-09-12T16:02:50.832464+0000","last_fullsized":"2023-09-12T16:02:50.832464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:41:32.639331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834992+0000","last_change":"2023-09-12T16:02:09.841919+0000","last_active":"2023-09-12T16:02:50.834992+0000","last_peered":"2023-09-12T16:02:50.834992+0000","last_clean":"2023-09-12T16:02:50.834992+0000","last_became_active":"2023-09-12T16:02:09.841410+0000","last_became_peered":"2023-09-12T16:02:09.841410+0000","last_unstale":"2023-09-12T16:02:50.834992+0000","last_undegraded":"2023-09-12T16:02:50.834992+0000","last_fullsized":"2023-09-12T16:02:50.834992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:29:57.511693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.840116+0000","last_change":"2023-09-12T16:02:51.840116+0000","last_active":"2023-09-12T16:02:51.840116+0000","last_peered":"2023-09-12T16:02:51.840116+0000","last_clean":"2023-09-12T16:02:51.840116+0000","last_became_active":"2023-09-12T16:02:51.839698+0000","last_became_peered":"2023-09-12T16:02:51.839698+0000","last_unstale":"2023-09-12T16:02:51.840116+0000","last_undegraded":"2023-09-12T16:02:51.840116+0000","last_fullsized":"2023-09-12T16:02:51.840116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:29.475855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:43.368647+0000","last_change":"2023-09-12T16:02:04.032892+0000","last_active":"2023-09-12T16:03:43.368647+0000","last_peered":"2023-09-12T16:03:43.368647+0000","last_clean":"2023-09-12T16:03:43.368647+0000","last_became_active":"2023-09-12T16:02:04.032637+0000","last_became_peered":"2023-09-12T16:02:04.032637+0000","last_unstale":"2023-09-12T16:03:43.368647+0000","last_undegraded":"2023-09-12T16:03:43.368647+0000","last_fullsized":"2023-09-12T16:03:43.368647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:22:55.805000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830231+0000","last_change":"2023-09-12T16:02:09.285127+0000","last_active":"2023-09-12T16:02:50.830231+0000","last_peered":"2023-09-12T16:02:50.830231+0000","last_clean":"2023-09-12T16:02:50.830231+0000","last_became_active":"2023-09-12T16:02:09.284790+0000","last_became_peered":"2023-09-12T16:02:09.284790+0000","last_unstale":"2023-09-12T16:02:50.830231+0000","last_undegraded":"2023-09-12T16:02:50.830231+0000","last_fullsized":"2023-09-12T16:02:50.830231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:50:50.578069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832025+0000","last_change":"2023-09-12T16:02:05.027877+0000","last_active":"2023-09-12T16:02:50.832025+0000","last_peered":"2023-09-12T16:02:50.832025+0000","last_clean":"2023-09-12T16:02:50.832025+0000","last_became_active":"2023-09-12T16:02:05.027582+0000","last_became_peered":"2023-09-12T16:02:05.027582+0000","last_unstale":"2023-09-12T16:02:50.832025+0000","last_undegraded":"2023-09-12T16:02:50.832025+0000","last_fullsized":"2023-09-12T16:02:50.832025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:47.835790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835647+0000","last_change":"2023-09-12T16:02:04.024800+0000","last_active":"2023-09-12T16:02:50.835647+0000","last_peered":"2023-09-12T16:02:50.835647+0000","last_clean":"2023-09-12T16:02:50.835647+0000","last_became_active":"2023-09-12T16:02:04.024436+0000","last_became_peered":"2023-09-12T16:02:04.024436+0000","last_unstale":"2023-09-12T16:02:50.835647+0000","last_undegraded":"2023-09-12T16:02:50.835647+0000","last_fullsized":"2023-09-12T16:02:50.835647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:57:35.093322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829225+0000","last_change":"2023-09-12T16:01:04.599706+0000","last_active":"2023-09-12T16:02:50.829225+0000","last_peered":"2023-09-12T16:02:50.829225+0000","last_clean":"2023-09-12T16:02:50.829225+0000","last_became_active":"2023-09-12T16:01:04.599328+0000","last_became_peered":"2023-09-12T16:01:04.599328+0000","last_unstale":"2023-09-12T16:02:50.829225+0000","last_undegraded":"2023-09-12T16:02:50.829225+0000","last_fullsized":"2023-09-12T16:02:50.829225+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_clean_scrub_stamp":"2023-09-12T15:58:15.250648+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:57:56.846488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833462+0000","last_change":"2023-09-12T16:02:09.841960+0000","last_active":"2023-09-12T16:02:50.833462+0000","last_peered":"2023-09-12T16:02:50.833462+0000","last_clean":"2023-09-12T16:02:50.833462+0000","last_became_active":"2023-09-12T16:02:09.841279+0000","last_became_peered":"2023-09-12T16:02:09.841279+0000","last_unstale":"2023-09-12T16:02:50.833462+0000","last_undegraded":"2023-09-12T16:02:50.833462+0000","last_fullsized":"2023-09-12T16:02:50.833462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:17:19.419350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.847600+0000","last_change":"2023-09-12T16:02:51.847600+0000","last_active":"2023-09-12T16:02:51.847600+0000","last_peered":"2023-09-12T16:02:51.847600+0000","last_clean":"2023-09-12T16:02:51.847600+0000","last_became_active":"2023-09-12T16:02:51.847292+0000","last_became_peered":"2023-09-12T16:02:51.847292+0000","last_unstale":"2023-09-12T16:02:51.847600+0000","last_undegraded":"2023-09-12T16:02:51.847600+0000","last_fullsized":"2023-09-12T16:02:51.847600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:25:58.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829344+0000","last_change":"2023-09-12T16:02:04.029790+0000","last_active":"2023-09-12T16:02:50.829344+0000","last_peered":"2023-09-12T16:02:50.829344+0000","last_clean":"2023-09-12T16:02:50.829344+0000","last_became_active":"2023-09-12T16:02:04.029440+0000","last_became_peered":"2023-09-12T16:02:04.029440+0000","last_unstale":"2023-09-12T16:02:50.829344+0000","last_undegraded":"2023-09-12T16:02:50.829344+0000","last_fullsized":"2023-09-12T16:02:50.829344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:50:33.690628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025552+0000","last_change":"2023-09-12T16:02:09.289757+0000","last_active":"2023-09-12T16:02:51.025552+0000","last_peered":"2023-09-12T16:02:51.025552+0000","last_clean":"2023-09-12T16:02:51.025552+0000","last_became_active":"2023-09-12T16:02:09.289348+0000","last_became_peered":"2023-09-12T16:02:09.289348+0000","last_unstale":"2023-09-12T16:02:51.025552+0000","last_undegraded":"2023-09-12T16:02:51.025552+0000","last_fullsized":"2023-09-12T16:02:51.025552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:15:14.695521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831296+0000","last_change":"2023-09-12T16:02:04.033213+0000","last_active":"2023-09-12T16:02:50.831296+0000","last_peered":"2023-09-12T16:02:50.831296+0000","last_clean":"2023-09-12T16:02:50.831296+0000","last_became_active":"2023-09-12T16:02:04.032978+0000","last_became_peered":"2023-09-12T16:02:04.032978+0000","last_unstale":"2023-09-12T16:02:50.831296+0000","last_undegraded":"2023-09-12T16:02:50.831296+0000","last_fullsized":"2023-09-12T16:02:50.831296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:25:33.718954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835250+0000","last_change":"2023-09-12T16:02:05.043252+0000","last_active":"2023-09-12T16:02:50.835250+0000","last_peered":"2023-09-12T16:02:50.835250+0000","last_clean":"2023-09-12T16:02:50.835250+0000","last_became_active":"2023-09-12T16:02:05.042975+0000","last_became_peered":"2023-09-12T16:02:05.042975+0000","last_unstale":"2023-09-12T16:02:50.835250+0000","last_undegraded":"2023-09-12T16:02:50.835250+0000","last_fullsized":"2023-09-12T16:02:50.835250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:46:31.549547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831861+0000","last_change":"2023-09-12T16:02:09.841728+0000","last_active":"2023-09-12T16:02:50.831861+0000","last_peered":"2023-09-12T16:02:50.831861+0000","last_clean":"2023-09-12T16:02:50.831861+0000","last_became_active":"2023-09-12T16:02:09.841351+0000","last_became_peered":"2023-09-12T16:02:09.841351+0000","last_unstale":"2023-09-12T16:02:50.831861+0000","last_undegraded":"2023-09-12T16:02:50.831861+0000","last_fullsized":"2023-09-12T16:02:50.831861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:00:50.637272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833531+0000","last_change":"2023-09-12T16:02:04.025624+0000","last_active":"2023-09-12T16:02:50.833531+0000","last_peered":"2023-09-12T16:02:50.833531+0000","last_clean":"2023-09-12T16:02:50.833531+0000","last_became_active":"2023-09-12T16:02:04.025330+0000","last_became_peered":"2023-09-12T16:02:04.025330+0000","last_unstale":"2023-09-12T16:02:50.833531+0000","last_undegraded":"2023-09-12T16:02:50.833531+0000","last_fullsized":"2023-09-12T16:02:50.833531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:00:57.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833174+0000","last_change":"2023-09-12T16:02:05.028414+0000","last_active":"2023-09-12T16:02:50.833174+0000","last_peered":"2023-09-12T16:02:50.833174+0000","last_clean":"2023-09-12T16:02:50.833174+0000","last_became_active":"2023-09-12T16:02:05.028081+0000","last_became_peered":"2023-09-12T16:02:05.028081+0000","last_unstale":"2023-09-12T16:02:50.833174+0000","last_undegraded":"2023-09-12T16:02:50.833174+0000","last_fullsized":"2023-09-12T16:02:50.833174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:57:58.245105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834801+0000","last_change":"2023-09-12T16:02:09.283222+0000","last_active":"2023-09-12T16:02:50.834801+0000","last_peered":"2023-09-12T16:02:50.834801+0000","last_clean":"2023-09-12T16:02:50.834801+0000","last_became_active":"2023-09-12T16:02:09.282814+0000","last_became_peered":"2023-09-12T16:02:09.282814+0000","last_unstale":"2023-09-12T16:02:50.834801+0000","last_undegraded":"2023-09-12T16:02:50.834801+0000","last_fullsized":"2023-09-12T16:02:50.834801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:18.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836561+0000","last_change":"2023-09-12T16:02:04.026694+0000","last_active":"2023-09-12T16:02:50.836561+0000","last_peered":"2023-09-12T16:02:50.836561+0000","last_clean":"2023-09-12T16:02:50.836561+0000","last_became_active":"2023-09-12T16:02:04.026368+0000","last_became_peered":"2023-09-12T16:02:04.026368+0000","last_unstale":"2023-09-12T16:02:50.836561+0000","last_undegraded":"2023-09-12T16:02:50.836561+0000","last_fullsized":"2023-09-12T16:02:50.836561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:14:10.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829216+0000","last_change":"2023-09-12T16:02:05.039037+0000","last_active":"2023-09-12T16:02:50.829216+0000","last_peered":"2023-09-12T16:02:50.829216+0000","last_clean":"2023-09-12T16:02:50.829216+0000","last_became_active":"2023-09-12T16:02:05.038708+0000","last_became_peered":"2023-09-12T16:02:05.038708+0000","last_unstale":"2023-09-12T16:02:50.829216+0000","last_undegraded":"2023-09-12T16:02:50.829216+0000","last_fullsized":"2023-09-12T16:02:50.829216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:13:09.805646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836160+0000","last_change":"2023-09-12T16:02:09.841757+0000","last_active":"2023-09-12T16:02:50.836160+0000","last_peered":"2023-09-12T16:02:50.836160+0000","last_clean":"2023-09-12T16:02:50.836160+0000","last_became_active":"2023-09-12T16:02:09.841173+0000","last_became_peered":"2023-09-12T16:02:09.841173+0000","last_unstale":"2023-09-12T16:02:50.836160+0000","last_undegraded":"2023-09-12T16:02:50.836160+0000","last_fullsized":"2023-09-12T16:02:50.836160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:57:06.626828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836147+0000","last_change":"2023-09-12T16:02:04.031009+0000","last_active":"2023-09-12T16:02:50.836147+0000","last_peered":"2023-09-12T16:02:50.836147+0000","last_clean":"2023-09-12T16:02:50.836147+0000","last_became_active":"2023-09-12T16:02:04.030714+0000","last_became_peered":"2023-09-12T16:02:04.030714+0000","last_unstale":"2023-09-12T16:02:50.836147+0000","last_undegraded":"2023-09-12T16:02:50.836147+0000","last_fullsized":"2023-09-12T16:02:50.836147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:12.578686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830789+0000","last_change":"2023-09-12T16:02:05.030778+0000","last_active":"2023-09-12T16:02:50.830789+0000","last_peered":"2023-09-12T16:02:50.830789+0000","last_clean":"2023-09-12T16:02:50.830789+0000","last_became_active":"2023-09-12T16:02:05.030454+0000","last_became_peered":"2023-09-12T16:02:05.030454+0000","last_unstale":"2023-09-12T16:02:50.830789+0000","last_undegraded":"2023-09-12T16:02:50.830789+0000","last_fullsized":"2023-09-12T16:02:50.830789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:52.951246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834030+0000","last_change":"2023-09-12T16:02:09.289668+0000","last_active":"2023-09-12T16:02:50.834030+0000","last_peered":"2023-09-12T16:02:50.834030+0000","last_clean":"2023-09-12T16:02:50.834030+0000","last_became_active":"2023-09-12T16:02:09.289301+0000","last_became_peered":"2023-09-12T16:02:09.289301+0000","last_unstale":"2023-09-12T16:02:50.834030+0000","last_undegraded":"2023-09-12T16:02:50.834030+0000","last_fullsized":"2023-09-12T16:02:50.834030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:20:04.528146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832522+0000","last_change":"2023-09-12T16:02:04.036175+0000","last_active":"2023-09-12T16:02:50.832522+0000","last_peered":"2023-09-12T16:02:50.832522+0000","last_clean":"2023-09-12T16:02:50.832522+0000","last_became_active":"2023-09-12T16:02:04.035879+0000","last_became_peered":"2023-09-12T16:02:04.035879+0000","last_unstale":"2023-09-12T16:02:50.832522+0000","last_undegraded":"2023-09-12T16:02:50.832522+0000","last_fullsized":"2023-09-12T16:02:50.832522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:29:20.798716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025379+0000","last_change":"2023-09-12T16:02:05.034195+0000","last_active":"2023-09-12T16:02:51.025379+0000","last_peered":"2023-09-12T16:02:51.025379+0000","last_clean":"2023-09-12T16:02:51.025379+0000","last_became_active":"2023-09-12T16:02:05.033534+0000","last_became_peered":"2023-09-12T16:02:05.033534+0000","last_unstale":"2023-09-12T16:02:51.025379+0000","last_undegraded":"2023-09-12T16:02:51.025379+0000","last_fullsized":"2023-09-12T16:02:51.025379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:34:04.180218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834899+0000","last_change":"2023-09-12T16:02:09.287543+0000","last_active":"2023-09-12T16:02:50.834899+0000","last_peered":"2023-09-12T16:02:50.834899+0000","last_clean":"2023-09-12T16:02:50.834899+0000","last_became_active":"2023-09-12T16:02:09.287055+0000","last_became_peered":"2023-09-12T16:02:09.287055+0000","last_unstale":"2023-09-12T16:02:50.834899+0000","last_undegraded":"2023-09-12T16:02:50.834899+0000","last_fullsized":"2023-09-12T16:02:50.834899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:58.214381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834994+0000","last_change":"2023-09-12T16:02:04.030206+0000","last_active":"2023-09-12T16:02:50.834994+0000","last_peered":"2023-09-12T16:02:50.834994+0000","last_clean":"2023-09-12T16:02:50.834994+0000","last_became_active":"2023-09-12T16:02:04.029765+0000","last_became_peered":"2023-09-12T16:02:04.029765+0000","last_unstale":"2023-09-12T16:02:50.834994+0000","last_undegraded":"2023-09-12T16:02:50.834994+0000","last_fullsized":"2023-09-12T16:02:50.834994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:56:07.551681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832663+0000","last_change":"2023-09-12T16:02:05.028300+0000","last_active":"2023-09-12T16:02:50.832663+0000","last_peered":"2023-09-12T16:02:50.832663+0000","last_clean":"2023-09-12T16:02:50.832663+0000","last_became_active":"2023-09-12T16:02:05.027970+0000","last_became_peered":"2023-09-12T16:02:05.027970+0000","last_unstale":"2023-09-12T16:02:50.832663+0000","last_undegraded":"2023-09-12T16:02:50.832663+0000","last_fullsized":"2023-09-12T16:02:50.832663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:39:44.571593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835970+0000","last_change":"2023-09-12T16:02:09.841661+0000","last_active":"2023-09-12T16:02:50.835970+0000","last_peered":"2023-09-12T16:02:50.835970+0000","last_clean":"2023-09-12T16:02:50.835970+0000","last_became_active":"2023-09-12T16:02:09.841039+0000","last_became_peered":"2023-09-12T16:02:09.841039+0000","last_unstale":"2023-09-12T16:02:50.835970+0000","last_undegraded":"2023-09-12T16:02:50.835970+0000","last_fullsized":"2023-09-12T16:02:50.835970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:34.529098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830867+0000","last_change":"2023-09-12T16:02:04.031927+0000","last_active":"2023-09-12T16:02:50.830867+0000","last_peered":"2023-09-12T16:02:50.830867+0000","last_clean":"2023-09-12T16:02:50.830867+0000","last_became_active":"2023-09-12T16:02:04.031614+0000","last_became_peered":"2023-09-12T16:02:04.031614+0000","last_unstale":"2023-09-12T16:02:50.830867+0000","last_undegraded":"2023-09-12T16:02:50.830867+0000","last_fullsized":"2023-09-12T16:02:50.830867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:00:23.423933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829483+0000","last_change":"2023-09-12T16:02:05.032647+0000","last_active":"2023-09-12T16:02:50.829483+0000","last_peered":"2023-09-12T16:02:50.829483+0000","last_clean":"2023-09-12T16:02:50.829483+0000","last_became_active":"2023-09-12T16:02:05.031701+0000","last_became_peered":"2023-09-12T16:02:05.031701+0000","last_unstale":"2023-09-12T16:02:50.829483+0000","last_undegraded":"2023-09-12T16:02:50.829483+0000","last_fullsized":"2023-09-12T16:02:50.829483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:02:19.596913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837136+0000","last_change":"2023-09-12T16:02:09.291043+0000","last_active":"2023-09-12T16:02:50.837136+0000","last_peered":"2023-09-12T16:02:50.837136+0000","last_clean":"2023-09-12T16:02:50.837136+0000","last_became_active":"2023-09-12T16:02:09.290666+0000","last_became_peered":"2023-09-12T16:02:09.290666+0000","last_unstale":"2023-09-12T16:02:50.837136+0000","last_undegraded":"2023-09-12T16:02:50.837136+0000","last_fullsized":"2023-09-12T16:02:50.837136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:10:47.354479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835888+0000","last_change":"2023-09-12T16:02:04.030501+0000","last_active":"2023-09-12T16:02:50.835888+0000","last_peered":"2023-09-12T16:02:50.835888+0000","last_clean":"2023-09-12T16:02:50.835888+0000","last_became_active":"2023-09-12T16:02:04.030196+0000","last_became_peered":"2023-09-12T16:02:04.030196+0000","last_unstale":"2023-09-12T16:02:50.835888+0000","last_undegraded":"2023-09-12T16:02:50.835888+0000","last_fullsized":"2023-09-12T16:02:50.835888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:58:34.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830617+0000","last_change":"2023-09-12T16:02:05.030466+0000","last_active":"2023-09-12T16:02:50.830617+0000","last_peered":"2023-09-12T16:02:50.830617+0000","last_clean":"2023-09-12T16:02:50.830617+0000","last_became_active":"2023-09-12T16:02:05.029988+0000","last_became_peered":"2023-09-12T16:02:05.029988+0000","last_unstale":"2023-09-12T16:02:50.830617+0000","last_undegraded":"2023-09-12T16:02:50.830617+0000","last_fullsized":"2023-09-12T16:02:50.830617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:08:56.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846403+0000","last_change":"2023-09-12T16:02:51.846403+0000","last_active":"2023-09-12T16:02:51.846403+0000","last_peered":"2023-09-12T16:02:51.846403+0000","last_clean":"2023-09-12T16:02:51.846403+0000","last_became_active":"2023-09-12T16:02:51.846047+0000","last_became_peered":"2023-09-12T16:02:51.846047+0000","last_unstale":"2023-09-12T16:02:51.846403+0000","last_undegraded":"2023-09-12T16:02:51.846403+0000","last_fullsized":"2023-09-12T16:02:51.846403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:58:11.458339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832945+0000","last_change":"2023-09-12T16:02:04.027380+0000","last_active":"2023-09-12T16:02:50.832945+0000","last_peered":"2023-09-12T16:02:50.832945+0000","last_clean":"2023-09-12T16:02:50.832945+0000","last_became_active":"2023-09-12T16:02:04.027098+0000","last_became_peered":"2023-09-12T16:02:04.027098+0000","last_unstale":"2023-09-12T16:02:50.832945+0000","last_undegraded":"2023-09-12T16:02:50.832945+0000","last_fullsized":"2023-09-12T16:02:50.832945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:34:12.260747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829993+0000","last_change":"2023-09-12T16:02:05.029249+0000","last_active":"2023-09-12T16:02:50.829993+0000","last_peered":"2023-09-12T16:02:50.829993+0000","last_clean":"2023-09-12T16:02:50.829993+0000","last_became_active":"2023-09-12T16:02:05.028955+0000","last_became_peered":"2023-09-12T16:02:05.028955+0000","last_unstale":"2023-09-12T16:02:50.829993+0000","last_undegraded":"2023-09-12T16:02:50.829993+0000","last_fullsized":"2023-09-12T16:02:50.829993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:02:10.382712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831923+0000","last_change":"2023-09-12T16:02:09.288724+0000","last_active":"2023-09-12T16:02:50.831923+0000","last_peered":"2023-09-12T16:02:50.831923+0000","last_clean":"2023-09-12T16:02:50.831923+0000","last_became_active":"2023-09-12T16:02:09.288206+0000","last_became_peered":"2023-09-12T16:02:09.288206+0000","last_unstale":"2023-09-12T16:02:50.831923+0000","last_undegraded":"2023-09-12T16:02:50.831923+0000","last_fullsized":"2023-09-12T16:02:50.831923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:45:10.154195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835225+0000","last_change":"2023-09-12T16:02:04.031219+0000","last_active":"2023-09-12T16:02:50.835225+0000","last_peered":"2023-09-12T16:02:50.835225+0000","last_clean":"2023-09-12T16:02:50.835225+0000","last_became_active":"2023-09-12T16:02:04.030867+0000","last_became_peered":"2023-09-12T16:02:04.030867+0000","last_unstale":"2023-09-12T16:02:50.835225+0000","last_undegraded":"2023-09-12T16:02:50.835225+0000","last_fullsized":"2023-09-12T16:02:50.835225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:16.885094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831081+0000","last_change":"2023-09-12T16:02:05.028004+0000","last_active":"2023-09-12T16:02:50.831081+0000","last_peered":"2023-09-12T16:02:50.831081+0000","last_clean":"2023-09-12T16:02:50.831081+0000","last_became_active":"2023-09-12T16:02:05.027668+0000","last_became_peered":"2023-09-12T16:02:05.027668+0000","last_unstale":"2023-09-12T16:02:50.831081+0000","last_undegraded":"2023-09-12T16:02:50.831081+0000","last_fullsized":"2023-09-12T16:02:50.831081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:41:23.019243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025479+0000","last_change":"2023-09-12T16:02:05.034380+0000","last_active":"2023-09-12T16:02:51.025479+0000","last_peered":"2023-09-12T16:02:51.025479+0000","last_clean":"2023-09-12T16:02:51.025479+0000","last_became_active":"2023-09-12T16:02:05.034022+0000","last_became_peered":"2023-09-12T16:02:05.034022+0000","last_unstale":"2023-09-12T16:02:51.025479+0000","last_undegraded":"2023-09-12T16:02:51.025479+0000","last_fullsized":"2023-09-12T16:02:51.025479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:55:56.404742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835155+0000","last_change":"2023-09-12T16:02:05.031723+0000","last_active":"2023-09-12T16:02:50.835155+0000","last_peered":"2023-09-12T16:02:50.835155+0000","last_clean":"2023-09-12T16:02:50.835155+0000","last_became_active":"2023-09-12T16:02:05.030954+0000","last_became_peered":"2023-09-12T16:02:05.030954+0000","last_unstale":"2023-09-12T16:02:50.835155+0000","last_undegraded":"2023-09-12T16:02:50.835155+0000","last_fullsized":"2023-09-12T16:02:50.835155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.319954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834938+0000","last_change":"2023-09-12T16:02:05.034851+0000","last_active":"2023-09-12T16:02:50.834938+0000","last_peered":"2023-09-12T16:02:50.834938+0000","last_clean":"2023-09-12T16:02:50.834938+0000","last_became_active":"2023-09-12T16:02:05.030733+0000","last_became_peered":"2023-09-12T16:02:05.030733+0000","last_unstale":"2023-09-12T16:02:50.834938+0000","last_undegraded":"2023-09-12T16:02:50.834938+0000","last_fullsized":"2023-09-12T16:02:50.834938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:33.605216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830111+0000","last_change":"2023-09-12T16:02:05.037331+0000","last_active":"2023-09-12T16:02:50.830111+0000","last_peered":"2023-09-12T16:02:50.830111+0000","last_clean":"2023-09-12T16:02:50.830111+0000","last_became_active":"2023-09-12T16:02:05.037015+0000","last_became_peered":"2023-09-12T16:02:05.037015+0000","last_unstale":"2023-09-12T16:02:50.830111+0000","last_undegraded":"2023-09-12T16:02:50.830111+0000","last_fullsized":"2023-09-12T16:02:50.830111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:29:14.035891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835319+0000","last_change":"2023-09-12T16:02:05.039803+0000","last_active":"2023-09-12T16:02:50.835319+0000","last_peered":"2023-09-12T16:02:50.835319+0000","last_clean":"2023-09-12T16:02:50.835319+0000","last_became_active":"2023-09-12T16:02:05.039390+0000","last_became_peered":"2023-09-12T16:02:05.039390+0000","last_unstale":"2023-09-12T16:02:50.835319+0000","last_undegraded":"2023-09-12T16:02:50.835319+0000","last_fullsized":"2023-09-12T16:02:50.835319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:22.422090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025700+0000","last_change":"2023-09-12T16:02:05.033153+0000","last_active":"2023-09-12T16:02:51.025700+0000","last_peered":"2023-09-12T16:02:51.025700+0000","last_clean":"2023-09-12T16:02:51.025700+0000","last_became_active":"2023-09-12T16:02:05.032800+0000","last_became_peered":"2023-09-12T16:02:05.032800+0000","last_unstale":"2023-09-12T16:02:51.025700+0000","last_undegraded":"2023-09-12T16:02:51.025700+0000","last_fullsized":"2023-09-12T16:02:51.025700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:31:43.090031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836428+0000","last_change":"2023-09-12T16:02:05.031519+0000","last_active":"2023-09-12T16:02:50.836428+0000","last_peered":"2023-09-12T16:02:50.836428+0000","last_clean":"2023-09-12T16:02:50.836428+0000","last_became_active":"2023-09-12T16:02:05.030227+0000","last_became_peered":"2023-09-12T16:02:05.030227+0000","last_unstale":"2023-09-12T16:02:50.836428+0000","last_undegraded":"2023-09-12T16:02:50.836428+0000","last_fullsized":"2023-09-12T16:02:50.836428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:00.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833413+0000","last_change":"2023-09-12T16:02:05.031763+0000","last_active":"2023-09-12T16:02:50.833413+0000","last_peered":"2023-09-12T16:02:50.833413+0000","last_clean":"2023-09-12T16:02:50.833413+0000","last_became_active":"2023-09-12T16:02:05.031441+0000","last_became_peered":"2023-09-12T16:02:05.031441+0000","last_unstale":"2023-09-12T16:02:50.833413+0000","last_undegraded":"2023-09-12T16:02:50.833413+0000","last_fullsized":"2023-09-12T16:02:50.833413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:55.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832249+0000","last_change":"2023-09-12T16:02:05.033786+0000","last_active":"2023-09-12T16:02:50.832249+0000","last_peered":"2023-09-12T16:02:50.832249+0000","last_clean":"2023-09-12T16:02:50.832249+0000","last_became_active":"2023-09-12T16:02:05.033497+0000","last_became_peered":"2023-09-12T16:02:05.033497+0000","last_unstale":"2023-09-12T16:02:50.832249+0000","last_undegraded":"2023-09-12T16:02:50.832249+0000","last_fullsized":"2023-09-12T16:02:50.832249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:01:58.912488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832560+0000","last_change":"2023-09-12T16:02:05.031144+0000","last_active":"2023-09-12T16:02:50.832560+0000","last_peered":"2023-09-12T16:02:50.832560+0000","last_clean":"2023-09-12T16:02:50.832560+0000","last_became_active":"2023-09-12T16:02:05.030827+0000","last_became_peered":"2023-09-12T16:02:05.030827+0000","last_unstale":"2023-09-12T16:02:50.832560+0000","last_undegraded":"2023-09-12T16:02:50.832560+0000","last_fullsized":"2023-09-12T16:02:50.832560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:40:48.014344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837378+0000","last_change":"2023-09-12T16:02:05.021273+0000","last_active":"2023-09-12T16:02:50.837378+0000","last_peered":"2023-09-12T16:02:50.837378+0000","last_clean":"2023-09-12T16:02:50.837378+0000","last_became_active":"2023-09-12T16:02:05.020953+0000","last_became_peered":"2023-09-12T16:02:05.020953+0000","last_unstale":"2023-09-12T16:02:50.837378+0000","last_undegraded":"2023-09-12T16:02:50.837378+0000","last_fullsized":"2023-09-12T16:02:50.837378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:33:25.462412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835081+0000","last_change":"2023-09-12T16:02:05.039019+0000","last_active":"2023-09-12T16:02:50.835081+0000","last_peered":"2023-09-12T16:02:50.835081+0000","last_clean":"2023-09-12T16:02:50.835081+0000","last_became_active":"2023-09-12T16:02:05.038621+0000","last_became_peered":"2023-09-12T16:02:05.038621+0000","last_unstale":"2023-09-12T16:02:50.835081+0000","last_undegraded":"2023-09-12T16:02:50.835081+0000","last_fullsized":"2023-09-12T16:02:50.835081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:08:08.934375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025761+0000","last_change":"2023-09-12T16:02:05.039737+0000","last_active":"2023-09-12T16:02:51.025761+0000","last_peered":"2023-09-12T16:02:51.025761+0000","last_clean":"2023-09-12T16:02:51.025761+0000","last_became_active":"2023-09-12T16:02:05.039304+0000","last_became_peered":"2023-09-12T16:02:05.039304+0000","last_unstale":"2023-09-12T16:02:51.025761+0000","last_undegraded":"2023-09-12T16:02:51.025761+0000","last_fullsized":"2023-09-12T16:02:51.025761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:50:13.141915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244717,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69913,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":489,"num_read_kb":318,"num_write":158,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":59815,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":201417,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":489432,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.876}]},{"osd":1,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.755,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80600000000000005}]},{"osd":2,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.5,"5min":0.496,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77600000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78200000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80400000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.85699999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.54900000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89900000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71899999999999997,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.84799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.89100000000000001}]},{"osd":8,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75900000000000001}]},{"osd":9,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.65200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.621,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.84199999999999997}]},{"osd":10,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.61399999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.98199999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.93000000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776163,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":1142931,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.5,"15min":0.501},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.503},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80000000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.752},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82899999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.58999999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.90800000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.79300000000000004}]},{"osd":6,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95299999999999996,"5min":1.042,"15min":1.042},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.56499999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.85099999999999998,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.80700000000000005}]},{"osd":8,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.63700000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.008,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.875},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93400000000000005,"5min":1.073,"15min":1.073},"last":0.86599999999999999}]},{"osd":9,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96299999999999997,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.099,"15min":1.099},"last":0.81999999999999995}]},{"osd":11,"last update":"Tue Sep 12 16:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.79100000000000004,"15min":0.78400000000000003},"min":{"1min":0.68899999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.66000000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.89500000000000002}]}]},{"osd":9,"up_from":63,"seq":270582939688,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":493764,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.48599999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.42799999999999999,"15min":0.436},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.82499999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.372}]},{"osd":1,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.312}]},{"osd":2,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70499999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.46500000000000002,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.72099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81799999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996}]},{"osd":6,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.495},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88700000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.41799999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.70099999999999996,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.80800000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.621}]},{"osd":10,"last update":"Tue Sep 12 16:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.69999999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46000000000000002}]},{"osd":11,"last update":"Tue Sep 12 16:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61199999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168620,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":488675,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66100000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55100000000000005}]},{"osd":1,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.63,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.57399999999999995,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71499999999999997}]},{"osd":2,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.44400000000000001},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.48399999999999999,"15min":0.45400000000000001},"min":{"1min":0.34799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.47399999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":2.5710000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.95999999999999996}]},{"osd":4,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68400000000000005,"15min":0.65500000000000003},"min":{"1min":0.58199999999999996,"5min":0.373,"15min":0.373},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.54000000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.012,"15min":1.012},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71399999999999997,"15min":0.69099999999999995},"min":{"1min":0.50600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0469999999999999,"5min":1.075,"15min":1.075},"last":0.80500000000000005}]},{"osd":7,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.76300000000000001,"15min":0.73599999999999999},"min":{"1min":0.69199999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.872},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.56499999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.75800000000000001}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.84799999999999998}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71899999999999997,"15min":0.70299999999999996},"min":{"1min":0.60199999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.67500000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92600000000000005}]},{"osd":11,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.88600000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1085873,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.40200000000000002,"15min":0.38600000000000001},"min":{"1min":0.34200000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63600000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.433,"15min":0.40699999999999997},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.505}]},{"osd":2,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59099999999999997,"15min":0.55800000000000005},"min":{"1min":0.439,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1639999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0640000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.78200000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.623}]},{"osd":4,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.91200000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.35099999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72599999999999998,"5min":1.069,"15min":1.069},"last":0.749}]},{"osd":5,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65100000000000002,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93999999999999995,"5min":1.909,"15min":1.909},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65000000000000002,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.139,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.90500000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96299999999999997,"5min":1.119,"15min":1.119},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84699999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.85099999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0169999999999999,"5min":1.879,"15min":1.879},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98499999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.998}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96499999999999997,"5min":1.329,"15min":1.329},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.0309999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.59799999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.872}]},{"osd":11,"last update":"Tue Sep 12 16:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.97599999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":484844,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.39200000000000002,"15min":0.39000000000000001},"min":{"1min":0.25600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56999999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.35899999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70099999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.69799999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71899999999999997,"5min":1.079,"15min":1.079},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":1.129,"15min":1.129},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.75,"5min":1.083,"15min":1.083},"last":0.439},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.68300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72699999999999998,"5min":1.161,"15min":1.161},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67800000000000005}]},{"osd":5,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81999999999999995,"5min":1.022,"15min":1.022},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.65300000000000002}]},{"osd":8,"last update":"Tue Sep 12 16:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68400000000000005,"15min":0.70399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83399999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.628}]},{"osd":9,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71799999999999997,"15min":0.72899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.495}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90800000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60499999999999998}]},{"osd":11,"last update":"Tue Sep 12 16:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":545948,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.38100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":6.3369999999999997}]},{"osd":2,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.61099999999999999,"15min":0.54200000000000004},"min":{"1min":0.55000000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.628}]},{"osd":3,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57099999999999995,"15min":0.51900000000000002},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":1.002,"15min":1.002},"last":6.5170000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.498,"15min":0.45600000000000002},"min":{"1min":0.40899999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.435},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.57499999999999996,"15min":0.51900000000000002},"min":{"1min":0.52400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.69999999999999996,"15min":0.65200000000000002},"min":{"1min":0.52100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.036,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.66900000000000004,"15min":0.623},"min":{"1min":0.63,"5min":0.317,"15min":0.317},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.68100000000000005,"15min":0.63700000000000001},"min":{"1min":0.70499999999999996,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60099999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.77300000000000002,"15min":0.73699999999999999},"min":{"1min":0.59599999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1399999999999999,"5min":1.167,"15min":1.167},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69499999999999995,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73299999999999998}]},{"osd":9,"last update":"Tue Sep 12 16:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81000000000000005,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.81999999999999995,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.88900000000000001}]},{"osd":10,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.67300000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77000000000000002}]},{"osd":11,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.81499999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":486232,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58099999999999996,"15min":0.60599999999999998},"min":{"1min":0.38300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82999999999999996,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.48699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":2.113,"15min":2.113},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.52100000000000002},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79300000000000004,"5min":0.874,"15min":0.874},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76600000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.441}]},{"osd":3,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.308},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.33100000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76600000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.74399999999999999,"5min":0.879,"15min":0.879},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":1.004,"15min":1.004},"last":0.41699999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59699999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.53500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.66000000000000003,"15min":0.67600000000000005},"min":{"1min":0.47099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.501}]},{"osd":9,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58099999999999996}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.028,"15min":1.028},"last":0.65700000000000003}]},{"osd":11,"last update":"Tue Sep 12 16:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.59599999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485552,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.377,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79600000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.437},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.499,"15min":0.47899999999999998},"min":{"1min":0.42299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.57999999999999996,"15min":0.51200000000000001},"min":{"1min":0.432,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90600000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.61299999999999999,"15min":0.52400000000000002},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.878}]},{"osd":3,"last update":"Tue Sep 12 16:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.89700000000000002}]},{"osd":5,"last update":"Tue Sep 12 16:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.53300000000000003,"15min":0.48699999999999999},"min":{"1min":0.35399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.875,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.56200000000000006,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85099999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:03:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73599999999999999}]},{"osd":8,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":1.05,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65100000000000002,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.747}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.81399999999999995}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.60799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.747,"15min":0.72099999999999997},"min":{"1min":0.64400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.95599999999999996}]},{"osd":11,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.93100000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485038,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.34999999999999998},"min":{"1min":0.27000000000000002,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.167,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.26700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.317}]},{"osd":1,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82299999999999995,"5min":3.552,"15min":3.552},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.752,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56599999999999995,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86399999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76300000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.44500000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78100000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72799999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56699999999999995}]},{"osd":5,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82899999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.46999999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47599999999999998},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65000000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.36599999999999999}]},{"osd":7,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.71599999999999997,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":7.6929999999999996,"5min":7.6929999999999996,"15min":7.6929999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.50900000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66100000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.083,"15min":1.083},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.75,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.45400000000000001}]},{"osd":10,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.84899999999999998,"5min":0.875,"15min":0.875},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53900000000000003}]},{"osd":11,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58899999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":550496,"data_compressed":30118,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.46999999999999997},"min":{"1min":0.38500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.45800000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88800000000000001,"5min":1.458,"15min":1.458},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.36299999999999999},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.41799999999999998,"15min":0.374},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.436}]},{"osd":2,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58199999999999996,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.316,"15min":0.316},"max":{"1min":1.0249999999999999,"5min":1.383,"15min":1.383},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52900000000000003,"15min":0.49399999999999999},"min":{"1min":0.38600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41199999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.66300000000000003,"15min":0.626},"min":{"1min":0.55100000000000005,"5min":0.379,"15min":0.379},"max":{"1min":0.92100000000000004,"5min":1.54,"15min":1.54},"last":0.752},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.52600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90800000000000003,"5min":1.502,"15min":1.502},"last":0.84599999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.496,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93999999999999995,"5min":1.639,"15min":1.639},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53400000000000003}]},{"osd":8,"last update":"Tue Sep 12 16:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.58799999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.80800000000000005,"15min":0.79600000000000004},"min":{"1min":0.67400000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.018,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.87}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97499999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.622}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.755,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.93000000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1,"5min":1,"15min":1},"last":0.79100000000000004}]},{"osd":11,"last update":"Tue Sep 12 16:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.82299999999999995}]}]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1080753,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.48099999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.376},"min":{"1min":0.29599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.38},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.32400000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40300000000000002}]},{"osd":3,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53000000000000003,"15min":0.54700000000000004},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.495},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.41499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.47399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.433,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81200000000000006}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.62},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.879,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.57499999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72099999999999997}]},{"osd":8,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48899999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58699999999999997}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.752}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.56899999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":61919,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T16:03:48.023 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T16:03:48.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.174+0000 7f5df0ae3700 1 -- 172.21.15.130:0/1624771307 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec100800 msgr2=0x7f5dec1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.174+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/1624771307 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec100800 0x7f5dec1070b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5de400b0d0 tx=0x7f5de400b3e0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.174+0000 7f5df0ae3700 1 -- 172.21.15.130:0/1624771307 shutdown_connections 2023-09-12T16:03:48.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.174+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/1624771307 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec100800 0x7f5dec1070b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.174+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/1624771307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dec0fec70 0x7f5dec0ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.174+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/1624771307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dec0ffe60 0x7f5dec1002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.174+0000 7f5df0ae3700 1 -- 172.21.15.130:0/1624771307 >> 172.21.15.130:0/1624771307 conn(0x7f5dec0fa1c0 msgr2=0x7f5dec0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:48.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.174+0000 7f5df0ae3700 1 -- 172.21.15.130:0/1624771307 shutdown_connections 2023-09-12T16:03:48.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.174+0000 7f5df0ae3700 1 -- 172.21.15.130:0/1624771307 wait complete. 2023-09-12T16:03:48.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.175+0000 7f5df0ae3700 1 Processor -- start 2023-09-12T16:03:48.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.175+0000 7f5df0ae3700 1 -- start start 2023-09-12T16:03:48.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5df0ae3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec0fec70 0x7f5dec195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5df0ae3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dec0ffe60 0x7f5dec1957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5df0ae3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dec100800 0x7f5dec19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5df0ae3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec19cde0 con 0x7f5dec0fec70 2023-09-12T16:03:48.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5df0ae3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec19cf20 con 0x7f5dec100800 2023-09-12T16:03:48.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5df0ae3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec107a00 con 0x7f5dec0ffe60 2023-09-12T16:03:48.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dea59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec0fec70 0x7f5dec195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dea59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec0fec70 0x7f5dec195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55200/0 (socket says 172.21.15.130:55200) 2023-09-12T16:03:48.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dea59c700 1 -- 172.21.15.130:0/2273428784 learned_addr learned my addr 172.21.15.130:0/2273428784 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:48.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dead9d700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dec100800 0x7f5dec19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5de9d9b700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dec0ffe60 0x7f5dec1957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dea59c700 1 -- 172.21.15.130:0/2273428784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dec0ffe60 msgr2=0x7f5dec1957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dea59c700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dec0ffe60 0x7f5dec1957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dea59c700 1 -- 172.21.15.130:0/2273428784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dec100800 msgr2=0x7f5dec19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dea59c700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dec100800 0x7f5dec19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dea59c700 1 -- 172.21.15.130:0/2273428784 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de4009d00 con 0x7f5dec0fec70 2023-09-12T16:03:48.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5dea59c700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec0fec70 0x7f5dec195260 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d980 tx=0x7f5dd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:48.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5de37fe700 1 -- 172.21.15.130:0/2273428784 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4004020 con 0x7f5dec0fec70 2023-09-12T16:03:48.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5de37fe700 1 -- 172.21.15.130:0/2273428784 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd4009730 con 0x7f5dec0fec70 2023-09-12T16:03:48.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5de37fe700 1 -- 172.21.15.130:0/2273428784 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4003a80 con 0x7f5dec0fec70 2023-09-12T16:03:48.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec107ce0 con 0x7f5dec0fec70 2023-09-12T16:03:48.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.176+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec1081a0 con 0x7f5dec0fec70 2023-09-12T16:03:48.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.177+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec10ad20 con 0x7f5dec0fec70 2023-09-12T16:03:48.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.178+0000 7f5de37fe700 1 -- 172.21.15.130:0/2273428784 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f5dd40098a0 con 0x7f5dec0fec70 2023-09-12T16:03:48.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.179+0000 7f5de37fe700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5dd80a0710 0x7f5dd80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.179+0000 7f5de37fe700 1 -- 172.21.15.130:0/2273428784 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5dd40cc070 con 0x7f5dec0fec70 2023-09-12T16:03:48.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.179+0000 7f5de9d9b700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5dd80a0710 0x7f5dd80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.179+0000 7f5de9d9b700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5dd80a0710 0x7f5dd80a2bd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009bf0 tx=0x7f5ddc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:48.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.182+0000 7f5de37fe700 1 -- 172.21.15.130:0/2273428784 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5dd4095730 con 0x7f5dec0fec70 2023-09-12T16:03:48.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:47 smithi159 ceph-mon[102522]: pgmap v316: 129 pgs: 129 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-12T16:03:48.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:47 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2926351807' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:48.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:47 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/378061592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:48.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:47 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2333250305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:48.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:47 smithi203 ceph-mon[103431]: pgmap v316: 129 pgs: 129 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2023-09-12T16:03:48.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:47 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2926351807' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:48.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:47 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/378061592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:48.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:47 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2333250305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:48.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.368+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5dec108960 con 0x7f5dd80a0710 2023-09-12T16:03:48.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.380+0000 7f5de37fe700 1 -- 172.21.15.130:0/2273428784 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427369 (secure 0 0 0) 0x7f5dec108960 con 0x7f5dd80a0710 2023-09-12T16:03:48.385 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:48.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5dd80a0710 msgr2=0x7f5dd80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5dd80a0710 0x7f5dd80a2bd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009bf0 tx=0x7f5ddc009260 comp rx=0 tx=0).stop 2023-09-12T16:03:48.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec0fec70 msgr2=0x7f5dec195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec0fec70 0x7f5dec195260 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d980 tx=0x7f5dd400dc90 comp rx=0 tx=0).stop 2023-09-12T16:03:48.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 shutdown_connections 2023-09-12T16:03:48.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f5dd80a0710 0x7f5dd80a2bd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dec100800 0x7f5dec19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5dec0fec70 0x7f5dec195260 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 --2- 172.21.15.130:0/2273428784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dec0ffe60 0x7f5dec1957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 >> 172.21.15.130:0/2273428784 conn(0x7f5dec0fa1c0 msgr2=0x7f5dec0fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:48.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 shutdown_connections 2023-09-12T16:03:48.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.384+0000 7f5df0ae3700 1 -- 172.21.15.130:0/2273428784 wait complete. 2023-09-12T16:03:48.392 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2023-09-12T16:03:48.398 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2023-09-12T16:03:46.782696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660527,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":615,"num_read_kb":427,"num_write":341,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1240551,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335964,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605860,"statfs":{"total":1151940427776,"available":1151596400640,"internally_reserved":0,"allocated":11980800,"data_stored":7819538,"data_compressed":363759,"data_compressed_allocated":491520,"data_compressed_original":2514944,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4638,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":46,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003026"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025907+0000","last_change":"2023-09-12T16:02:05.039617+0000","last_active":"2023-09-12T16:02:51.025907+0000","last_peered":"2023-09-12T16:02:51.025907+0000","last_clean":"2023-09-12T16:02:51.025907+0000","last_became_active":"2023-09-12T16:02:05.039264+0000","last_became_peered":"2023-09-12T16:02:05.039264+0000","last_unstale":"2023-09-12T16:02:51.025907+0000","last_undegraded":"2023-09-12T16:02:51.025907+0000","last_fullsized":"2023-09-12T16:02:51.025907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:37:01.381406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829291+0000","last_change":"2023-09-12T16:02:05.033151+0000","last_active":"2023-09-12T16:02:50.829291+0000","last_peered":"2023-09-12T16:02:50.829291+0000","last_clean":"2023-09-12T16:02:50.829291+0000","last_became_active":"2023-09-12T16:02:05.032866+0000","last_became_peered":"2023-09-12T16:02:05.032866+0000","last_unstale":"2023-09-12T16:02:50.829291+0000","last_undegraded":"2023-09-12T16:02:50.829291+0000","last_fullsized":"2023-09-12T16:02:50.829291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:35.921428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.032478+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.032124+0000","last_became_peered":"2023-09-12T16:02:05.032124+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:58:54.112577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835786+0000","last_change":"2023-09-12T16:02:05.033088+0000","last_active":"2023-09-12T16:02:50.835786+0000","last_peered":"2023-09-12T16:02:50.835786+0000","last_clean":"2023-09-12T16:02:50.835786+0000","last_became_active":"2023-09-12T16:02:05.032332+0000","last_became_peered":"2023-09-12T16:02:05.032332+0000","last_unstale":"2023-09-12T16:02:50.835786+0000","last_undegraded":"2023-09-12T16:02:50.835786+0000","last_fullsized":"2023-09-12T16:02:50.835786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:25:38.751961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:05.036612+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:05.036215+0000","last_became_peered":"2023-09-12T16:02:05.036215+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:15:04.561751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832126+0000","last_change":"2023-09-12T16:02:05.043816+0000","last_active":"2023-09-12T16:02:50.832126+0000","last_peered":"2023-09-12T16:02:50.832126+0000","last_clean":"2023-09-12T16:02:50.832126+0000","last_became_active":"2023-09-12T16:02:05.043599+0000","last_became_peered":"2023-09-12T16:02:05.043599+0000","last_unstale":"2023-09-12T16:02:50.832126+0000","last_undegraded":"2023-09-12T16:02:50.832126+0000","last_fullsized":"2023-09-12T16:02:50.832126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:32:00.286196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829345+0000","last_change":"2023-09-12T16:02:05.031476+0000","last_active":"2023-09-12T16:02:50.829345+0000","last_peered":"2023-09-12T16:02:50.829345+0000","last_clean":"2023-09-12T16:02:50.829345+0000","last_became_active":"2023-09-12T16:02:05.031150+0000","last_became_peered":"2023-09-12T16:02:05.031150+0000","last_unstale":"2023-09-12T16:02:50.829345+0000","last_undegraded":"2023-09-12T16:02:50.829345+0000","last_fullsized":"2023-09-12T16:02:50.829345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:22:27.405505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831825+0000","last_change":"2023-09-12T16:02:05.031981+0000","last_active":"2023-09-12T16:02:50.831825+0000","last_peered":"2023-09-12T16:02:50.831825+0000","last_clean":"2023-09-12T16:02:50.831825+0000","last_became_active":"2023-09-12T16:02:05.031692+0000","last_became_peered":"2023-09-12T16:02:05.031692+0000","last_unstale":"2023-09-12T16:02:50.831825+0000","last_undegraded":"2023-09-12T16:02:50.831825+0000","last_fullsized":"2023-09-12T16:02:50.831825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:51:00.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838117+0000","last_change":"2023-09-12T16:02:05.029898+0000","last_active":"2023-09-12T16:02:50.838117+0000","last_peered":"2023-09-12T16:02:50.838117+0000","last_clean":"2023-09-12T16:02:50.838117+0000","last_became_active":"2023-09-12T16:02:05.029731+0000","last_became_peered":"2023-09-12T16:02:05.029731+0000","last_unstale":"2023-09-12T16:02:50.838117+0000","last_undegraded":"2023-09-12T16:02:50.838117+0000","last_fullsized":"2023-09-12T16:02:50.838117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:16:31.194561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833228+0000","last_change":"2023-09-12T16:02:05.044846+0000","last_active":"2023-09-12T16:02:50.833228+0000","last_peered":"2023-09-12T16:02:50.833228+0000","last_clean":"2023-09-12T16:02:50.833228+0000","last_became_active":"2023-09-12T16:02:05.044430+0000","last_became_peered":"2023-09-12T16:02:05.044430+0000","last_unstale":"2023-09-12T16:02:50.833228+0000","last_undegraded":"2023-09-12T16:02:50.833228+0000","last_fullsized":"2023-09-12T16:02:50.833228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:15.999080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830392+0000","last_change":"2023-09-12T16:02:05.030285+0000","last_active":"2023-09-12T16:02:50.830392+0000","last_peered":"2023-09-12T16:02:50.830392+0000","last_clean":"2023-09-12T16:02:50.830392+0000","last_became_active":"2023-09-12T16:02:05.029585+0000","last_became_peered":"2023-09-12T16:02:05.029585+0000","last_unstale":"2023-09-12T16:02:50.830392+0000","last_undegraded":"2023-09-12T16:02:50.830392+0000","last_fullsized":"2023-09-12T16:02:50.830392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:52:27.471412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835453+0000","last_change":"2023-09-12T16:02:05.037258+0000","last_active":"2023-09-12T16:02:50.835453+0000","last_peered":"2023-09-12T16:02:50.835453+0000","last_clean":"2023-09-12T16:02:50.835453+0000","last_became_active":"2023-09-12T16:02:05.036941+0000","last_became_peered":"2023-09-12T16:02:05.036941+0000","last_unstale":"2023-09-12T16:02:50.835453+0000","last_undegraded":"2023-09-12T16:02:50.835453+0000","last_fullsized":"2023-09-12T16:02:50.835453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:04:17.998927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835219+0000","last_change":"2023-09-12T16:02:05.041986+0000","last_active":"2023-09-12T16:02:50.835219+0000","last_peered":"2023-09-12T16:02:50.835219+0000","last_clean":"2023-09-12T16:02:50.835219+0000","last_became_active":"2023-09-12T16:02:05.041584+0000","last_became_peered":"2023-09-12T16:02:05.041584+0000","last_unstale":"2023-09-12T16:02:50.835219+0000","last_undegraded":"2023-09-12T16:02:50.835219+0000","last_fullsized":"2023-09-12T16:02:50.835219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:39.065525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828785+0000","last_change":"2023-09-12T16:02:05.030405+0000","last_active":"2023-09-12T16:02:50.828785+0000","last_peered":"2023-09-12T16:02:50.828785+0000","last_clean":"2023-09-12T16:02:50.828785+0000","last_became_active":"2023-09-12T16:02:05.029488+0000","last_became_peered":"2023-09-12T16:02:05.029488+0000","last_unstale":"2023-09-12T16:02:50.828785+0000","last_undegraded":"2023-09-12T16:02:50.828785+0000","last_fullsized":"2023-09-12T16:02:50.828785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.646069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829401+0000","last_change":"2023-09-12T16:02:05.032576+0000","last_active":"2023-09-12T16:02:50.829401+0000","last_peered":"2023-09-12T16:02:50.829401+0000","last_clean":"2023-09-12T16:02:50.829401+0000","last_became_active":"2023-09-12T16:02:05.032277+0000","last_became_peered":"2023-09-12T16:02:05.032277+0000","last_unstale":"2023-09-12T16:02:50.829401+0000","last_undegraded":"2023-09-12T16:02:50.829401+0000","last_fullsized":"2023-09-12T16:02:50.829401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:47:32.734830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836039+0000","last_change":"2023-09-12T16:02:05.023468+0000","last_active":"2023-09-12T16:02:50.836039+0000","last_peered":"2023-09-12T16:02:50.836039+0000","last_clean":"2023-09-12T16:02:50.836039+0000","last_became_active":"2023-09-12T16:02:05.023132+0000","last_became_peered":"2023-09-12T16:02:05.023132+0000","last_unstale":"2023-09-12T16:02:50.836039+0000","last_undegraded":"2023-09-12T16:02:50.836039+0000","last_fullsized":"2023-09-12T16:02:50.836039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:21:49.378779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025810+0000","last_change":"2023-09-12T16:02:05.039910+0000","last_active":"2023-09-12T16:02:51.025810+0000","last_peered":"2023-09-12T16:02:51.025810+0000","last_clean":"2023-09-12T16:02:51.025810+0000","last_became_active":"2023-09-12T16:02:05.039605+0000","last_became_peered":"2023-09-12T16:02:05.039605+0000","last_unstale":"2023-09-12T16:02:51.025810+0000","last_undegraded":"2023-09-12T16:02:51.025810+0000","last_fullsized":"2023-09-12T16:02:51.025810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:19:58.823471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833500+0000","last_change":"2023-09-12T16:02:05.031667+0000","last_active":"2023-09-12T16:02:50.833500+0000","last_peered":"2023-09-12T16:02:50.833500+0000","last_clean":"2023-09-12T16:02:50.833500+0000","last_became_active":"2023-09-12T16:02:05.030886+0000","last_became_peered":"2023-09-12T16:02:05.030886+0000","last_unstale":"2023-09-12T16:02:50.833500+0000","last_undegraded":"2023-09-12T16:02:50.833500+0000","last_fullsized":"2023-09-12T16:02:50.833500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:24:02.312902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836131+0000","last_change":"2023-09-12T16:02:05.037607+0000","last_active":"2023-09-12T16:02:50.836131+0000","last_peered":"2023-09-12T16:02:50.836131+0000","last_clean":"2023-09-12T16:02:50.836131+0000","last_became_active":"2023-09-12T16:02:05.037095+0000","last_became_peered":"2023-09-12T16:02:05.037095+0000","last_unstale":"2023-09-12T16:02:50.836131+0000","last_undegraded":"2023-09-12T16:02:50.836131+0000","last_fullsized":"2023-09-12T16:02:50.836131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:17:04.886737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834805+0000","last_change":"2023-09-12T16:02:09.292093+0000","last_active":"2023-09-12T16:02:50.834805+0000","last_peered":"2023-09-12T16:02:50.834805+0000","last_clean":"2023-09-12T16:02:50.834805+0000","last_became_active":"2023-09-12T16:02:09.291602+0000","last_became_peered":"2023-09-12T16:02:09.291602+0000","last_unstale":"2023-09-12T16:02:50.834805+0000","last_undegraded":"2023-09-12T16:02:50.834805+0000","last_fullsized":"2023-09-12T16:02:50.834805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:36:23.089498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.857296+0000","last_change":"2023-09-12T16:02:51.857296+0000","last_active":"2023-09-12T16:02:51.857296+0000","last_peered":"2023-09-12T16:02:51.857296+0000","last_clean":"2023-09-12T16:02:51.857296+0000","last_became_active":"2023-09-12T16:02:51.843961+0000","last_became_peered":"2023-09-12T16:02:51.843961+0000","last_unstale":"2023-09-12T16:02:51.857296+0000","last_undegraded":"2023-09-12T16:02:51.857296+0000","last_fullsized":"2023-09-12T16:02:51.857296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:55:57.938037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832962+0000","last_change":"2023-09-12T16:02:05.032191+0000","last_active":"2023-09-12T16:02:50.832962+0000","last_peered":"2023-09-12T16:02:50.832962+0000","last_clean":"2023-09-12T16:02:50.832962+0000","last_became_active":"2023-09-12T16:02:05.031888+0000","last_became_peered":"2023-09-12T16:02:05.031888+0000","last_unstale":"2023-09-12T16:02:50.832962+0000","last_undegraded":"2023-09-12T16:02:50.832962+0000","last_fullsized":"2023-09-12T16:02:50.832962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:50:15.775078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834369+0000","last_change":"2023-09-12T16:02:09.290650+0000","last_active":"2023-09-12T16:02:50.834369+0000","last_peered":"2023-09-12T16:02:50.834369+0000","last_clean":"2023-09-12T16:02:50.834369+0000","last_became_active":"2023-09-12T16:02:09.290205+0000","last_became_peered":"2023-09-12T16:02:09.290205+0000","last_unstale":"2023-09-12T16:02:50.834369+0000","last_undegraded":"2023-09-12T16:02:50.834369+0000","last_fullsized":"2023-09-12T16:02:50.834369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:44:04.930063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833124+0000","last_change":"2023-09-12T16:02:04.032506+0000","last_active":"2023-09-12T16:02:50.833124+0000","last_peered":"2023-09-12T16:02:50.833124+0000","last_clean":"2023-09-12T16:02:50.833124+0000","last_became_active":"2023-09-12T16:02:04.032047+0000","last_became_peered":"2023-09-12T16:02:04.032047+0000","last_unstale":"2023-09-12T16:02:50.833124+0000","last_undegraded":"2023-09-12T16:02:50.833124+0000","last_fullsized":"2023-09-12T16:02:50.833124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:30:22.664465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.025692+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.025370+0000","last_became_peered":"2023-09-12T16:02:05.025370+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.305375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.068789+0000","last_change":"2023-09-12T16:02:52.068789+0000","last_active":"2023-09-12T16:02:52.068789+0000","last_peered":"2023-09-12T16:02:52.068789+0000","last_clean":"2023-09-12T16:02:52.068789+0000","last_became_active":"2023-09-12T16:02:52.068276+0000","last_became_peered":"2023-09-12T16:02:52.068276+0000","last_unstale":"2023-09-12T16:02:52.068789+0000","last_undegraded":"2023-09-12T16:02:52.068789+0000","last_fullsized":"2023-09-12T16:02:52.068789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:19:19.009173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831481+0000","last_change":"2023-09-12T16:02:04.032079+0000","last_active":"2023-09-12T16:02:50.831481+0000","last_peered":"2023-09-12T16:02:50.831481+0000","last_clean":"2023-09-12T16:02:50.831481+0000","last_became_active":"2023-09-12T16:02:04.031835+0000","last_became_peered":"2023-09-12T16:02:04.031835+0000","last_unstale":"2023-09-12T16:02:50.831481+0000","last_undegraded":"2023-09-12T16:02:50.831481+0000","last_fullsized":"2023-09-12T16:02:50.831481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:48:33.935012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837888+0000","last_change":"2023-09-12T16:02:05.037671+0000","last_active":"2023-09-12T16:02:50.837888+0000","last_peered":"2023-09-12T16:02:50.837888+0000","last_clean":"2023-09-12T16:02:50.837888+0000","last_became_active":"2023-09-12T16:02:05.037307+0000","last_became_peered":"2023-09-12T16:02:05.037307+0000","last_unstale":"2023-09-12T16:02:50.837888+0000","last_undegraded":"2023-09-12T16:02:50.837888+0000","last_fullsized":"2023-09-12T16:02:50.837888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:47:23.836612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834100+0000","last_change":"2023-09-12T16:02:09.289165+0000","last_active":"2023-09-12T16:02:50.834100+0000","last_peered":"2023-09-12T16:02:50.834100+0000","last_clean":"2023-09-12T16:02:50.834100+0000","last_became_active":"2023-09-12T16:02:09.288903+0000","last_became_peered":"2023-09-12T16:02:09.288903+0000","last_unstale":"2023-09-12T16:02:50.834100+0000","last_undegraded":"2023-09-12T16:02:50.834100+0000","last_fullsized":"2023-09-12T16:02:50.834100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:27:09.160822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831262+0000","last_change":"2023-09-12T16:02:04.027279+0000","last_active":"2023-09-12T16:02:50.831262+0000","last_peered":"2023-09-12T16:02:50.831262+0000","last_clean":"2023-09-12T16:02:50.831262+0000","last_became_active":"2023-09-12T16:02:04.026975+0000","last_became_peered":"2023-09-12T16:02:04.026975+0000","last_unstale":"2023-09-12T16:02:50.831262+0000","last_undegraded":"2023-09-12T16:02:50.831262+0000","last_fullsized":"2023-09-12T16:02:50.831262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:45.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832885+0000","last_change":"2023-09-12T16:02:05.027582+0000","last_active":"2023-09-12T16:02:50.832885+0000","last_peered":"2023-09-12T16:02:50.832885+0000","last_clean":"2023-09-12T16:02:50.832885+0000","last_became_active":"2023-09-12T16:02:05.027236+0000","last_became_peered":"2023-09-12T16:02:05.027236+0000","last_unstale":"2023-09-12T16:02:50.832885+0000","last_undegraded":"2023-09-12T16:02:50.832885+0000","last_fullsized":"2023-09-12T16:02:50.832885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:06:24.812731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836055+0000","last_change":"2023-09-12T16:02:09.284381+0000","last_active":"2023-09-12T16:02:50.836055+0000","last_peered":"2023-09-12T16:02:50.836055+0000","last_clean":"2023-09-12T16:02:50.836055+0000","last_became_active":"2023-09-12T16:02:09.283947+0000","last_became_peered":"2023-09-12T16:02:09.283947+0000","last_unstale":"2023-09-12T16:02:50.836055+0000","last_undegraded":"2023-09-12T16:02:50.836055+0000","last_fullsized":"2023-09-12T16:02:50.836055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:05:24.803781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:40.770486+0000","last_change":"2023-09-12T16:02:04.027693+0000","last_active":"2023-09-12T16:03:40.770486+0000","last_peered":"2023-09-12T16:03:40.770486+0000","last_clean":"2023-09-12T16:03:40.770486+0000","last_became_active":"2023-09-12T16:02:04.027315+0000","last_became_peered":"2023-09-12T16:02:04.027315+0000","last_unstale":"2023-09-12T16:03:40.770486+0000","last_undegraded":"2023-09-12T16:03:40.770486+0000","last_fullsized":"2023-09-12T16:03:40.770486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:32:21.756802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831054+0000","last_change":"2023-09-12T16:02:05.031673+0000","last_active":"2023-09-12T16:02:50.831054+0000","last_peered":"2023-09-12T16:02:50.831054+0000","last_clean":"2023-09-12T16:02:50.831054+0000","last_became_active":"2023-09-12T16:02:05.031364+0000","last_became_peered":"2023-09-12T16:02:05.031364+0000","last_unstale":"2023-09-12T16:02:50.831054+0000","last_undegraded":"2023-09-12T16:02:50.831054+0000","last_fullsized":"2023-09-12T16:02:50.831054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:48:05.222530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835641+0000","last_change":"2023-09-12T16:02:09.288536+0000","last_active":"2023-09-12T16:02:50.835641+0000","last_peered":"2023-09-12T16:02:50.835641+0000","last_clean":"2023-09-12T16:02:50.835641+0000","last_became_active":"2023-09-12T16:02:09.288062+0000","last_became_peered":"2023-09-12T16:02:09.288062+0000","last_unstale":"2023-09-12T16:02:50.835641+0000","last_undegraded":"2023-09-12T16:02:50.835641+0000","last_fullsized":"2023-09-12T16:02:50.835641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:39:28.935563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832024+0000","last_change":"2023-09-12T16:02:04.026298+0000","last_active":"2023-09-12T16:02:50.832024+0000","last_peered":"2023-09-12T16:02:50.832024+0000","last_clean":"2023-09-12T16:02:50.832024+0000","last_became_active":"2023-09-12T16:02:04.025975+0000","last_became_peered":"2023-09-12T16:02:04.025975+0000","last_unstale":"2023-09-12T16:02:50.832024+0000","last_undegraded":"2023-09-12T16:02:50.832024+0000","last_fullsized":"2023-09-12T16:02:50.832024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:12:04.338497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832502+0000","last_change":"2023-09-12T16:02:05.028337+0000","last_active":"2023-09-12T16:02:50.832502+0000","last_peered":"2023-09-12T16:02:50.832502+0000","last_clean":"2023-09-12T16:02:50.832502+0000","last_became_active":"2023-09-12T16:02:05.028036+0000","last_became_peered":"2023-09-12T16:02:05.028036+0000","last_unstale":"2023-09-12T16:02:50.832502+0000","last_undegraded":"2023-09-12T16:02:50.832502+0000","last_fullsized":"2023-09-12T16:02:50.832502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:39:09.387191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025809+0000","last_change":"2023-09-12T16:02:09.290950+0000","last_active":"2023-09-12T16:02:51.025809+0000","last_peered":"2023-09-12T16:02:51.025809+0000","last_clean":"2023-09-12T16:02:51.025809+0000","last_became_active":"2023-09-12T16:02:09.290477+0000","last_became_peered":"2023-09-12T16:02:09.290477+0000","last_unstale":"2023-09-12T16:02:51.025809+0000","last_undegraded":"2023-09-12T16:02:51.025809+0000","last_fullsized":"2023-09-12T16:02:51.025809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:53:45.907575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834845+0000","last_change":"2023-09-12T16:02:04.026025+0000","last_active":"2023-09-12T16:02:50.834845+0000","last_peered":"2023-09-12T16:02:50.834845+0000","last_clean":"2023-09-12T16:02:50.834845+0000","last_became_active":"2023-09-12T16:02:04.025722+0000","last_became_peered":"2023-09-12T16:02:04.025722+0000","last_unstale":"2023-09-12T16:02:50.834845+0000","last_undegraded":"2023-09-12T16:02:50.834845+0000","last_fullsized":"2023-09-12T16:02:50.834845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:29:58.357960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831920+0000","last_change":"2023-09-12T16:02:05.028461+0000","last_active":"2023-09-12T16:02:50.831920+0000","last_peered":"2023-09-12T16:02:50.831920+0000","last_clean":"2023-09-12T16:02:50.831920+0000","last_became_active":"2023-09-12T16:02:05.028176+0000","last_became_peered":"2023-09-12T16:02:05.028176+0000","last_unstale":"2023-09-12T16:02:50.831920+0000","last_undegraded":"2023-09-12T16:02:50.831920+0000","last_fullsized":"2023-09-12T16:02:50.831920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:16:48.652513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829138+0000","last_change":"2023-09-12T16:02:09.295245+0000","last_active":"2023-09-12T16:02:50.829138+0000","last_peered":"2023-09-12T16:02:50.829138+0000","last_clean":"2023-09-12T16:02:50.829138+0000","last_became_active":"2023-09-12T16:02:09.294872+0000","last_became_peered":"2023-09-12T16:02:09.294872+0000","last_unstale":"2023-09-12T16:02:50.829138+0000","last_undegraded":"2023-09-12T16:02:50.829138+0000","last_fullsized":"2023-09-12T16:02:50.829138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:43:49.462317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832140+0000","last_change":"2023-09-12T16:02:04.030815+0000","last_active":"2023-09-12T16:02:50.832140+0000","last_peered":"2023-09-12T16:02:50.832140+0000","last_clean":"2023-09-12T16:02:50.832140+0000","last_became_active":"2023-09-12T16:02:04.030435+0000","last_became_peered":"2023-09-12T16:02:04.030435+0000","last_unstale":"2023-09-12T16:02:50.832140+0000","last_undegraded":"2023-09-12T16:02:50.832140+0000","last_fullsized":"2023-09-12T16:02:50.832140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:39:59.661143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832565+0000","last_change":"2023-09-12T16:02:05.031155+0000","last_active":"2023-09-12T16:02:50.832565+0000","last_peered":"2023-09-12T16:02:50.832565+0000","last_clean":"2023-09-12T16:02:50.832565+0000","last_became_active":"2023-09-12T16:02:05.030803+0000","last_became_peered":"2023-09-12T16:02:05.030803+0000","last_unstale":"2023-09-12T16:02:50.832565+0000","last_undegraded":"2023-09-12T16:02:50.832565+0000","last_fullsized":"2023-09-12T16:02:50.832565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:55:21.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832473+0000","last_change":"2023-09-12T16:02:09.286445+0000","last_active":"2023-09-12T16:02:50.832473+0000","last_peered":"2023-09-12T16:02:50.832473+0000","last_clean":"2023-09-12T16:02:50.832473+0000","last_became_active":"2023-09-12T16:02:09.286049+0000","last_became_peered":"2023-09-12T16:02:09.286049+0000","last_unstale":"2023-09-12T16:02:50.832473+0000","last_undegraded":"2023-09-12T16:02:50.832473+0000","last_fullsized":"2023-09-12T16:02:50.832473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:43:13.943984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:44.775459+0000","last_change":"2023-09-12T16:02:04.035341+0000","last_active":"2023-09-12T16:03:44.775459+0000","last_peered":"2023-09-12T16:03:44.775459+0000","last_clean":"2023-09-12T16:03:44.775459+0000","last_became_active":"2023-09-12T16:02:04.035092+0000","last_became_peered":"2023-09-12T16:02:04.035092+0000","last_unstale":"2023-09-12T16:03:44.775459+0000","last_undegraded":"2023-09-12T16:03:44.775459+0000","last_fullsized":"2023-09-12T16:03:44.775459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:33:00.784401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838363+0000","last_change":"2023-09-12T16:02:05.025229+0000","last_active":"2023-09-12T16:02:50.838363+0000","last_peered":"2023-09-12T16:02:50.838363+0000","last_clean":"2023-09-12T16:02:50.838363+0000","last_became_active":"2023-09-12T16:02:05.024963+0000","last_became_peered":"2023-09-12T16:02:05.024963+0000","last_unstale":"2023-09-12T16:02:50.838363+0000","last_undegraded":"2023-09-12T16:02:50.838363+0000","last_fullsized":"2023-09-12T16:02:50.838363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:46:04.318082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832595+0000","last_change":"2023-09-12T16:02:09.295011+0000","last_active":"2023-09-12T16:02:50.832595+0000","last_peered":"2023-09-12T16:02:50.832595+0000","last_clean":"2023-09-12T16:02:50.832595+0000","last_became_active":"2023-09-12T16:02:09.294617+0000","last_became_peered":"2023-09-12T16:02:09.294617+0000","last_unstale":"2023-09-12T16:02:50.832595+0000","last_undegraded":"2023-09-12T16:02:50.832595+0000","last_fullsized":"2023-09-12T16:02:50.832595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:57:53.902946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830613+0000","last_change":"2023-09-12T16:02:04.028086+0000","last_active":"2023-09-12T16:02:50.830613+0000","last_peered":"2023-09-12T16:02:50.830613+0000","last_clean":"2023-09-12T16:02:50.830613+0000","last_became_active":"2023-09-12T16:02:04.027798+0000","last_became_peered":"2023-09-12T16:02:04.027798+0000","last_unstale":"2023-09-12T16:02:50.830613+0000","last_undegraded":"2023-09-12T16:02:50.830613+0000","last_fullsized":"2023-09-12T16:02:50.830613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:11:16.921824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829002+0000","last_change":"2023-09-12T16:02:05.030691+0000","last_active":"2023-09-12T16:02:50.829002+0000","last_peered":"2023-09-12T16:02:50.829002+0000","last_clean":"2023-09-12T16:02:50.829002+0000","last_became_active":"2023-09-12T16:02:05.030379+0000","last_became_peered":"2023-09-12T16:02:05.030379+0000","last_unstale":"2023-09-12T16:02:50.829002+0000","last_undegraded":"2023-09-12T16:02:50.829002+0000","last_fullsized":"2023-09-12T16:02:50.829002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:15:05.221078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835660+0000","last_change":"2023-09-12T16:02:09.290703+0000","last_active":"2023-09-12T16:02:50.835660+0000","last_peered":"2023-09-12T16:02:50.835660+0000","last_clean":"2023-09-12T16:02:50.835660+0000","last_became_active":"2023-09-12T16:02:09.290227+0000","last_became_peered":"2023-09-12T16:02:09.290227+0000","last_unstale":"2023-09-12T16:02:50.835660+0000","last_undegraded":"2023-09-12T16:02:50.835660+0000","last_fullsized":"2023-09-12T16:02:50.835660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:07:06.182295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832824+0000","last_change":"2023-09-12T16:02:04.028630+0000","last_active":"2023-09-12T16:02:50.832824+0000","last_peered":"2023-09-12T16:02:50.832824+0000","last_clean":"2023-09-12T16:02:50.832824+0000","last_became_active":"2023-09-12T16:02:04.028319+0000","last_became_peered":"2023-09-12T16:02:04.028319+0000","last_unstale":"2023-09-12T16:02:50.832824+0000","last_undegraded":"2023-09-12T16:02:50.832824+0000","last_fullsized":"2023-09-12T16:02:50.832824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:13:43.349425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838071+0000","last_change":"2023-09-12T16:02:05.032392+0000","last_active":"2023-09-12T16:02:50.838071+0000","last_peered":"2023-09-12T16:02:50.838071+0000","last_clean":"2023-09-12T16:02:50.838071+0000","last_became_active":"2023-09-12T16:02:05.032099+0000","last_became_peered":"2023-09-12T16:02:05.032099+0000","last_unstale":"2023-09-12T16:02:50.838071+0000","last_undegraded":"2023-09-12T16:02:50.838071+0000","last_fullsized":"2023-09-12T16:02:50.838071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:46:03.108459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832745+0000","last_change":"2023-09-12T16:02:09.289573+0000","last_active":"2023-09-12T16:02:50.832745+0000","last_peered":"2023-09-12T16:02:50.832745+0000","last_clean":"2023-09-12T16:02:50.832745+0000","last_became_active":"2023-09-12T16:02:09.289179+0000","last_became_peered":"2023-09-12T16:02:09.289179+0000","last_unstale":"2023-09-12T16:02:50.832745+0000","last_undegraded":"2023-09-12T16:02:50.832745+0000","last_fullsized":"2023-09-12T16:02:50.832745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:52:35.905346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:44.931972+0000","last_change":"2023-09-12T16:02:51.851180+0000","last_active":"2023-09-12T16:03:44.931972+0000","last_peered":"2023-09-12T16:03:44.931972+0000","last_clean":"2023-09-12T16:03:44.931972+0000","last_became_active":"2023-09-12T16:02:51.841452+0000","last_became_peered":"2023-09-12T16:02:51.841452+0000","last_unstale":"2023-09-12T16:03:44.931972+0000","last_undegraded":"2023-09-12T16:03:44.931972+0000","last_fullsized":"2023-09-12T16:03:44.931972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:20:20.903637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65523,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":138,"num_write":35,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":59233,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831192+0000","last_change":"2023-09-12T16:02:05.031409+0000","last_active":"2023-09-12T16:02:50.831192+0000","last_peered":"2023-09-12T16:02:50.831192+0000","last_clean":"2023-09-12T16:02:50.831192+0000","last_became_active":"2023-09-12T16:02:05.031110+0000","last_became_peered":"2023-09-12T16:02:05.031110+0000","last_unstale":"2023-09-12T16:02:50.831192+0000","last_undegraded":"2023-09-12T16:02:50.831192+0000","last_fullsized":"2023-09-12T16:02:50.831192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:10:54.482078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833098+0000","last_change":"2023-09-12T16:02:09.291493+0000","last_active":"2023-09-12T16:02:50.833098+0000","last_peered":"2023-09-12T16:02:50.833098+0000","last_clean":"2023-09-12T16:02:50.833098+0000","last_became_active":"2023-09-12T16:02:09.291044+0000","last_became_peered":"2023-09-12T16:02:09.291044+0000","last_unstale":"2023-09-12T16:02:50.833098+0000","last_undegraded":"2023-09-12T16:02:50.833098+0000","last_fullsized":"2023-09-12T16:02:50.833098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:34:30.269720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835682+0000","last_change":"2023-09-12T16:02:04.029283+0000","last_active":"2023-09-12T16:02:50.835682+0000","last_peered":"2023-09-12T16:02:50.835682+0000","last_clean":"2023-09-12T16:02:50.835682+0000","last_became_active":"2023-09-12T16:02:04.028959+0000","last_became_peered":"2023-09-12T16:02:04.028959+0000","last_unstale":"2023-09-12T16:02:50.835682+0000","last_undegraded":"2023-09-12T16:02:50.835682+0000","last_fullsized":"2023-09-12T16:02:50.835682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:28:52.750278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834981+0000","last_change":"2023-09-12T16:02:05.034140+0000","last_active":"2023-09-12T16:02:50.834981+0000","last_peered":"2023-09-12T16:02:50.834981+0000","last_clean":"2023-09-12T16:02:50.834981+0000","last_became_active":"2023-09-12T16:02:05.033840+0000","last_became_peered":"2023-09-12T16:02:05.033840+0000","last_unstale":"2023-09-12T16:02:50.834981+0000","last_undegraded":"2023-09-12T16:02:50.834981+0000","last_fullsized":"2023-09-12T16:02:50.834981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:59:55.522608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025577+0000","last_change":"2023-09-12T16:02:09.289998+0000","last_active":"2023-09-12T16:02:51.025577+0000","last_peered":"2023-09-12T16:02:51.025577+0000","last_clean":"2023-09-12T16:02:51.025577+0000","last_became_active":"2023-09-12T16:02:09.289561+0000","last_became_peered":"2023-09-12T16:02:09.289561+0000","last_unstale":"2023-09-12T16:02:51.025577+0000","last_undegraded":"2023-09-12T16:02:51.025577+0000","last_fullsized":"2023-09-12T16:02:51.025577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:09:47.327213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832456+0000","last_change":"2023-09-12T16:02:04.029883+0000","last_active":"2023-09-12T16:02:50.832456+0000","last_peered":"2023-09-12T16:02:50.832456+0000","last_clean":"2023-09-12T16:02:50.832456+0000","last_became_active":"2023-09-12T16:02:04.029547+0000","last_became_peered":"2023-09-12T16:02:04.029547+0000","last_unstale":"2023-09-12T16:02:50.832456+0000","last_undegraded":"2023-09-12T16:02:50.832456+0000","last_fullsized":"2023-09-12T16:02:50.832456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:47:46.478325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846042+0000","last_change":"2023-09-12T16:02:51.846042+0000","last_active":"2023-09-12T16:02:51.846042+0000","last_peered":"2023-09-12T16:02:51.846042+0000","last_clean":"2023-09-12T16:02:51.846042+0000","last_became_active":"2023-09-12T16:02:51.845572+0000","last_became_peered":"2023-09-12T16:02:51.845572+0000","last_unstale":"2023-09-12T16:02:51.846042+0000","last_undegraded":"2023-09-12T16:02:51.846042+0000","last_fullsized":"2023-09-12T16:02:51.846042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:55.335485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069758+0000","last_change":"2023-09-12T16:02:52.069157+0000","last_active":"2023-09-12T16:02:52.069758+0000","last_peered":"2023-09-12T16:02:52.069758+0000","last_clean":"2023-09-12T16:02:52.069758+0000","last_became_active":"2023-09-12T16:02:52.068600+0000","last_became_peered":"2023-09-12T16:02:52.068600+0000","last_unstale":"2023-09-12T16:02:52.069758+0000","last_undegraded":"2023-09-12T16:02:52.069758+0000","last_fullsized":"2023-09-12T16:02:52.069758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:28:44.160612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:40.769628+0000","last_change":"2023-09-12T16:02:04.025723+0000","last_active":"2023-09-12T16:03:40.769628+0000","last_peered":"2023-09-12T16:03:40.769628+0000","last_clean":"2023-09-12T16:03:40.769628+0000","last_became_active":"2023-09-12T16:02:04.025432+0000","last_became_peered":"2023-09-12T16:02:04.025432+0000","last_unstale":"2023-09-12T16:03:40.769628+0000","last_undegraded":"2023-09-12T16:03:40.769628+0000","last_fullsized":"2023-09-12T16:03:40.769628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:56.279910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.839291+0000","last_change":"2023-09-12T16:02:51.839291+0000","last_active":"2023-09-12T16:02:51.839291+0000","last_peered":"2023-09-12T16:02:51.839291+0000","last_clean":"2023-09-12T16:02:51.839291+0000","last_became_active":"2023-09-12T16:02:51.838968+0000","last_became_peered":"2023-09-12T16:02:51.838968+0000","last_unstale":"2023-09-12T16:02:51.839291+0000","last_undegraded":"2023-09-12T16:02:51.839291+0000","last_fullsized":"2023-09-12T16:02:51.839291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:21:12.542019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835124+0000","last_change":"2023-09-12T16:02:09.286447+0000","last_active":"2023-09-12T16:02:50.835124+0000","last_peered":"2023-09-12T16:02:50.835124+0000","last_clean":"2023-09-12T16:02:50.835124+0000","last_became_active":"2023-09-12T16:02:09.285805+0000","last_became_peered":"2023-09-12T16:02:09.285805+0000","last_unstale":"2023-09-12T16:02:50.835124+0000","last_undegraded":"2023-09-12T16:02:50.835124+0000","last_fullsized":"2023-09-12T16:02:50.835124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:35:05.518872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830710+0000","last_change":"2023-09-12T16:02:04.032527+0000","last_active":"2023-09-12T16:02:50.830710+0000","last_peered":"2023-09-12T16:02:50.830710+0000","last_clean":"2023-09-12T16:02:50.830710+0000","last_became_active":"2023-09-12T16:02:04.032264+0000","last_became_peered":"2023-09-12T16:02:04.032264+0000","last_unstale":"2023-09-12T16:02:50.830710+0000","last_undegraded":"2023-09-12T16:02:50.830710+0000","last_fullsized":"2023-09-12T16:02:50.830710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:53:52.433549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833597+0000","last_change":"2023-09-12T16:02:05.029281+0000","last_active":"2023-09-12T16:02:50.833597+0000","last_peered":"2023-09-12T16:02:50.833597+0000","last_clean":"2023-09-12T16:02:50.833597+0000","last_became_active":"2023-09-12T16:02:05.028919+0000","last_became_peered":"2023-09-12T16:02:05.028919+0000","last_unstale":"2023-09-12T16:02:50.833597+0000","last_undegraded":"2023-09-12T16:02:50.833597+0000","last_fullsized":"2023-09-12T16:02:50.833597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:26.667060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069744+0000","last_change":"2023-09-12T16:02:52.069336+0000","last_active":"2023-09-12T16:02:52.069744+0000","last_peered":"2023-09-12T16:02:52.069744+0000","last_clean":"2023-09-12T16:02:52.069744+0000","last_became_active":"2023-09-12T16:02:52.068805+0000","last_became_peered":"2023-09-12T16:02:52.068805+0000","last_unstale":"2023-09-12T16:02:52.069744+0000","last_undegraded":"2023-09-12T16:02:52.069744+0000","last_fullsized":"2023-09-12T16:02:52.069744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:09.095952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831444+0000","last_change":"2023-09-12T16:02:04.026373+0000","last_active":"2023-09-12T16:02:50.831444+0000","last_peered":"2023-09-12T16:02:50.831444+0000","last_clean":"2023-09-12T16:02:50.831444+0000","last_became_active":"2023-09-12T16:02:04.025907+0000","last_became_peered":"2023-09-12T16:02:04.025907+0000","last_unstale":"2023-09-12T16:02:50.831444+0000","last_undegraded":"2023-09-12T16:02:50.831444+0000","last_fullsized":"2023-09-12T16:02:50.831444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:33:15.152128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025705+0000","last_change":"2023-09-12T16:02:05.040143+0000","last_active":"2023-09-12T16:02:51.025705+0000","last_peered":"2023-09-12T16:02:51.025705+0000","last_clean":"2023-09-12T16:02:51.025705+0000","last_became_active":"2023-09-12T16:02:05.039749+0000","last_became_peered":"2023-09-12T16:02:05.039749+0000","last_unstale":"2023-09-12T16:02:51.025705+0000","last_undegraded":"2023-09-12T16:02:51.025705+0000","last_fullsized":"2023-09-12T16:02:51.025705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:49.561252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:09.841670+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:09.841203+0000","last_became_peered":"2023-09-12T16:02:09.841203+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:55:46.128996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833178+0000","last_change":"2023-09-12T16:02:04.027953+0000","last_active":"2023-09-12T16:02:50.833178+0000","last_peered":"2023-09-12T16:02:50.833178+0000","last_clean":"2023-09-12T16:02:50.833178+0000","last_became_active":"2023-09-12T16:02:04.027624+0000","last_became_peered":"2023-09-12T16:02:04.027624+0000","last_unstale":"2023-09-12T16:02:50.833178+0000","last_undegraded":"2023-09-12T16:02:50.833178+0000","last_fullsized":"2023-09-12T16:02:50.833178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:33:58.356358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832388+0000","last_change":"2023-09-12T16:02:05.036593+0000","last_active":"2023-09-12T16:02:50.832388+0000","last_peered":"2023-09-12T16:02:50.832388+0000","last_clean":"2023-09-12T16:02:50.832388+0000","last_became_active":"2023-09-12T16:02:05.036204+0000","last_became_peered":"2023-09-12T16:02:05.036204+0000","last_unstale":"2023-09-12T16:02:50.832388+0000","last_undegraded":"2023-09-12T16:02:50.832388+0000","last_fullsized":"2023-09-12T16:02:50.832388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:12:31.068837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835475+0000","last_change":"2023-09-12T16:02:09.292405+0000","last_active":"2023-09-12T16:02:50.835475+0000","last_peered":"2023-09-12T16:02:50.835475+0000","last_clean":"2023-09-12T16:02:50.835475+0000","last_became_active":"2023-09-12T16:02:09.291877+0000","last_became_peered":"2023-09-12T16:02:09.291877+0000","last_unstale":"2023-09-12T16:02:50.835475+0000","last_undegraded":"2023-09-12T16:02:50.835475+0000","last_fullsized":"2023-09-12T16:02:50.835475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:53:04.847310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829465+0000","last_change":"2023-09-12T16:02:04.034738+0000","last_active":"2023-09-12T16:02:50.829465+0000","last_peered":"2023-09-12T16:02:50.829465+0000","last_clean":"2023-09-12T16:02:50.829465+0000","last_became_active":"2023-09-12T16:02:04.034408+0000","last_became_peered":"2023-09-12T16:02:04.034408+0000","last_unstale":"2023-09-12T16:02:50.829465+0000","last_undegraded":"2023-09-12T16:02:50.829465+0000","last_fullsized":"2023-09-12T16:02:50.829465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:27.845770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832464+0000","last_change":"2023-09-12T16:02:05.044936+0000","last_active":"2023-09-12T16:02:50.832464+0000","last_peered":"2023-09-12T16:02:50.832464+0000","last_clean":"2023-09-12T16:02:50.832464+0000","last_became_active":"2023-09-12T16:02:05.044617+0000","last_became_peered":"2023-09-12T16:02:05.044617+0000","last_unstale":"2023-09-12T16:02:50.832464+0000","last_undegraded":"2023-09-12T16:02:50.832464+0000","last_fullsized":"2023-09-12T16:02:50.832464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:41:32.639331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834992+0000","last_change":"2023-09-12T16:02:09.841919+0000","last_active":"2023-09-12T16:02:50.834992+0000","last_peered":"2023-09-12T16:02:50.834992+0000","last_clean":"2023-09-12T16:02:50.834992+0000","last_became_active":"2023-09-12T16:02:09.841410+0000","last_became_peered":"2023-09-12T16:02:09.841410+0000","last_unstale":"2023-09-12T16:02:50.834992+0000","last_undegraded":"2023-09-12T16:02:50.834992+0000","last_fullsized":"2023-09-12T16:02:50.834992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:29:57.511693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.840116+0000","last_change":"2023-09-12T16:02:51.840116+0000","last_active":"2023-09-12T16:02:51.840116+0000","last_peered":"2023-09-12T16:02:51.840116+0000","last_clean":"2023-09-12T16:02:51.840116+0000","last_became_active":"2023-09-12T16:02:51.839698+0000","last_became_peered":"2023-09-12T16:02:51.839698+0000","last_unstale":"2023-09-12T16:02:51.840116+0000","last_undegraded":"2023-09-12T16:02:51.840116+0000","last_fullsized":"2023-09-12T16:02:51.840116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:29.475855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:43.368647+0000","last_change":"2023-09-12T16:02:04.032892+0000","last_active":"2023-09-12T16:03:43.368647+0000","last_peered":"2023-09-12T16:03:43.368647+0000","last_clean":"2023-09-12T16:03:43.368647+0000","last_became_active":"2023-09-12T16:02:04.032637+0000","last_became_peered":"2023-09-12T16:02:04.032637+0000","last_unstale":"2023-09-12T16:03:43.368647+0000","last_undegraded":"2023-09-12T16:03:43.368647+0000","last_fullsized":"2023-09-12T16:03:43.368647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:22:55.805000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830231+0000","last_change":"2023-09-12T16:02:09.285127+0000","last_active":"2023-09-12T16:02:50.830231+0000","last_peered":"2023-09-12T16:02:50.830231+0000","last_clean":"2023-09-12T16:02:50.830231+0000","last_became_active":"2023-09-12T16:02:09.284790+0000","last_became_peered":"2023-09-12T16:02:09.284790+0000","last_unstale":"2023-09-12T16:02:50.830231+0000","last_undegraded":"2023-09-12T16:02:50.830231+0000","last_fullsized":"2023-09-12T16:02:50.830231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:50:50.578069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832025+0000","last_change":"2023-09-12T16:02:05.027877+0000","last_active":"2023-09-12T16:02:50.832025+0000","last_peered":"2023-09-12T16:02:50.832025+0000","last_clean":"2023-09-12T16:02:50.832025+0000","last_became_active":"2023-09-12T16:02:05.027582+0000","last_became_peered":"2023-09-12T16:02:05.027582+0000","last_unstale":"2023-09-12T16:02:50.832025+0000","last_undegraded":"2023-09-12T16:02:50.832025+0000","last_fullsized":"2023-09-12T16:02:50.832025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:47.835790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835647+0000","last_change":"2023-09-12T16:02:04.024800+0000","last_active":"2023-09-12T16:02:50.835647+0000","last_peered":"2023-09-12T16:02:50.835647+0000","last_clean":"2023-09-12T16:02:50.835647+0000","last_became_active":"2023-09-12T16:02:04.024436+0000","last_became_peered":"2023-09-12T16:02:04.024436+0000","last_unstale":"2023-09-12T16:02:50.835647+0000","last_undegraded":"2023-09-12T16:02:50.835647+0000","last_fullsized":"2023-09-12T16:02:50.835647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:57:35.093322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829225+0000","last_change":"2023-09-12T16:01:04.599706+0000","last_active":"2023-09-12T16:02:50.829225+0000","last_peered":"2023-09-12T16:02:50.829225+0000","last_clean":"2023-09-12T16:02:50.829225+0000","last_became_active":"2023-09-12T16:01:04.599328+0000","last_became_peered":"2023-09-12T16:01:04.599328+0000","last_unstale":"2023-09-12T16:02:50.829225+0000","last_undegraded":"2023-09-12T16:02:50.829225+0000","last_fullsized":"2023-09-12T16:02:50.829225+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_clean_scrub_stamp":"2023-09-12T15:58:15.250648+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:57:56.846488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833462+0000","last_change":"2023-09-12T16:02:09.841960+0000","last_active":"2023-09-12T16:02:50.833462+0000","last_peered":"2023-09-12T16:02:50.833462+0000","last_clean":"2023-09-12T16:02:50.833462+0000","last_became_active":"2023-09-12T16:02:09.841279+0000","last_became_peered":"2023-09-12T16:02:09.841279+0000","last_unstale":"2023-09-12T16:02:50.833462+0000","last_undegraded":"2023-09-12T16:02:50.833462+0000","last_fullsized":"2023-09-12T16:02:50.833462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:17:19.419350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.847600+0000","last_change":"2023-09-12T16:02:51.847600+0000","last_active":"2023-09-12T16:02:51.847600+0000","last_peered":"2023-09-12T16:02:51.847600+0000","last_clean":"2023-09-12T16:02:51.847600+0000","last_became_active":"2023-09-12T16:02:51.847292+0000","last_became_peered":"2023-09-12T16:02:51.847292+0000","last_unstale":"2023-09-12T16:02:51.847600+0000","last_undegraded":"2023-09-12T16:02:51.847600+0000","last_fullsized":"2023-09-12T16:02:51.847600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:25:58.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829344+0000","last_change":"2023-09-12T16:02:04.029790+0000","last_active":"2023-09-12T16:02:50.829344+0000","last_peered":"2023-09-12T16:02:50.829344+0000","last_clean":"2023-09-12T16:02:50.829344+0000","last_became_active":"2023-09-12T16:02:04.029440+0000","last_became_peered":"2023-09-12T16:02:04.029440+0000","last_unstale":"2023-09-12T16:02:50.829344+0000","last_undegraded":"2023-09-12T16:02:50.829344+0000","last_fullsized":"2023-09-12T16:02:50.829344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:50:33.690628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025552+0000","last_change":"2023-09-12T16:02:09.289757+0000","last_active":"2023-09-12T16:02:51.025552+0000","last_peered":"2023-09-12T16:02:51.025552+0000","last_clean":"2023-09-12T16:02:51.025552+0000","last_became_active":"2023-09-12T16:02:09.289348+0000","last_became_peered":"2023-09-12T16:02:09.289348+0000","last_unstale":"2023-09-12T16:02:51.025552+0000","last_undegraded":"2023-09-12T16:02:51.025552+0000","last_fullsized":"2023-09-12T16:02:51.025552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:15:14.695521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831296+0000","last_change":"2023-09-12T16:02:04.033213+0000","last_active":"2023-09-12T16:02:50.831296+0000","last_peered":"2023-09-12T16:02:50.831296+0000","last_clean":"2023-09-12T16:02:50.831296+0000","last_became_active":"2023-09-12T16:02:04.032978+0000","last_became_peered":"2023-09-12T16:02:04.032978+0000","last_unstale":"2023-09-12T16:02:50.831296+0000","last_undegraded":"2023-09-12T16:02:50.831296+0000","last_fullsized":"2023-09-12T16:02:50.831296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:25:33.718954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835250+0000","last_change":"2023-09-12T16:02:05.043252+0000","last_active":"2023-09-12T16:02:50.835250+0000","last_peered":"2023-09-12T16:02:50.835250+0000","last_clean":"2023-09-12T16:02:50.835250+0000","last_became_active":"2023-09-12T16:02:05.042975+0000","last_became_peered":"2023-09-12T16:02:05.042975+0000","last_unstale":"2023-09-12T16:02:50.835250+0000","last_undegraded":"2023-09-12T16:02:50.835250+0000","last_fullsized":"2023-09-12T16:02:50.835250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:46:31.549547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831861+0000","last_change":"2023-09-12T16:02:09.841728+0000","last_active":"2023-09-12T16:02:50.831861+0000","last_peered":"2023-09-12T16:02:50.831861+0000","last_clean":"2023-09-12T16:02:50.831861+0000","last_became_active":"2023-09-12T16:02:09.841351+0000","last_became_peered":"2023-09-12T16:02:09.841351+0000","last_unstale":"2023-09-12T16:02:50.831861+0000","last_undegraded":"2023-09-12T16:02:50.831861+0000","last_fullsized":"2023-09-12T16:02:50.831861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:00:50.637272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833531+0000","last_change":"2023-09-12T16:02:04.025624+0000","last_active":"2023-09-12T16:02:50.833531+0000","last_peered":"2023-09-12T16:02:50.833531+0000","last_clean":"2023-09-12T16:02:50.833531+0000","last_became_active":"2023-09-12T16:02:04.025330+0000","last_became_peered":"2023-09-12T16:02:04.025330+0000","last_unstale":"2023-09-12T16:02:50.833531+0000","last_undegraded":"2023-09-12T16:02:50.833531+0000","last_fullsized":"2023-09-12T16:02:50.833531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:00:57.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833174+0000","last_change":"2023-09-12T16:02:05.028414+0000","last_active":"2023-09-12T16:02:50.833174+0000","last_peered":"2023-09-12T16:02:50.833174+0000","last_clean":"2023-09-12T16:02:50.833174+0000","last_became_active":"2023-09-12T16:02:05.028081+0000","last_became_peered":"2023-09-12T16:02:05.028081+0000","last_unstale":"2023-09-12T16:02:50.833174+0000","last_undegraded":"2023-09-12T16:02:50.833174+0000","last_fullsized":"2023-09-12T16:02:50.833174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:57:58.245105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834801+0000","last_change":"2023-09-12T16:02:09.283222+0000","last_active":"2023-09-12T16:02:50.834801+0000","last_peered":"2023-09-12T16:02:50.834801+0000","last_clean":"2023-09-12T16:02:50.834801+0000","last_became_active":"2023-09-12T16:02:09.282814+0000","last_became_peered":"2023-09-12T16:02:09.282814+0000","last_unstale":"2023-09-12T16:02:50.834801+0000","last_undegraded":"2023-09-12T16:02:50.834801+0000","last_fullsized":"2023-09-12T16:02:50.834801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:18.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836561+0000","last_change":"2023-09-12T16:02:04.026694+0000","last_active":"2023-09-12T16:02:50.836561+0000","last_peered":"2023-09-12T16:02:50.836561+0000","last_clean":"2023-09-12T16:02:50.836561+0000","last_became_active":"2023-09-12T16:02:04.026368+0000","last_became_peered":"2023-09-12T16:02:04.026368+0000","last_unstale":"2023-09-12T16:02:50.836561+0000","last_undegraded":"2023-09-12T16:02:50.836561+0000","last_fullsized":"2023-09-12T16:02:50.836561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:14:10.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829216+0000","last_change":"2023-09-12T16:02:05.039037+0000","last_active":"2023-09-12T16:02:50.829216+0000","last_peered":"2023-09-12T16:02:50.829216+0000","last_clean":"2023-09-12T16:02:50.829216+0000","last_became_active":"2023-09-12T16:02:05.038708+0000","last_became_peered":"2023-09-12T16:02:05.038708+0000","last_unstale":"2023-09-12T16:02:50.829216+0000","last_undegraded":"2023-09-12T16:02:50.829216+0000","last_fullsized":"2023-09-12T16:02:50.829216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:13:09.805646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836160+0000","last_change":"2023-09-12T16:02:09.841757+0000","last_active":"2023-09-12T16:02:50.836160+0000","last_peered":"2023-09-12T16:02:50.836160+0000","last_clean":"2023-09-12T16:02:50.836160+0000","last_became_active":"2023-09-12T16:02:09.841173+0000","last_became_peered":"2023-09-12T16:02:09.841173+0000","last_unstale":"2023-09-12T16:02:50.836160+0000","last_undegraded":"2023-09-12T16:02:50.836160+0000","last_fullsized":"2023-09-12T16:02:50.836160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:57:06.626828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836147+0000","last_change":"2023-09-12T16:02:04.031009+0000","last_active":"2023-09-12T16:02:50.836147+0000","last_peered":"2023-09-12T16:02:50.836147+0000","last_clean":"2023-09-12T16:02:50.836147+0000","last_became_active":"2023-09-12T16:02:04.030714+0000","last_became_peered":"2023-09-12T16:02:04.030714+0000","last_unstale":"2023-09-12T16:02:50.836147+0000","last_undegraded":"2023-09-12T16:02:50.836147+0000","last_fullsized":"2023-09-12T16:02:50.836147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:12.578686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830789+0000","last_change":"2023-09-12T16:02:05.030778+0000","last_active":"2023-09-12T16:02:50.830789+0000","last_peered":"2023-09-12T16:02:50.830789+0000","last_clean":"2023-09-12T16:02:50.830789+0000","last_became_active":"2023-09-12T16:02:05.030454+0000","last_became_peered":"2023-09-12T16:02:05.030454+0000","last_unstale":"2023-09-12T16:02:50.830789+0000","last_undegraded":"2023-09-12T16:02:50.830789+0000","last_fullsized":"2023-09-12T16:02:50.830789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:52.951246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834030+0000","last_change":"2023-09-12T16:02:09.289668+0000","last_active":"2023-09-12T16:02:50.834030+0000","last_peered":"2023-09-12T16:02:50.834030+0000","last_clean":"2023-09-12T16:02:50.834030+0000","last_became_active":"2023-09-12T16:02:09.289301+0000","last_became_peered":"2023-09-12T16:02:09.289301+0000","last_unstale":"2023-09-12T16:02:50.834030+0000","last_undegraded":"2023-09-12T16:02:50.834030+0000","last_fullsized":"2023-09-12T16:02:50.834030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:20:04.528146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832522+0000","last_change":"2023-09-12T16:02:04.036175+0000","last_active":"2023-09-12T16:02:50.832522+0000","last_peered":"2023-09-12T16:02:50.832522+0000","last_clean":"2023-09-12T16:02:50.832522+0000","last_became_active":"2023-09-12T16:02:04.035879+0000","last_became_peered":"2023-09-12T16:02:04.035879+0000","last_unstale":"2023-09-12T16:02:50.832522+0000","last_undegraded":"2023-09-12T16:02:50.832522+0000","last_fullsized":"2023-09-12T16:02:50.832522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:29:20.798716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025379+0000","last_change":"2023-09-12T16:02:05.034195+0000","last_active":"2023-09-12T16:02:51.025379+0000","last_peered":"2023-09-12T16:02:51.025379+0000","last_clean":"2023-09-12T16:02:51.025379+0000","last_became_active":"2023-09-12T16:02:05.033534+0000","last_became_peered":"2023-09-12T16:02:05.033534+0000","last_unstale":"2023-09-12T16:02:51.025379+0000","last_undegraded":"2023-09-12T16:02:51.025379+0000","last_fullsized":"2023-09-12T16:02:51.025379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:34:04.180218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834899+0000","last_change":"2023-09-12T16:02:09.287543+0000","last_active":"2023-09-12T16:02:50.834899+0000","last_peered":"2023-09-12T16:02:50.834899+0000","last_clean":"2023-09-12T16:02:50.834899+0000","last_became_active":"2023-09-12T16:02:09.287055+0000","last_became_peered":"2023-09-12T16:02:09.287055+0000","last_unstale":"2023-09-12T16:02:50.834899+0000","last_undegraded":"2023-09-12T16:02:50.834899+0000","last_fullsized":"2023-09-12T16:02:50.834899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:58.214381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834994+0000","last_change":"2023-09-12T16:02:04.030206+0000","last_active":"2023-09-12T16:02:50.834994+0000","last_peered":"2023-09-12T16:02:50.834994+0000","last_clean":"2023-09-12T16:02:50.834994+0000","last_became_active":"2023-09-12T16:02:04.029765+0000","last_became_peered":"2023-09-12T16:02:04.029765+0000","last_unstale":"2023-09-12T16:02:50.834994+0000","last_undegraded":"2023-09-12T16:02:50.834994+0000","last_fullsized":"2023-09-12T16:02:50.834994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:56:07.551681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832663+0000","last_change":"2023-09-12T16:02:05.028300+0000","last_active":"2023-09-12T16:02:50.832663+0000","last_peered":"2023-09-12T16:02:50.832663+0000","last_clean":"2023-09-12T16:02:50.832663+0000","last_became_active":"2023-09-12T16:02:05.027970+0000","last_became_peered":"2023-09-12T16:02:05.027970+0000","last_unstale":"2023-09-12T16:02:50.832663+0000","last_undegraded":"2023-09-12T16:02:50.832663+0000","last_fullsized":"2023-09-12T16:02:50.832663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:39:44.571593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835970+0000","last_change":"2023-09-12T16:02:09.841661+0000","last_active":"2023-09-12T16:02:50.835970+0000","last_peered":"2023-09-12T16:02:50.835970+0000","last_clean":"2023-09-12T16:02:50.835970+0000","last_became_active":"2023-09-12T16:02:09.841039+0000","last_became_peered":"2023-09-12T16:02:09.841039+0000","last_unstale":"2023-09-12T16:02:50.835970+0000","last_undegraded":"2023-09-12T16:02:50.835970+0000","last_fullsized":"2023-09-12T16:02:50.835970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:34.529098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830867+0000","last_change":"2023-09-12T16:02:04.031927+0000","last_active":"2023-09-12T16:02:50.830867+0000","last_peered":"2023-09-12T16:02:50.830867+0000","last_clean":"2023-09-12T16:02:50.830867+0000","last_became_active":"2023-09-12T16:02:04.031614+0000","last_became_peered":"2023-09-12T16:02:04.031614+0000","last_unstale":"2023-09-12T16:02:50.830867+0000","last_undegraded":"2023-09-12T16:02:50.830867+0000","last_fullsized":"2023-09-12T16:02:50.830867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:00:23.423933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829483+0000","last_change":"2023-09-12T16:02:05.032647+0000","last_active":"2023-09-12T16:02:50.829483+0000","last_peered":"2023-09-12T16:02:50.829483+0000","last_clean":"2023-09-12T16:02:50.829483+0000","last_became_active":"2023-09-12T16:02:05.031701+0000","last_became_peered":"2023-09-12T16:02:05.031701+0000","last_unstale":"2023-09-12T16:02:50.829483+0000","last_undegraded":"2023-09-12T16:02:50.829483+0000","last_fullsized":"2023-09-12T16:02:50.829483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:02:19.596913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837136+0000","last_change":"2023-09-12T16:02:09.291043+0000","last_active":"2023-09-12T16:02:50.837136+0000","last_peered":"2023-09-12T16:02:50.837136+0000","last_clean":"2023-09-12T16:02:50.837136+0000","last_became_active":"2023-09-12T16:02:09.290666+0000","last_became_peered":"2023-09-12T16:02:09.290666+0000","last_unstale":"2023-09-12T16:02:50.837136+0000","last_undegraded":"2023-09-12T16:02:50.837136+0000","last_fullsized":"2023-09-12T16:02:50.837136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:10:47.354479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835888+0000","last_change":"2023-09-12T16:02:04.030501+0000","last_active":"2023-09-12T16:02:50.835888+0000","last_peered":"2023-09-12T16:02:50.835888+0000","last_clean":"2023-09-12T16:02:50.835888+0000","last_became_active":"2023-09-12T16:02:04.030196+0000","last_became_peered":"2023-09-12T16:02:04.030196+0000","last_unstale":"2023-09-12T16:02:50.835888+0000","last_undegraded":"2023-09-12T16:02:50.835888+0000","last_fullsized":"2023-09-12T16:02:50.835888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:58:34.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830617+0000","last_change":"2023-09-12T16:02:05.030466+0000","last_active":"2023-09-12T16:02:50.830617+0000","last_peered":"2023-09-12T16:02:50.830617+0000","last_clean":"2023-09-12T16:02:50.830617+0000","last_became_active":"2023-09-12T16:02:05.029988+0000","last_became_peered":"2023-09-12T16:02:05.029988+0000","last_unstale":"2023-09-12T16:02:50.830617+0000","last_undegraded":"2023-09-12T16:02:50.830617+0000","last_fullsized":"2023-09-12T16:02:50.830617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:08:56.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846403+0000","last_change":"2023-09-12T16:02:51.846403+0000","last_active":"2023-09-12T16:02:51.846403+0000","last_peered":"2023-09-12T16:02:51.846403+0000","last_clean":"2023-09-12T16:02:51.846403+0000","last_became_active":"2023-09-12T16:02:51.846047+0000","last_became_peered":"2023-09-12T16:02:51.846047+0000","last_unstale":"2023-09-12T16:02:51.846403+0000","last_undegraded":"2023-09-12T16:02:51.846403+0000","last_fullsized":"2023-09-12T16:02:51.846403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:58:11.458339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832945+0000","last_change":"2023-09-12T16:02:04.027380+0000","last_active":"2023-09-12T16:02:50.832945+0000","last_peered":"2023-09-12T16:02:50.832945+0000","last_clean":"2023-09-12T16:02:50.832945+0000","last_became_active":"2023-09-12T16:02:04.027098+0000","last_became_peered":"2023-09-12T16:02:04.027098+0000","last_unstale":"2023-09-12T16:02:50.832945+0000","last_undegraded":"2023-09-12T16:02:50.832945+0000","last_fullsized":"2023-09-12T16:02:50.832945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:34:12.260747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829993+0000","last_change":"2023-09-12T16:02:05.029249+0000","last_active":"2023-09-12T16:02:50.829993+0000","last_peered":"2023-09-12T16:02:50.829993+0000","last_clean":"2023-09-12T16:02:50.829993+0000","last_became_active":"2023-09-12T16:02:05.028955+0000","last_became_peered":"2023-09-12T16:02:05.028955+0000","last_unstale":"2023-09-12T16:02:50.829993+0000","last_undegraded":"2023-09-12T16:02:50.829993+0000","last_fullsized":"2023-09-12T16:02:50.829993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:02:10.382712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831923+0000","last_change":"2023-09-12T16:02:09.288724+0000","last_active":"2023-09-12T16:02:50.831923+0000","last_peered":"2023-09-12T16:02:50.831923+0000","last_clean":"2023-09-12T16:02:50.831923+0000","last_became_active":"2023-09-12T16:02:09.288206+0000","last_became_peered":"2023-09-12T16:02:09.288206+0000","last_unstale":"2023-09-12T16:02:50.831923+0000","last_undegraded":"2023-09-12T16:02:50.831923+0000","last_fullsized":"2023-09-12T16:02:50.831923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:45:10.154195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835225+0000","last_change":"2023-09-12T16:02:04.031219+0000","last_active":"2023-09-12T16:02:50.835225+0000","last_peered":"2023-09-12T16:02:50.835225+0000","last_clean":"2023-09-12T16:02:50.835225+0000","last_became_active":"2023-09-12T16:02:04.030867+0000","last_became_peered":"2023-09-12T16:02:04.030867+0000","last_unstale":"2023-09-12T16:02:50.835225+0000","last_undegraded":"2023-09-12T16:02:50.835225+0000","last_fullsized":"2023-09-12T16:02:50.835225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:16.885094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831081+0000","last_change":"2023-09-12T16:02:05.028004+0000","last_active":"2023-09-12T16:02:50.831081+0000","last_peered":"2023-09-12T16:02:50.831081+0000","last_clean":"2023-09-12T16:02:50.831081+0000","last_became_active":"2023-09-12T16:02:05.027668+0000","last_became_peered":"2023-09-12T16:02:05.027668+0000","last_unstale":"2023-09-12T16:02:50.831081+0000","last_undegraded":"2023-09-12T16:02:50.831081+0000","last_fullsized":"2023-09-12T16:02:50.831081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:41:23.019243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025479+0000","last_change":"2023-09-12T16:02:05.034380+0000","last_active":"2023-09-12T16:02:51.025479+0000","last_peered":"2023-09-12T16:02:51.025479+0000","last_clean":"2023-09-12T16:02:51.025479+0000","last_became_active":"2023-09-12T16:02:05.034022+0000","last_became_peered":"2023-09-12T16:02:05.034022+0000","last_unstale":"2023-09-12T16:02:51.025479+0000","last_undegraded":"2023-09-12T16:02:51.025479+0000","last_fullsized":"2023-09-12T16:02:51.025479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:55:56.404742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835155+0000","last_change":"2023-09-12T16:02:05.031723+0000","last_active":"2023-09-12T16:02:50.835155+0000","last_peered":"2023-09-12T16:02:50.835155+0000","last_clean":"2023-09-12T16:02:50.835155+0000","last_became_active":"2023-09-12T16:02:05.030954+0000","last_became_peered":"2023-09-12T16:02:05.030954+0000","last_unstale":"2023-09-12T16:02:50.835155+0000","last_undegraded":"2023-09-12T16:02:50.835155+0000","last_fullsized":"2023-09-12T16:02:50.835155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.319954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834938+0000","last_change":"2023-09-12T16:02:05.034851+0000","last_active":"2023-09-12T16:02:50.834938+0000","last_peered":"2023-09-12T16:02:50.834938+0000","last_clean":"2023-09-12T16:02:50.834938+0000","last_became_active":"2023-09-12T16:02:05.030733+0000","last_became_peered":"2023-09-12T16:02:05.030733+0000","last_unstale":"2023-09-12T16:02:50.834938+0000","last_undegraded":"2023-09-12T16:02:50.834938+0000","last_fullsized":"2023-09-12T16:02:50.834938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:33.605216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830111+0000","last_change":"2023-09-12T16:02:05.037331+0000","last_active":"2023-09-12T16:02:50.830111+0000","last_peered":"2023-09-12T16:02:50.830111+0000","last_clean":"2023-09-12T16:02:50.830111+0000","last_became_active":"2023-09-12T16:02:05.037015+0000","last_became_peered":"2023-09-12T16:02:05.037015+0000","last_unstale":"2023-09-12T16:02:50.830111+0000","last_undegraded":"2023-09-12T16:02:50.830111+0000","last_fullsized":"2023-09-12T16:02:50.830111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:29:14.035891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835319+0000","last_change":"2023-09-12T16:02:05.039803+0000","last_active":"2023-09-12T16:02:50.835319+0000","last_peered":"2023-09-12T16:02:50.835319+0000","last_clean":"2023-09-12T16:02:50.835319+0000","last_became_active":"2023-09-12T16:02:05.039390+0000","last_became_peered":"2023-09-12T16:02:05.039390+0000","last_unstale":"2023-09-12T16:02:50.835319+0000","last_undegraded":"2023-09-12T16:02:50.835319+0000","last_fullsized":"2023-09-12T16:02:50.835319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:22.422090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025700+0000","last_change":"2023-09-12T16:02:05.033153+0000","last_active":"2023-09-12T16:02:51.025700+0000","last_peered":"2023-09-12T16:02:51.025700+0000","last_clean":"2023-09-12T16:02:51.025700+0000","last_became_active":"2023-09-12T16:02:05.032800+0000","last_became_peered":"2023-09-12T16:02:05.032800+0000","last_unstale":"2023-09-12T16:02:51.025700+0000","last_undegraded":"2023-09-12T16:02:51.025700+0000","last_fullsized":"2023-09-12T16:02:51.025700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:31:43.090031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836428+0000","last_change":"2023-09-12T16:02:05.031519+0000","last_active":"2023-09-12T16:02:50.836428+0000","last_peered":"2023-09-12T16:02:50.836428+0000","last_clean":"2023-09-12T16:02:50.836428+0000","last_became_active":"2023-09-12T16:02:05.030227+0000","last_became_peered":"2023-09-12T16:02:05.030227+0000","last_unstale":"2023-09-12T16:02:50.836428+0000","last_undegraded":"2023-09-12T16:02:50.836428+0000","last_fullsized":"2023-09-12T16:02:50.836428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:00.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833413+0000","last_change":"2023-09-12T16:02:05.031763+0000","last_active":"2023-09-12T16:02:50.833413+0000","last_peered":"2023-09-12T16:02:50.833413+0000","last_clean":"2023-09-12T16:02:50.833413+0000","last_became_active":"2023-09-12T16:02:05.031441+0000","last_became_peered":"2023-09-12T16:02:05.031441+0000","last_unstale":"2023-09-12T16:02:50.833413+0000","last_undegraded":"2023-09-12T16:02:50.833413+0000","last_fullsized":"2023-09-12T16:02:50.833413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:55.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832249+0000","last_change":"2023-09-12T16:02:05.033786+0000","last_active":"2023-09-12T16:02:50.832249+0000","last_peered":"2023-09-12T16:02:50.832249+0000","last_clean":"2023-09-12T16:02:50.832249+0000","last_became_active":"2023-09-12T16:02:05.033497+0000","last_became_peered":"2023-09-12T16:02:05.033497+0000","last_unstale":"2023-09-12T16:02:50.832249+0000","last_undegraded":"2023-09-12T16:02:50.832249+0000","last_fullsized":"2023-09-12T16:02:50.832249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:01:58.912488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832560+0000","last_change":"2023-09-12T16:02:05.031144+0000","last_active":"2023-09-12T16:02:50.832560+0000","last_peered":"2023-09-12T16:02:50.832560+0000","last_clean":"2023-09-12T16:02:50.832560+0000","last_became_active":"2023-09-12T16:02:05.030827+0000","last_became_peered":"2023-09-12T16:02:05.030827+0000","last_unstale":"2023-09-12T16:02:50.832560+0000","last_undegraded":"2023-09-12T16:02:50.832560+0000","last_fullsized":"2023-09-12T16:02:50.832560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:40:48.014344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837378+0000","last_change":"2023-09-12T16:02:05.021273+0000","last_active":"2023-09-12T16:02:50.837378+0000","last_peered":"2023-09-12T16:02:50.837378+0000","last_clean":"2023-09-12T16:02:50.837378+0000","last_became_active":"2023-09-12T16:02:05.020953+0000","last_became_peered":"2023-09-12T16:02:05.020953+0000","last_unstale":"2023-09-12T16:02:50.837378+0000","last_undegraded":"2023-09-12T16:02:50.837378+0000","last_fullsized":"2023-09-12T16:02:50.837378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:33:25.462412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835081+0000","last_change":"2023-09-12T16:02:05.039019+0000","last_active":"2023-09-12T16:02:50.835081+0000","last_peered":"2023-09-12T16:02:50.835081+0000","last_clean":"2023-09-12T16:02:50.835081+0000","last_became_active":"2023-09-12T16:02:05.038621+0000","last_became_peered":"2023-09-12T16:02:05.038621+0000","last_unstale":"2023-09-12T16:02:50.835081+0000","last_undegraded":"2023-09-12T16:02:50.835081+0000","last_fullsized":"2023-09-12T16:02:50.835081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:08:08.934375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025761+0000","last_change":"2023-09-12T16:02:05.039737+0000","last_active":"2023-09-12T16:02:51.025761+0000","last_peered":"2023-09-12T16:02:51.025761+0000","last_clean":"2023-09-12T16:02:51.025761+0000","last_became_active":"2023-09-12T16:02:05.039304+0000","last_became_peered":"2023-09-12T16:02:05.039304+0000","last_unstale":"2023-09-12T16:02:51.025761+0000","last_undegraded":"2023-09-12T16:02:51.025761+0000","last_fullsized":"2023-09-12T16:02:51.025761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:50:13.141915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244717,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69913,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":489,"num_read_kb":318,"num_write":158,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":59815,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":201417,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":489432,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.876}]},{"osd":1,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.755,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80600000000000005}]},{"osd":2,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.5,"5min":0.496,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77600000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78200000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80400000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.85699999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.54900000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89900000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71899999999999997,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.84799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.89100000000000001}]},{"osd":8,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75900000000000001}]},{"osd":9,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.65200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.621,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.84199999999999997}]},{"osd":10,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.61399999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.98199999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.93000000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776163,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":1142931,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.5,"15min":0.501},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.503},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80000000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.752},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82899999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.58999999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.90800000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.79300000000000004}]},{"osd":6,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95299999999999996,"5min":1.042,"15min":1.042},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.56499999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.85099999999999998,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.80700000000000005}]},{"osd":8,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.63700000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.008,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.875},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93400000000000005,"5min":1.073,"15min":1.073},"last":0.86599999999999999}]},{"osd":9,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96299999999999997,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.099,"15min":1.099},"last":0.81999999999999995}]},{"osd":11,"last update":"Tue Sep 12 16:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.79100000000000004,"15min":0.78400000000000003},"min":{"1min":0.68899999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.66000000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.89500000000000002}]}]},{"osd":9,"up_from":63,"seq":270582939688,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":493764,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.48599999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.42799999999999999,"15min":0.436},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.82499999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.372}]},{"osd":1,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.312}]},{"osd":2,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70499999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.46500000000000002,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.72099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81799999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996}]},{"osd":6,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.495},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88700000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.41799999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.70099999999999996,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.80800000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.621}]},{"osd":10,"last update":"Tue Sep 12 16:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.69999999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46000000000000002}]},{"osd":11,"last update":"Tue Sep 12 16:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61199999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168620,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":488675,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66100000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55100000000000005}]},{"osd":1,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.63,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.57399999999999995,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71499999999999997}]},{"osd":2,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.44400000000000001},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.48399999999999999,"15min":0.45400000000000001},"min":{"1min":0.34799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.47399999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":2.5710000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.95999999999999996}]},{"osd":4,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68400000000000005,"15min":0.65500000000000003},"min":{"1min":0.58199999999999996,"5min":0.373,"15min":0.373},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.54000000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.012,"15min":1.012},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71399999999999997,"15min":0.69099999999999995},"min":{"1min":0.50600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0469999999999999,"5min":1.075,"15min":1.075},"last":0.80500000000000005}]},{"osd":7,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.76300000000000001,"15min":0.73599999999999999},"min":{"1min":0.69199999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.872},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.56499999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.75800000000000001}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.84799999999999998}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71899999999999997,"15min":0.70299999999999996},"min":{"1min":0.60199999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.67500000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92600000000000005}]},{"osd":11,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.88600000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1085873,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.40200000000000002,"15min":0.38600000000000001},"min":{"1min":0.34200000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63600000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.433,"15min":0.40699999999999997},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.505}]},{"osd":2,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59099999999999997,"15min":0.55800000000000005},"min":{"1min":0.439,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1639999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0640000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.78200000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.623}]},{"osd":4,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.91200000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.35099999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72599999999999998,"5min":1.069,"15min":1.069},"last":0.749}]},{"osd":5,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65100000000000002,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93999999999999995,"5min":1.909,"15min":1.909},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65000000000000002,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.139,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.90500000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96299999999999997,"5min":1.119,"15min":1.119},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84699999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.85099999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0169999999999999,"5min":1.879,"15min":1.879},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98499999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.998}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96499999999999997,"5min":1.329,"15min":1.329},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.0309999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.59799999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.872}]},{"osd":11,"last update":"Tue Sep 12 16:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.97599999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":484844,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.39200000000000002,"15min":0.39000000000000001},"min":{"1min":0.25600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56999999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.35899999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70099999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.69799999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71899999999999997,"5min":1.079,"15min":1.079},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":1.129,"15min":1.129},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.75,"5min":1.083,"15min":1.083},"last":0.439},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.68300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72699999999999998,"5min":1.161,"15min":1.161},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67800000000000005}]},{"osd":5,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81999999999999995,"5min":1.022,"15min":1.022},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.65300000000000002}]},{"osd":8,"last update":"Tue Sep 12 16:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68400000000000005,"15min":0.70399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83399999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.628}]},{"osd":9,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71799999999999997,"15min":0.72899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.495}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90800000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60499999999999998}]},{"osd":11,"last update":"Tue Sep 12 16:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":545948,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.38100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":6.3369999999999997}]},{"osd":2,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.61099999999999999,"15min":0.54200000000000004},"min":{"1min":0.55000000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.628}]},{"osd":3,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57099999999999995,"15min":0.51900000000000002},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":1.002,"15min":1.002},"last":6.5170000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.498,"15min":0.45600000000000002},"min":{"1min":0.40899999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.435},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.57499999999999996,"15min":0.51900000000000002},"min":{"1min":0.52400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.69999999999999996,"15min":0.65200000000000002},"min":{"1min":0.52100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.036,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.66900000000000004,"15min":0.623},"min":{"1min":0.63,"5min":0.317,"15min":0.317},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.68100000000000005,"15min":0.63700000000000001},"min":{"1min":0.70499999999999996,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60099999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.77300000000000002,"15min":0.73699999999999999},"min":{"1min":0.59599999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1399999999999999,"5min":1.167,"15min":1.167},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69499999999999995,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73299999999999998}]},{"osd":9,"last update":"Tue Sep 12 16:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81000000000000005,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.81999999999999995,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.88900000000000001}]},{"osd":10,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.67300000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77000000000000002}]},{"osd":11,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.81499999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":486232,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58099999999999996,"15min":0.60599999999999998},"min":{"1min":0.38300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82999999999999996,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.48699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":2.113,"15min":2.113},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.52100000000000002},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79300000000000004,"5min":0.874,"15min":0.874},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76600000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.441}]},{"osd":3,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.308},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.33100000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76600000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.74399999999999999,"5min":0.879,"15min":0.879},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":1.004,"15min":1.004},"last":0.41699999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59699999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.53500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.66000000000000003,"15min":0.67600000000000005},"min":{"1min":0.47099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.501}]},{"osd":9,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58099999999999996}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.028,"15min":1.028},"last":0.65700000000000003}]},{"osd":11,"last update":"Tue Sep 12 16:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.59599999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485552,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.377,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79600000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.437},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.499,"15min":0.47899999999999998},"min":{"1min":0.42299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.57999999999999996,"15min":0.51200000000000001},"min":{"1min":0.432,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90600000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.61299999999999999,"15min":0.52400000000000002},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.878}]},{"osd":3,"last update":"Tue Sep 12 16:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.89700000000000002}]},{"osd":5,"last update":"Tue Sep 12 16:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.53300000000000003,"15min":0.48699999999999999},"min":{"1min":0.35399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.875,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.56200000000000006,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85099999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:03:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73599999999999999}]},{"osd":8,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":1.05,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65100000000000002,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.747}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.81399999999999995}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.60799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.747,"15min":0.72099999999999997},"min":{"1min":0.64400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.95599999999999996}]},{"osd":11,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.93100000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485038,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.34999999999999998},"min":{"1min":0.27000000000000002,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.167,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.26700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.317}]},{"osd":1,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82299999999999995,"5min":3.552,"15min":3.552},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.752,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56599999999999995,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86399999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76300000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.44500000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78100000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72799999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56699999999999995}]},{"osd":5,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82899999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.46999999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47599999999999998},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65000000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.36599999999999999}]},{"osd":7,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.71599999999999997,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":7.6929999999999996,"5min":7.6929999999999996,"15min":7.6929999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.50900000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66100000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.083,"15min":1.083},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.75,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.45400000000000001}]},{"osd":10,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.84899999999999998,"5min":0.875,"15min":0.875},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53900000000000003}]},{"osd":11,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58899999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":550496,"data_compressed":30118,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.46999999999999997},"min":{"1min":0.38500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.45800000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88800000000000001,"5min":1.458,"15min":1.458},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.36299999999999999},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.41799999999999998,"15min":0.374},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.436}]},{"osd":2,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58199999999999996,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.316,"15min":0.316},"max":{"1min":1.0249999999999999,"5min":1.383,"15min":1.383},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52900000000000003,"15min":0.49399999999999999},"min":{"1min":0.38600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41199999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.66300000000000003,"15min":0.626},"min":{"1min":0.55100000000000005,"5min":0.379,"15min":0.379},"max":{"1min":0.92100000000000004,"5min":1.54,"15min":1.54},"last":0.752},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.52600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90800000000000003,"5min":1.502,"15min":1.502},"last":0.84599999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.496,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93999999999999995,"5min":1.639,"15min":1.639},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53400000000000003}]},{"osd":8,"last update":"Tue Sep 12 16:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.58799999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.80800000000000005,"15min":0.79600000000000004},"min":{"1min":0.67400000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.018,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.87}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97499999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.622}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.755,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.93000000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1,"5min":1,"15min":1},"last":0.79100000000000004}]},{"osd":11,"last update":"Tue Sep 12 16:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.82299999999999995}]}]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1080753,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.48099999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.376},"min":{"1min":0.29599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.38},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.32400000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40300000000000002}]},{"osd":3,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53000000000000003,"15min":0.54700000000000004},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.495},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.41499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.47399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.433,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81200000000000006}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.62},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.879,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.57499999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72099999999999997}]},{"osd":8,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48899999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58699999999999997}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.752}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.56899999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":61919,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T16:03:48.408 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-09-12T16:03:48.408 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T16:03:48.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 -- 172.21.15.130:0/2948792003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 msgr2=0x7fe290100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 --2- 172.21.15.130:0/2948792003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 0x7fe290100b30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0099c0 tx=0x7fe28c009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.562 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 -- 172.21.15.130:0/2948792003 shutdown_connections 2023-09-12T16:03:48.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 --2- 172.21.15.130:0/2948792003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe290101070 0x7fe290107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 --2- 172.21.15.130:0/2948792003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2900ff4e0 0x7fe2900ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 --2- 172.21.15.130:0/2948792003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 0x7fe290100b30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 -- 172.21.15.130:0/2948792003 >> 172.21.15.130:0/2948792003 conn(0x7fe2900faa30 msgr2=0x7fe2900fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:48.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 -- 172.21.15.130:0/2948792003 shutdown_connections 2023-09-12T16:03:48.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 -- 172.21.15.130:0/2948792003 wait complete. 2023-09-12T16:03:48.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.559+0000 7fe298552700 1 Processor -- start 2023-09-12T16:03:48.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe298552700 1 -- start start 2023-09-12T16:03:48.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe298552700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2900ff4e0 0x7fe290195210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe298552700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 0x7fe290195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.565 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe298552700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe290101070 0x7fe29019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.565 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe298552700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29019cd90 con 0x7fe2901006d0 2023-09-12T16:03:48.565 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe298552700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29019ced0 con 0x7fe290101070 2023-09-12T16:03:48.565 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe298552700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29019d060 con 0x7fe2900ff4e0 2023-09-12T16:03:48.565 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe295aed700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 0x7fe290195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.566 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe295aed700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 0x7fe290195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55214/0 (socket says 172.21.15.130:55214) 2023-09-12T16:03:48.566 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe295aed700 1 -- 172.21.15.130:0/2103074665 learned_addr learned my addr 172.21.15.130:0/2103074665 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:48.566 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe2962ee700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2900ff4e0 0x7fe290195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.566 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe296aef700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe290101070 0x7fe29019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.566 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe295aed700 1 -- 172.21.15.130:0/2103074665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2900ff4e0 msgr2=0x7fe290195210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.567 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe295aed700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2900ff4e0 0x7fe290195210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.567 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe295aed700 1 -- 172.21.15.130:0/2103074665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe290101070 msgr2=0x7fe29019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.567 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe295aed700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe290101070 0x7fe29019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.567 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe295aed700 1 -- 172.21.15.130:0/2103074665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0095f0 con 0x7fe2901006d0 2023-09-12T16:03:48.567 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe296aef700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe290101070 0x7fe29019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:48.568 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe295aed700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 0x7fe290195750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009990 tx=0x7fe28c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:48.568 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe2877fe700 1 -- 172.21.15.130:0/2103074665 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c01d070 con 0x7fe2901006d0 2023-09-12T16:03:48.568 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe2877fe700 1 -- 172.21.15.130:0/2103074665 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe28c0054f0 con 0x7fe2901006d0 2023-09-12T16:03:48.568 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29019d2e0 con 0x7fe2901006d0 2023-09-12T16:03:48.568 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.560+0000 7fe2877fe700 1 -- 172.21.15.130:0/2103074665 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c004bd0 con 0x7fe2901006d0 2023-09-12T16:03:48.569 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.561+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29019d700 con 0x7fe2901006d0 2023-09-12T16:03:48.569 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.562+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe290106aa0 con 0x7fe2901006d0 2023-09-12T16:03:48.570 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.563+0000 7fe2877fe700 1 -- 172.21.15.130:0/2103074665 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fe28c00fb80 con 0x7fe2901006d0 2023-09-12T16:03:48.570 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.563+0000 7fe2877fe700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe27c0a0710 0x7fe27c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.570 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.563+0000 7fe2962ee700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe27c0a0710 0x7fe27c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.570 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.563+0000 7fe2877fe700 1 -- 172.21.15.130:0/2103074665 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe28c0cd2e0 con 0x7fe2901006d0 2023-09-12T16:03:48.571 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.564+0000 7fe2962ee700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe27c0a0710 0x7fe27c0a2bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe290100530 tx=0x7fe280009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:48.571 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.567+0000 7fe2877fe700 1 -- 172.21.15.130:0/2103074665 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe28c096a30 con 0x7fe2901006d0 2023-09-12T16:03:48.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.755+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 --> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe29002d0b0 con 0x7fe27c0a0710 2023-09-12T16:03:48.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.767+0000 7fe2877fe700 1 -- 172.21.15.130:0/2103074665 <== mgr.14150 v2:172.21.15.130:6800/4273925772 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427369 (secure 0 0 0) 0x7fe29002d0b0 con 0x7fe27c0a0710 2023-09-12T16:03:48.772 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:48.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe27c0a0710 msgr2=0x7fe27c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe27c0a0710 0x7fe27c0a2bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe290100530 tx=0x7fe280009260 comp rx=0 tx=0).stop 2023-09-12T16:03:48.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 msgr2=0x7fe290195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 0x7fe290195750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009990 tx=0x7fe28c00f690 comp rx=0 tx=0).stop 2023-09-12T16:03:48.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 shutdown_connections 2023-09-12T16:03:48.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe27c0a0710 0x7fe27c0a2bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe290101070 0x7fe29019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2900ff4e0 0x7fe290195210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 --2- 172.21.15.130:0/2103074665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2901006d0 0x7fe290195750 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 >> 172.21.15.130:0/2103074665 conn(0x7fe2900faa30 msgr2=0x7fe290103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:48.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 shutdown_connections 2023-09-12T16:03:48.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.771+0000 7fe298552700 1 -- 172.21.15.130:0/2103074665 wait complete. 2023-09-12T16:03:48.778 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2023-09-12T16:03:48.784 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2023-09-12T16:03:46.782696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660527,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":615,"num_read_kb":427,"num_write":341,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1240551,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335964,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605860,"statfs":{"total":1151940427776,"available":1151596400640,"internally_reserved":0,"allocated":11980800,"data_stored":7819538,"data_compressed":363759,"data_compressed_allocated":491520,"data_compressed_original":2514944,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4638,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":46,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003026"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025907+0000","last_change":"2023-09-12T16:02:05.039617+0000","last_active":"2023-09-12T16:02:51.025907+0000","last_peered":"2023-09-12T16:02:51.025907+0000","last_clean":"2023-09-12T16:02:51.025907+0000","last_became_active":"2023-09-12T16:02:05.039264+0000","last_became_peered":"2023-09-12T16:02:05.039264+0000","last_unstale":"2023-09-12T16:02:51.025907+0000","last_undegraded":"2023-09-12T16:02:51.025907+0000","last_fullsized":"2023-09-12T16:02:51.025907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:37:01.381406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829291+0000","last_change":"2023-09-12T16:02:05.033151+0000","last_active":"2023-09-12T16:02:50.829291+0000","last_peered":"2023-09-12T16:02:50.829291+0000","last_clean":"2023-09-12T16:02:50.829291+0000","last_became_active":"2023-09-12T16:02:05.032866+0000","last_became_peered":"2023-09-12T16:02:05.032866+0000","last_unstale":"2023-09-12T16:02:50.829291+0000","last_undegraded":"2023-09-12T16:02:50.829291+0000","last_fullsized":"2023-09-12T16:02:50.829291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:35.921428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.032478+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.032124+0000","last_became_peered":"2023-09-12T16:02:05.032124+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:58:54.112577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835786+0000","last_change":"2023-09-12T16:02:05.033088+0000","last_active":"2023-09-12T16:02:50.835786+0000","last_peered":"2023-09-12T16:02:50.835786+0000","last_clean":"2023-09-12T16:02:50.835786+0000","last_became_active":"2023-09-12T16:02:05.032332+0000","last_became_peered":"2023-09-12T16:02:05.032332+0000","last_unstale":"2023-09-12T16:02:50.835786+0000","last_undegraded":"2023-09-12T16:02:50.835786+0000","last_fullsized":"2023-09-12T16:02:50.835786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:25:38.751961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:05.036612+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:05.036215+0000","last_became_peered":"2023-09-12T16:02:05.036215+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:15:04.561751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832126+0000","last_change":"2023-09-12T16:02:05.043816+0000","last_active":"2023-09-12T16:02:50.832126+0000","last_peered":"2023-09-12T16:02:50.832126+0000","last_clean":"2023-09-12T16:02:50.832126+0000","last_became_active":"2023-09-12T16:02:05.043599+0000","last_became_peered":"2023-09-12T16:02:05.043599+0000","last_unstale":"2023-09-12T16:02:50.832126+0000","last_undegraded":"2023-09-12T16:02:50.832126+0000","last_fullsized":"2023-09-12T16:02:50.832126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:32:00.286196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829345+0000","last_change":"2023-09-12T16:02:05.031476+0000","last_active":"2023-09-12T16:02:50.829345+0000","last_peered":"2023-09-12T16:02:50.829345+0000","last_clean":"2023-09-12T16:02:50.829345+0000","last_became_active":"2023-09-12T16:02:05.031150+0000","last_became_peered":"2023-09-12T16:02:05.031150+0000","last_unstale":"2023-09-12T16:02:50.829345+0000","last_undegraded":"2023-09-12T16:02:50.829345+0000","last_fullsized":"2023-09-12T16:02:50.829345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:22:27.405505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831825+0000","last_change":"2023-09-12T16:02:05.031981+0000","last_active":"2023-09-12T16:02:50.831825+0000","last_peered":"2023-09-12T16:02:50.831825+0000","last_clean":"2023-09-12T16:02:50.831825+0000","last_became_active":"2023-09-12T16:02:05.031692+0000","last_became_peered":"2023-09-12T16:02:05.031692+0000","last_unstale":"2023-09-12T16:02:50.831825+0000","last_undegraded":"2023-09-12T16:02:50.831825+0000","last_fullsized":"2023-09-12T16:02:50.831825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:51:00.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838117+0000","last_change":"2023-09-12T16:02:05.029898+0000","last_active":"2023-09-12T16:02:50.838117+0000","last_peered":"2023-09-12T16:02:50.838117+0000","last_clean":"2023-09-12T16:02:50.838117+0000","last_became_active":"2023-09-12T16:02:05.029731+0000","last_became_peered":"2023-09-12T16:02:05.029731+0000","last_unstale":"2023-09-12T16:02:50.838117+0000","last_undegraded":"2023-09-12T16:02:50.838117+0000","last_fullsized":"2023-09-12T16:02:50.838117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:16:31.194561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833228+0000","last_change":"2023-09-12T16:02:05.044846+0000","last_active":"2023-09-12T16:02:50.833228+0000","last_peered":"2023-09-12T16:02:50.833228+0000","last_clean":"2023-09-12T16:02:50.833228+0000","last_became_active":"2023-09-12T16:02:05.044430+0000","last_became_peered":"2023-09-12T16:02:05.044430+0000","last_unstale":"2023-09-12T16:02:50.833228+0000","last_undegraded":"2023-09-12T16:02:50.833228+0000","last_fullsized":"2023-09-12T16:02:50.833228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:15.999080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830392+0000","last_change":"2023-09-12T16:02:05.030285+0000","last_active":"2023-09-12T16:02:50.830392+0000","last_peered":"2023-09-12T16:02:50.830392+0000","last_clean":"2023-09-12T16:02:50.830392+0000","last_became_active":"2023-09-12T16:02:05.029585+0000","last_became_peered":"2023-09-12T16:02:05.029585+0000","last_unstale":"2023-09-12T16:02:50.830392+0000","last_undegraded":"2023-09-12T16:02:50.830392+0000","last_fullsized":"2023-09-12T16:02:50.830392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:52:27.471412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835453+0000","last_change":"2023-09-12T16:02:05.037258+0000","last_active":"2023-09-12T16:02:50.835453+0000","last_peered":"2023-09-12T16:02:50.835453+0000","last_clean":"2023-09-12T16:02:50.835453+0000","last_became_active":"2023-09-12T16:02:05.036941+0000","last_became_peered":"2023-09-12T16:02:05.036941+0000","last_unstale":"2023-09-12T16:02:50.835453+0000","last_undegraded":"2023-09-12T16:02:50.835453+0000","last_fullsized":"2023-09-12T16:02:50.835453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:04:17.998927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835219+0000","last_change":"2023-09-12T16:02:05.041986+0000","last_active":"2023-09-12T16:02:50.835219+0000","last_peered":"2023-09-12T16:02:50.835219+0000","last_clean":"2023-09-12T16:02:50.835219+0000","last_became_active":"2023-09-12T16:02:05.041584+0000","last_became_peered":"2023-09-12T16:02:05.041584+0000","last_unstale":"2023-09-12T16:02:50.835219+0000","last_undegraded":"2023-09-12T16:02:50.835219+0000","last_fullsized":"2023-09-12T16:02:50.835219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:39.065525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828785+0000","last_change":"2023-09-12T16:02:05.030405+0000","last_active":"2023-09-12T16:02:50.828785+0000","last_peered":"2023-09-12T16:02:50.828785+0000","last_clean":"2023-09-12T16:02:50.828785+0000","last_became_active":"2023-09-12T16:02:05.029488+0000","last_became_peered":"2023-09-12T16:02:05.029488+0000","last_unstale":"2023-09-12T16:02:50.828785+0000","last_undegraded":"2023-09-12T16:02:50.828785+0000","last_fullsized":"2023-09-12T16:02:50.828785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.646069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829401+0000","last_change":"2023-09-12T16:02:05.032576+0000","last_active":"2023-09-12T16:02:50.829401+0000","last_peered":"2023-09-12T16:02:50.829401+0000","last_clean":"2023-09-12T16:02:50.829401+0000","last_became_active":"2023-09-12T16:02:05.032277+0000","last_became_peered":"2023-09-12T16:02:05.032277+0000","last_unstale":"2023-09-12T16:02:50.829401+0000","last_undegraded":"2023-09-12T16:02:50.829401+0000","last_fullsized":"2023-09-12T16:02:50.829401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:47:32.734830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836039+0000","last_change":"2023-09-12T16:02:05.023468+0000","last_active":"2023-09-12T16:02:50.836039+0000","last_peered":"2023-09-12T16:02:50.836039+0000","last_clean":"2023-09-12T16:02:50.836039+0000","last_became_active":"2023-09-12T16:02:05.023132+0000","last_became_peered":"2023-09-12T16:02:05.023132+0000","last_unstale":"2023-09-12T16:02:50.836039+0000","last_undegraded":"2023-09-12T16:02:50.836039+0000","last_fullsized":"2023-09-12T16:02:50.836039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:21:49.378779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025810+0000","last_change":"2023-09-12T16:02:05.039910+0000","last_active":"2023-09-12T16:02:51.025810+0000","last_peered":"2023-09-12T16:02:51.025810+0000","last_clean":"2023-09-12T16:02:51.025810+0000","last_became_active":"2023-09-12T16:02:05.039605+0000","last_became_peered":"2023-09-12T16:02:05.039605+0000","last_unstale":"2023-09-12T16:02:51.025810+0000","last_undegraded":"2023-09-12T16:02:51.025810+0000","last_fullsized":"2023-09-12T16:02:51.025810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:19:58.823471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833500+0000","last_change":"2023-09-12T16:02:05.031667+0000","last_active":"2023-09-12T16:02:50.833500+0000","last_peered":"2023-09-12T16:02:50.833500+0000","last_clean":"2023-09-12T16:02:50.833500+0000","last_became_active":"2023-09-12T16:02:05.030886+0000","last_became_peered":"2023-09-12T16:02:05.030886+0000","last_unstale":"2023-09-12T16:02:50.833500+0000","last_undegraded":"2023-09-12T16:02:50.833500+0000","last_fullsized":"2023-09-12T16:02:50.833500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:24:02.312902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836131+0000","last_change":"2023-09-12T16:02:05.037607+0000","last_active":"2023-09-12T16:02:50.836131+0000","last_peered":"2023-09-12T16:02:50.836131+0000","last_clean":"2023-09-12T16:02:50.836131+0000","last_became_active":"2023-09-12T16:02:05.037095+0000","last_became_peered":"2023-09-12T16:02:05.037095+0000","last_unstale":"2023-09-12T16:02:50.836131+0000","last_undegraded":"2023-09-12T16:02:50.836131+0000","last_fullsized":"2023-09-12T16:02:50.836131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:17:04.886737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834805+0000","last_change":"2023-09-12T16:02:09.292093+0000","last_active":"2023-09-12T16:02:50.834805+0000","last_peered":"2023-09-12T16:02:50.834805+0000","last_clean":"2023-09-12T16:02:50.834805+0000","last_became_active":"2023-09-12T16:02:09.291602+0000","last_became_peered":"2023-09-12T16:02:09.291602+0000","last_unstale":"2023-09-12T16:02:50.834805+0000","last_undegraded":"2023-09-12T16:02:50.834805+0000","last_fullsized":"2023-09-12T16:02:50.834805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:36:23.089498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.857296+0000","last_change":"2023-09-12T16:02:51.857296+0000","last_active":"2023-09-12T16:02:51.857296+0000","last_peered":"2023-09-12T16:02:51.857296+0000","last_clean":"2023-09-12T16:02:51.857296+0000","last_became_active":"2023-09-12T16:02:51.843961+0000","last_became_peered":"2023-09-12T16:02:51.843961+0000","last_unstale":"2023-09-12T16:02:51.857296+0000","last_undegraded":"2023-09-12T16:02:51.857296+0000","last_fullsized":"2023-09-12T16:02:51.857296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:55:57.938037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832962+0000","last_change":"2023-09-12T16:02:05.032191+0000","last_active":"2023-09-12T16:02:50.832962+0000","last_peered":"2023-09-12T16:02:50.832962+0000","last_clean":"2023-09-12T16:02:50.832962+0000","last_became_active":"2023-09-12T16:02:05.031888+0000","last_became_peered":"2023-09-12T16:02:05.031888+0000","last_unstale":"2023-09-12T16:02:50.832962+0000","last_undegraded":"2023-09-12T16:02:50.832962+0000","last_fullsized":"2023-09-12T16:02:50.832962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:50:15.775078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834369+0000","last_change":"2023-09-12T16:02:09.290650+0000","last_active":"2023-09-12T16:02:50.834369+0000","last_peered":"2023-09-12T16:02:50.834369+0000","last_clean":"2023-09-12T16:02:50.834369+0000","last_became_active":"2023-09-12T16:02:09.290205+0000","last_became_peered":"2023-09-12T16:02:09.290205+0000","last_unstale":"2023-09-12T16:02:50.834369+0000","last_undegraded":"2023-09-12T16:02:50.834369+0000","last_fullsized":"2023-09-12T16:02:50.834369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:44:04.930063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833124+0000","last_change":"2023-09-12T16:02:04.032506+0000","last_active":"2023-09-12T16:02:50.833124+0000","last_peered":"2023-09-12T16:02:50.833124+0000","last_clean":"2023-09-12T16:02:50.833124+0000","last_became_active":"2023-09-12T16:02:04.032047+0000","last_became_peered":"2023-09-12T16:02:04.032047+0000","last_unstale":"2023-09-12T16:02:50.833124+0000","last_undegraded":"2023-09-12T16:02:50.833124+0000","last_fullsized":"2023-09-12T16:02:50.833124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:30:22.664465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.828905+0000","last_change":"2023-09-12T16:02:05.025692+0000","last_active":"2023-09-12T16:02:50.828905+0000","last_peered":"2023-09-12T16:02:50.828905+0000","last_clean":"2023-09-12T16:02:50.828905+0000","last_became_active":"2023-09-12T16:02:05.025370+0000","last_became_peered":"2023-09-12T16:02:05.025370+0000","last_unstale":"2023-09-12T16:02:50.828905+0000","last_undegraded":"2023-09-12T16:02:50.828905+0000","last_fullsized":"2023-09-12T16:02:50.828905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:00:18.305375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.068789+0000","last_change":"2023-09-12T16:02:52.068789+0000","last_active":"2023-09-12T16:02:52.068789+0000","last_peered":"2023-09-12T16:02:52.068789+0000","last_clean":"2023-09-12T16:02:52.068789+0000","last_became_active":"2023-09-12T16:02:52.068276+0000","last_became_peered":"2023-09-12T16:02:52.068276+0000","last_unstale":"2023-09-12T16:02:52.068789+0000","last_undegraded":"2023-09-12T16:02:52.068789+0000","last_fullsized":"2023-09-12T16:02:52.068789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:19:19.009173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831481+0000","last_change":"2023-09-12T16:02:04.032079+0000","last_active":"2023-09-12T16:02:50.831481+0000","last_peered":"2023-09-12T16:02:50.831481+0000","last_clean":"2023-09-12T16:02:50.831481+0000","last_became_active":"2023-09-12T16:02:04.031835+0000","last_became_peered":"2023-09-12T16:02:04.031835+0000","last_unstale":"2023-09-12T16:02:50.831481+0000","last_undegraded":"2023-09-12T16:02:50.831481+0000","last_fullsized":"2023-09-12T16:02:50.831481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:48:33.935012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837888+0000","last_change":"2023-09-12T16:02:05.037671+0000","last_active":"2023-09-12T16:02:50.837888+0000","last_peered":"2023-09-12T16:02:50.837888+0000","last_clean":"2023-09-12T16:02:50.837888+0000","last_became_active":"2023-09-12T16:02:05.037307+0000","last_became_peered":"2023-09-12T16:02:05.037307+0000","last_unstale":"2023-09-12T16:02:50.837888+0000","last_undegraded":"2023-09-12T16:02:50.837888+0000","last_fullsized":"2023-09-12T16:02:50.837888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:47:23.836612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834100+0000","last_change":"2023-09-12T16:02:09.289165+0000","last_active":"2023-09-12T16:02:50.834100+0000","last_peered":"2023-09-12T16:02:50.834100+0000","last_clean":"2023-09-12T16:02:50.834100+0000","last_became_active":"2023-09-12T16:02:09.288903+0000","last_became_peered":"2023-09-12T16:02:09.288903+0000","last_unstale":"2023-09-12T16:02:50.834100+0000","last_undegraded":"2023-09-12T16:02:50.834100+0000","last_fullsized":"2023-09-12T16:02:50.834100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:27:09.160822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831262+0000","last_change":"2023-09-12T16:02:04.027279+0000","last_active":"2023-09-12T16:02:50.831262+0000","last_peered":"2023-09-12T16:02:50.831262+0000","last_clean":"2023-09-12T16:02:50.831262+0000","last_became_active":"2023-09-12T16:02:04.026975+0000","last_became_peered":"2023-09-12T16:02:04.026975+0000","last_unstale":"2023-09-12T16:02:50.831262+0000","last_undegraded":"2023-09-12T16:02:50.831262+0000","last_fullsized":"2023-09-12T16:02:50.831262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:45.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832885+0000","last_change":"2023-09-12T16:02:05.027582+0000","last_active":"2023-09-12T16:02:50.832885+0000","last_peered":"2023-09-12T16:02:50.832885+0000","last_clean":"2023-09-12T16:02:50.832885+0000","last_became_active":"2023-09-12T16:02:05.027236+0000","last_became_peered":"2023-09-12T16:02:05.027236+0000","last_unstale":"2023-09-12T16:02:50.832885+0000","last_undegraded":"2023-09-12T16:02:50.832885+0000","last_fullsized":"2023-09-12T16:02:50.832885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:06:24.812731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836055+0000","last_change":"2023-09-12T16:02:09.284381+0000","last_active":"2023-09-12T16:02:50.836055+0000","last_peered":"2023-09-12T16:02:50.836055+0000","last_clean":"2023-09-12T16:02:50.836055+0000","last_became_active":"2023-09-12T16:02:09.283947+0000","last_became_peered":"2023-09-12T16:02:09.283947+0000","last_unstale":"2023-09-12T16:02:50.836055+0000","last_undegraded":"2023-09-12T16:02:50.836055+0000","last_fullsized":"2023-09-12T16:02:50.836055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:05:24.803781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:40.770486+0000","last_change":"2023-09-12T16:02:04.027693+0000","last_active":"2023-09-12T16:03:40.770486+0000","last_peered":"2023-09-12T16:03:40.770486+0000","last_clean":"2023-09-12T16:03:40.770486+0000","last_became_active":"2023-09-12T16:02:04.027315+0000","last_became_peered":"2023-09-12T16:02:04.027315+0000","last_unstale":"2023-09-12T16:03:40.770486+0000","last_undegraded":"2023-09-12T16:03:40.770486+0000","last_fullsized":"2023-09-12T16:03:40.770486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:32:21.756802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831054+0000","last_change":"2023-09-12T16:02:05.031673+0000","last_active":"2023-09-12T16:02:50.831054+0000","last_peered":"2023-09-12T16:02:50.831054+0000","last_clean":"2023-09-12T16:02:50.831054+0000","last_became_active":"2023-09-12T16:02:05.031364+0000","last_became_peered":"2023-09-12T16:02:05.031364+0000","last_unstale":"2023-09-12T16:02:50.831054+0000","last_undegraded":"2023-09-12T16:02:50.831054+0000","last_fullsized":"2023-09-12T16:02:50.831054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:48:05.222530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835641+0000","last_change":"2023-09-12T16:02:09.288536+0000","last_active":"2023-09-12T16:02:50.835641+0000","last_peered":"2023-09-12T16:02:50.835641+0000","last_clean":"2023-09-12T16:02:50.835641+0000","last_became_active":"2023-09-12T16:02:09.288062+0000","last_became_peered":"2023-09-12T16:02:09.288062+0000","last_unstale":"2023-09-12T16:02:50.835641+0000","last_undegraded":"2023-09-12T16:02:50.835641+0000","last_fullsized":"2023-09-12T16:02:50.835641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:39:28.935563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832024+0000","last_change":"2023-09-12T16:02:04.026298+0000","last_active":"2023-09-12T16:02:50.832024+0000","last_peered":"2023-09-12T16:02:50.832024+0000","last_clean":"2023-09-12T16:02:50.832024+0000","last_became_active":"2023-09-12T16:02:04.025975+0000","last_became_peered":"2023-09-12T16:02:04.025975+0000","last_unstale":"2023-09-12T16:02:50.832024+0000","last_undegraded":"2023-09-12T16:02:50.832024+0000","last_fullsized":"2023-09-12T16:02:50.832024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:12:04.338497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832502+0000","last_change":"2023-09-12T16:02:05.028337+0000","last_active":"2023-09-12T16:02:50.832502+0000","last_peered":"2023-09-12T16:02:50.832502+0000","last_clean":"2023-09-12T16:02:50.832502+0000","last_became_active":"2023-09-12T16:02:05.028036+0000","last_became_peered":"2023-09-12T16:02:05.028036+0000","last_unstale":"2023-09-12T16:02:50.832502+0000","last_undegraded":"2023-09-12T16:02:50.832502+0000","last_fullsized":"2023-09-12T16:02:50.832502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:39:09.387191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025809+0000","last_change":"2023-09-12T16:02:09.290950+0000","last_active":"2023-09-12T16:02:51.025809+0000","last_peered":"2023-09-12T16:02:51.025809+0000","last_clean":"2023-09-12T16:02:51.025809+0000","last_became_active":"2023-09-12T16:02:09.290477+0000","last_became_peered":"2023-09-12T16:02:09.290477+0000","last_unstale":"2023-09-12T16:02:51.025809+0000","last_undegraded":"2023-09-12T16:02:51.025809+0000","last_fullsized":"2023-09-12T16:02:51.025809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:53:45.907575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834845+0000","last_change":"2023-09-12T16:02:04.026025+0000","last_active":"2023-09-12T16:02:50.834845+0000","last_peered":"2023-09-12T16:02:50.834845+0000","last_clean":"2023-09-12T16:02:50.834845+0000","last_became_active":"2023-09-12T16:02:04.025722+0000","last_became_peered":"2023-09-12T16:02:04.025722+0000","last_unstale":"2023-09-12T16:02:50.834845+0000","last_undegraded":"2023-09-12T16:02:50.834845+0000","last_fullsized":"2023-09-12T16:02:50.834845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:29:58.357960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831920+0000","last_change":"2023-09-12T16:02:05.028461+0000","last_active":"2023-09-12T16:02:50.831920+0000","last_peered":"2023-09-12T16:02:50.831920+0000","last_clean":"2023-09-12T16:02:50.831920+0000","last_became_active":"2023-09-12T16:02:05.028176+0000","last_became_peered":"2023-09-12T16:02:05.028176+0000","last_unstale":"2023-09-12T16:02:50.831920+0000","last_undegraded":"2023-09-12T16:02:50.831920+0000","last_fullsized":"2023-09-12T16:02:50.831920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:16:48.652513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829138+0000","last_change":"2023-09-12T16:02:09.295245+0000","last_active":"2023-09-12T16:02:50.829138+0000","last_peered":"2023-09-12T16:02:50.829138+0000","last_clean":"2023-09-12T16:02:50.829138+0000","last_became_active":"2023-09-12T16:02:09.294872+0000","last_became_peered":"2023-09-12T16:02:09.294872+0000","last_unstale":"2023-09-12T16:02:50.829138+0000","last_undegraded":"2023-09-12T16:02:50.829138+0000","last_fullsized":"2023-09-12T16:02:50.829138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:43:49.462317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832140+0000","last_change":"2023-09-12T16:02:04.030815+0000","last_active":"2023-09-12T16:02:50.832140+0000","last_peered":"2023-09-12T16:02:50.832140+0000","last_clean":"2023-09-12T16:02:50.832140+0000","last_became_active":"2023-09-12T16:02:04.030435+0000","last_became_peered":"2023-09-12T16:02:04.030435+0000","last_unstale":"2023-09-12T16:02:50.832140+0000","last_undegraded":"2023-09-12T16:02:50.832140+0000","last_fullsized":"2023-09-12T16:02:50.832140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:39:59.661143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832565+0000","last_change":"2023-09-12T16:02:05.031155+0000","last_active":"2023-09-12T16:02:50.832565+0000","last_peered":"2023-09-12T16:02:50.832565+0000","last_clean":"2023-09-12T16:02:50.832565+0000","last_became_active":"2023-09-12T16:02:05.030803+0000","last_became_peered":"2023-09-12T16:02:05.030803+0000","last_unstale":"2023-09-12T16:02:50.832565+0000","last_undegraded":"2023-09-12T16:02:50.832565+0000","last_fullsized":"2023-09-12T16:02:50.832565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:55:21.951073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832473+0000","last_change":"2023-09-12T16:02:09.286445+0000","last_active":"2023-09-12T16:02:50.832473+0000","last_peered":"2023-09-12T16:02:50.832473+0000","last_clean":"2023-09-12T16:02:50.832473+0000","last_became_active":"2023-09-12T16:02:09.286049+0000","last_became_peered":"2023-09-12T16:02:09.286049+0000","last_unstale":"2023-09-12T16:02:50.832473+0000","last_undegraded":"2023-09-12T16:02:50.832473+0000","last_fullsized":"2023-09-12T16:02:50.832473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:43:13.943984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:44.775459+0000","last_change":"2023-09-12T16:02:04.035341+0000","last_active":"2023-09-12T16:03:44.775459+0000","last_peered":"2023-09-12T16:03:44.775459+0000","last_clean":"2023-09-12T16:03:44.775459+0000","last_became_active":"2023-09-12T16:02:04.035092+0000","last_became_peered":"2023-09-12T16:02:04.035092+0000","last_unstale":"2023-09-12T16:03:44.775459+0000","last_undegraded":"2023-09-12T16:03:44.775459+0000","last_fullsized":"2023-09-12T16:03:44.775459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:33:00.784401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838363+0000","last_change":"2023-09-12T16:02:05.025229+0000","last_active":"2023-09-12T16:02:50.838363+0000","last_peered":"2023-09-12T16:02:50.838363+0000","last_clean":"2023-09-12T16:02:50.838363+0000","last_became_active":"2023-09-12T16:02:05.024963+0000","last_became_peered":"2023-09-12T16:02:05.024963+0000","last_unstale":"2023-09-12T16:02:50.838363+0000","last_undegraded":"2023-09-12T16:02:50.838363+0000","last_fullsized":"2023-09-12T16:02:50.838363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:46:04.318082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832595+0000","last_change":"2023-09-12T16:02:09.295011+0000","last_active":"2023-09-12T16:02:50.832595+0000","last_peered":"2023-09-12T16:02:50.832595+0000","last_clean":"2023-09-12T16:02:50.832595+0000","last_became_active":"2023-09-12T16:02:09.294617+0000","last_became_peered":"2023-09-12T16:02:09.294617+0000","last_unstale":"2023-09-12T16:02:50.832595+0000","last_undegraded":"2023-09-12T16:02:50.832595+0000","last_fullsized":"2023-09-12T16:02:50.832595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:57:53.902946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830613+0000","last_change":"2023-09-12T16:02:04.028086+0000","last_active":"2023-09-12T16:02:50.830613+0000","last_peered":"2023-09-12T16:02:50.830613+0000","last_clean":"2023-09-12T16:02:50.830613+0000","last_became_active":"2023-09-12T16:02:04.027798+0000","last_became_peered":"2023-09-12T16:02:04.027798+0000","last_unstale":"2023-09-12T16:02:50.830613+0000","last_undegraded":"2023-09-12T16:02:50.830613+0000","last_fullsized":"2023-09-12T16:02:50.830613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:11:16.921824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829002+0000","last_change":"2023-09-12T16:02:05.030691+0000","last_active":"2023-09-12T16:02:50.829002+0000","last_peered":"2023-09-12T16:02:50.829002+0000","last_clean":"2023-09-12T16:02:50.829002+0000","last_became_active":"2023-09-12T16:02:05.030379+0000","last_became_peered":"2023-09-12T16:02:05.030379+0000","last_unstale":"2023-09-12T16:02:50.829002+0000","last_undegraded":"2023-09-12T16:02:50.829002+0000","last_fullsized":"2023-09-12T16:02:50.829002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:15:05.221078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835660+0000","last_change":"2023-09-12T16:02:09.290703+0000","last_active":"2023-09-12T16:02:50.835660+0000","last_peered":"2023-09-12T16:02:50.835660+0000","last_clean":"2023-09-12T16:02:50.835660+0000","last_became_active":"2023-09-12T16:02:09.290227+0000","last_became_peered":"2023-09-12T16:02:09.290227+0000","last_unstale":"2023-09-12T16:02:50.835660+0000","last_undegraded":"2023-09-12T16:02:50.835660+0000","last_fullsized":"2023-09-12T16:02:50.835660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:07:06.182295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832824+0000","last_change":"2023-09-12T16:02:04.028630+0000","last_active":"2023-09-12T16:02:50.832824+0000","last_peered":"2023-09-12T16:02:50.832824+0000","last_clean":"2023-09-12T16:02:50.832824+0000","last_became_active":"2023-09-12T16:02:04.028319+0000","last_became_peered":"2023-09-12T16:02:04.028319+0000","last_unstale":"2023-09-12T16:02:50.832824+0000","last_undegraded":"2023-09-12T16:02:50.832824+0000","last_fullsized":"2023-09-12T16:02:50.832824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:13:43.349425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.838071+0000","last_change":"2023-09-12T16:02:05.032392+0000","last_active":"2023-09-12T16:02:50.838071+0000","last_peered":"2023-09-12T16:02:50.838071+0000","last_clean":"2023-09-12T16:02:50.838071+0000","last_became_active":"2023-09-12T16:02:05.032099+0000","last_became_peered":"2023-09-12T16:02:05.032099+0000","last_unstale":"2023-09-12T16:02:50.838071+0000","last_undegraded":"2023-09-12T16:02:50.838071+0000","last_fullsized":"2023-09-12T16:02:50.838071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:46:03.108459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832745+0000","last_change":"2023-09-12T16:02:09.289573+0000","last_active":"2023-09-12T16:02:50.832745+0000","last_peered":"2023-09-12T16:02:50.832745+0000","last_clean":"2023-09-12T16:02:50.832745+0000","last_became_active":"2023-09-12T16:02:09.289179+0000","last_became_peered":"2023-09-12T16:02:09.289179+0000","last_unstale":"2023-09-12T16:02:50.832745+0000","last_undegraded":"2023-09-12T16:02:50.832745+0000","last_fullsized":"2023-09-12T16:02:50.832745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:52:35.905346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:44.931972+0000","last_change":"2023-09-12T16:02:51.851180+0000","last_active":"2023-09-12T16:03:44.931972+0000","last_peered":"2023-09-12T16:03:44.931972+0000","last_clean":"2023-09-12T16:03:44.931972+0000","last_became_active":"2023-09-12T16:02:51.841452+0000","last_became_peered":"2023-09-12T16:02:51.841452+0000","last_unstale":"2023-09-12T16:03:44.931972+0000","last_undegraded":"2023-09-12T16:03:44.931972+0000","last_fullsized":"2023-09-12T16:03:44.931972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:20:20.903637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65523,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":138,"num_write":35,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":59233,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831192+0000","last_change":"2023-09-12T16:02:05.031409+0000","last_active":"2023-09-12T16:02:50.831192+0000","last_peered":"2023-09-12T16:02:50.831192+0000","last_clean":"2023-09-12T16:02:50.831192+0000","last_became_active":"2023-09-12T16:02:05.031110+0000","last_became_peered":"2023-09-12T16:02:05.031110+0000","last_unstale":"2023-09-12T16:02:50.831192+0000","last_undegraded":"2023-09-12T16:02:50.831192+0000","last_fullsized":"2023-09-12T16:02:50.831192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:10:54.482078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833098+0000","last_change":"2023-09-12T16:02:09.291493+0000","last_active":"2023-09-12T16:02:50.833098+0000","last_peered":"2023-09-12T16:02:50.833098+0000","last_clean":"2023-09-12T16:02:50.833098+0000","last_became_active":"2023-09-12T16:02:09.291044+0000","last_became_peered":"2023-09-12T16:02:09.291044+0000","last_unstale":"2023-09-12T16:02:50.833098+0000","last_undegraded":"2023-09-12T16:02:50.833098+0000","last_fullsized":"2023-09-12T16:02:50.833098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:34:30.269720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835682+0000","last_change":"2023-09-12T16:02:04.029283+0000","last_active":"2023-09-12T16:02:50.835682+0000","last_peered":"2023-09-12T16:02:50.835682+0000","last_clean":"2023-09-12T16:02:50.835682+0000","last_became_active":"2023-09-12T16:02:04.028959+0000","last_became_peered":"2023-09-12T16:02:04.028959+0000","last_unstale":"2023-09-12T16:02:50.835682+0000","last_undegraded":"2023-09-12T16:02:50.835682+0000","last_fullsized":"2023-09-12T16:02:50.835682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:28:52.750278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834981+0000","last_change":"2023-09-12T16:02:05.034140+0000","last_active":"2023-09-12T16:02:50.834981+0000","last_peered":"2023-09-12T16:02:50.834981+0000","last_clean":"2023-09-12T16:02:50.834981+0000","last_became_active":"2023-09-12T16:02:05.033840+0000","last_became_peered":"2023-09-12T16:02:05.033840+0000","last_unstale":"2023-09-12T16:02:50.834981+0000","last_undegraded":"2023-09-12T16:02:50.834981+0000","last_fullsized":"2023-09-12T16:02:50.834981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:59:55.522608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025577+0000","last_change":"2023-09-12T16:02:09.289998+0000","last_active":"2023-09-12T16:02:51.025577+0000","last_peered":"2023-09-12T16:02:51.025577+0000","last_clean":"2023-09-12T16:02:51.025577+0000","last_became_active":"2023-09-12T16:02:09.289561+0000","last_became_peered":"2023-09-12T16:02:09.289561+0000","last_unstale":"2023-09-12T16:02:51.025577+0000","last_undegraded":"2023-09-12T16:02:51.025577+0000","last_fullsized":"2023-09-12T16:02:51.025577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:09:47.327213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832456+0000","last_change":"2023-09-12T16:02:04.029883+0000","last_active":"2023-09-12T16:02:50.832456+0000","last_peered":"2023-09-12T16:02:50.832456+0000","last_clean":"2023-09-12T16:02:50.832456+0000","last_became_active":"2023-09-12T16:02:04.029547+0000","last_became_peered":"2023-09-12T16:02:04.029547+0000","last_unstale":"2023-09-12T16:02:50.832456+0000","last_undegraded":"2023-09-12T16:02:50.832456+0000","last_fullsized":"2023-09-12T16:02:50.832456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:47:46.478325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846042+0000","last_change":"2023-09-12T16:02:51.846042+0000","last_active":"2023-09-12T16:02:51.846042+0000","last_peered":"2023-09-12T16:02:51.846042+0000","last_clean":"2023-09-12T16:02:51.846042+0000","last_became_active":"2023-09-12T16:02:51.845572+0000","last_became_peered":"2023-09-12T16:02:51.845572+0000","last_unstale":"2023-09-12T16:02:51.846042+0000","last_undegraded":"2023-09-12T16:02:51.846042+0000","last_fullsized":"2023-09-12T16:02:51.846042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:55.335485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069758+0000","last_change":"2023-09-12T16:02:52.069157+0000","last_active":"2023-09-12T16:02:52.069758+0000","last_peered":"2023-09-12T16:02:52.069758+0000","last_clean":"2023-09-12T16:02:52.069758+0000","last_became_active":"2023-09-12T16:02:52.068600+0000","last_became_peered":"2023-09-12T16:02:52.068600+0000","last_unstale":"2023-09-12T16:02:52.069758+0000","last_undegraded":"2023-09-12T16:02:52.069758+0000","last_fullsized":"2023-09-12T16:02:52.069758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:28:44.160612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:40.769628+0000","last_change":"2023-09-12T16:02:04.025723+0000","last_active":"2023-09-12T16:03:40.769628+0000","last_peered":"2023-09-12T16:03:40.769628+0000","last_clean":"2023-09-12T16:03:40.769628+0000","last_became_active":"2023-09-12T16:02:04.025432+0000","last_became_peered":"2023-09-12T16:02:04.025432+0000","last_unstale":"2023-09-12T16:03:40.769628+0000","last_undegraded":"2023-09-12T16:03:40.769628+0000","last_fullsized":"2023-09-12T16:03:40.769628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:56.279910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.839291+0000","last_change":"2023-09-12T16:02:51.839291+0000","last_active":"2023-09-12T16:02:51.839291+0000","last_peered":"2023-09-12T16:02:51.839291+0000","last_clean":"2023-09-12T16:02:51.839291+0000","last_became_active":"2023-09-12T16:02:51.838968+0000","last_became_peered":"2023-09-12T16:02:51.838968+0000","last_unstale":"2023-09-12T16:02:51.839291+0000","last_undegraded":"2023-09-12T16:02:51.839291+0000","last_fullsized":"2023-09-12T16:02:51.839291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:21:12.542019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835124+0000","last_change":"2023-09-12T16:02:09.286447+0000","last_active":"2023-09-12T16:02:50.835124+0000","last_peered":"2023-09-12T16:02:50.835124+0000","last_clean":"2023-09-12T16:02:50.835124+0000","last_became_active":"2023-09-12T16:02:09.285805+0000","last_became_peered":"2023-09-12T16:02:09.285805+0000","last_unstale":"2023-09-12T16:02:50.835124+0000","last_undegraded":"2023-09-12T16:02:50.835124+0000","last_fullsized":"2023-09-12T16:02:50.835124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:35:05.518872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830710+0000","last_change":"2023-09-12T16:02:04.032527+0000","last_active":"2023-09-12T16:02:50.830710+0000","last_peered":"2023-09-12T16:02:50.830710+0000","last_clean":"2023-09-12T16:02:50.830710+0000","last_became_active":"2023-09-12T16:02:04.032264+0000","last_became_peered":"2023-09-12T16:02:04.032264+0000","last_unstale":"2023-09-12T16:02:50.830710+0000","last_undegraded":"2023-09-12T16:02:50.830710+0000","last_fullsized":"2023-09-12T16:02:50.830710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:53:52.433549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833597+0000","last_change":"2023-09-12T16:02:05.029281+0000","last_active":"2023-09-12T16:02:50.833597+0000","last_peered":"2023-09-12T16:02:50.833597+0000","last_clean":"2023-09-12T16:02:50.833597+0000","last_became_active":"2023-09-12T16:02:05.028919+0000","last_became_peered":"2023-09-12T16:02:05.028919+0000","last_unstale":"2023-09-12T16:02:50.833597+0000","last_undegraded":"2023-09-12T16:02:50.833597+0000","last_fullsized":"2023-09-12T16:02:50.833597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:11:26.667060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:52.069744+0000","last_change":"2023-09-12T16:02:52.069336+0000","last_active":"2023-09-12T16:02:52.069744+0000","last_peered":"2023-09-12T16:02:52.069744+0000","last_clean":"2023-09-12T16:02:52.069744+0000","last_became_active":"2023-09-12T16:02:52.068805+0000","last_became_peered":"2023-09-12T16:02:52.068805+0000","last_unstale":"2023-09-12T16:02:52.069744+0000","last_undegraded":"2023-09-12T16:02:52.069744+0000","last_fullsized":"2023-09-12T16:02:52.069744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:09.095952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831444+0000","last_change":"2023-09-12T16:02:04.026373+0000","last_active":"2023-09-12T16:02:50.831444+0000","last_peered":"2023-09-12T16:02:50.831444+0000","last_clean":"2023-09-12T16:02:50.831444+0000","last_became_active":"2023-09-12T16:02:04.025907+0000","last_became_peered":"2023-09-12T16:02:04.025907+0000","last_unstale":"2023-09-12T16:02:50.831444+0000","last_undegraded":"2023-09-12T16:02:50.831444+0000","last_fullsized":"2023-09-12T16:02:50.831444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:33:15.152128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025705+0000","last_change":"2023-09-12T16:02:05.040143+0000","last_active":"2023-09-12T16:02:51.025705+0000","last_peered":"2023-09-12T16:02:51.025705+0000","last_clean":"2023-09-12T16:02:51.025705+0000","last_became_active":"2023-09-12T16:02:05.039749+0000","last_became_peered":"2023-09-12T16:02:05.039749+0000","last_unstale":"2023-09-12T16:02:51.025705+0000","last_undegraded":"2023-09-12T16:02:51.025705+0000","last_fullsized":"2023-09-12T16:02:51.025705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:49.561252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831034+0000","last_change":"2023-09-12T16:02:09.841670+0000","last_active":"2023-09-12T16:02:50.831034+0000","last_peered":"2023-09-12T16:02:50.831034+0000","last_clean":"2023-09-12T16:02:50.831034+0000","last_became_active":"2023-09-12T16:02:09.841203+0000","last_became_peered":"2023-09-12T16:02:09.841203+0000","last_unstale":"2023-09-12T16:02:50.831034+0000","last_undegraded":"2023-09-12T16:02:50.831034+0000","last_fullsized":"2023-09-12T16:02:50.831034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:55:46.128996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833178+0000","last_change":"2023-09-12T16:02:04.027953+0000","last_active":"2023-09-12T16:02:50.833178+0000","last_peered":"2023-09-12T16:02:50.833178+0000","last_clean":"2023-09-12T16:02:50.833178+0000","last_became_active":"2023-09-12T16:02:04.027624+0000","last_became_peered":"2023-09-12T16:02:04.027624+0000","last_unstale":"2023-09-12T16:02:50.833178+0000","last_undegraded":"2023-09-12T16:02:50.833178+0000","last_fullsized":"2023-09-12T16:02:50.833178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:33:58.356358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832388+0000","last_change":"2023-09-12T16:02:05.036593+0000","last_active":"2023-09-12T16:02:50.832388+0000","last_peered":"2023-09-12T16:02:50.832388+0000","last_clean":"2023-09-12T16:02:50.832388+0000","last_became_active":"2023-09-12T16:02:05.036204+0000","last_became_peered":"2023-09-12T16:02:05.036204+0000","last_unstale":"2023-09-12T16:02:50.832388+0000","last_undegraded":"2023-09-12T16:02:50.832388+0000","last_fullsized":"2023-09-12T16:02:50.832388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:12:31.068837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835475+0000","last_change":"2023-09-12T16:02:09.292405+0000","last_active":"2023-09-12T16:02:50.835475+0000","last_peered":"2023-09-12T16:02:50.835475+0000","last_clean":"2023-09-12T16:02:50.835475+0000","last_became_active":"2023-09-12T16:02:09.291877+0000","last_became_peered":"2023-09-12T16:02:09.291877+0000","last_unstale":"2023-09-12T16:02:50.835475+0000","last_undegraded":"2023-09-12T16:02:50.835475+0000","last_fullsized":"2023-09-12T16:02:50.835475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:53:04.847310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829465+0000","last_change":"2023-09-12T16:02:04.034738+0000","last_active":"2023-09-12T16:02:50.829465+0000","last_peered":"2023-09-12T16:02:50.829465+0000","last_clean":"2023-09-12T16:02:50.829465+0000","last_became_active":"2023-09-12T16:02:04.034408+0000","last_became_peered":"2023-09-12T16:02:04.034408+0000","last_unstale":"2023-09-12T16:02:50.829465+0000","last_undegraded":"2023-09-12T16:02:50.829465+0000","last_fullsized":"2023-09-12T16:02:50.829465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:27.845770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832464+0000","last_change":"2023-09-12T16:02:05.044936+0000","last_active":"2023-09-12T16:02:50.832464+0000","last_peered":"2023-09-12T16:02:50.832464+0000","last_clean":"2023-09-12T16:02:50.832464+0000","last_became_active":"2023-09-12T16:02:05.044617+0000","last_became_peered":"2023-09-12T16:02:05.044617+0000","last_unstale":"2023-09-12T16:02:50.832464+0000","last_undegraded":"2023-09-12T16:02:50.832464+0000","last_fullsized":"2023-09-12T16:02:50.832464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:41:32.639331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834992+0000","last_change":"2023-09-12T16:02:09.841919+0000","last_active":"2023-09-12T16:02:50.834992+0000","last_peered":"2023-09-12T16:02:50.834992+0000","last_clean":"2023-09-12T16:02:50.834992+0000","last_became_active":"2023-09-12T16:02:09.841410+0000","last_became_peered":"2023-09-12T16:02:09.841410+0000","last_unstale":"2023-09-12T16:02:50.834992+0000","last_undegraded":"2023-09-12T16:02:50.834992+0000","last_fullsized":"2023-09-12T16:02:50.834992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:29:57.511693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.840116+0000","last_change":"2023-09-12T16:02:51.840116+0000","last_active":"2023-09-12T16:02:51.840116+0000","last_peered":"2023-09-12T16:02:51.840116+0000","last_clean":"2023-09-12T16:02:51.840116+0000","last_became_active":"2023-09-12T16:02:51.839698+0000","last_became_peered":"2023-09-12T16:02:51.839698+0000","last_unstale":"2023-09-12T16:02:51.840116+0000","last_undegraded":"2023-09-12T16:02:51.840116+0000","last_fullsized":"2023-09-12T16:02:51.840116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:29.475855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:03:43.368647+0000","last_change":"2023-09-12T16:02:04.032892+0000","last_active":"2023-09-12T16:03:43.368647+0000","last_peered":"2023-09-12T16:03:43.368647+0000","last_clean":"2023-09-12T16:03:43.368647+0000","last_became_active":"2023-09-12T16:02:04.032637+0000","last_became_peered":"2023-09-12T16:02:04.032637+0000","last_unstale":"2023-09-12T16:03:43.368647+0000","last_undegraded":"2023-09-12T16:03:43.368647+0000","last_fullsized":"2023-09-12T16:03:43.368647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:22:55.805000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830231+0000","last_change":"2023-09-12T16:02:09.285127+0000","last_active":"2023-09-12T16:02:50.830231+0000","last_peered":"2023-09-12T16:02:50.830231+0000","last_clean":"2023-09-12T16:02:50.830231+0000","last_became_active":"2023-09-12T16:02:09.284790+0000","last_became_peered":"2023-09-12T16:02:09.284790+0000","last_unstale":"2023-09-12T16:02:50.830231+0000","last_undegraded":"2023-09-12T16:02:50.830231+0000","last_fullsized":"2023-09-12T16:02:50.830231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:50:50.578069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832025+0000","last_change":"2023-09-12T16:02:05.027877+0000","last_active":"2023-09-12T16:02:50.832025+0000","last_peered":"2023-09-12T16:02:50.832025+0000","last_clean":"2023-09-12T16:02:50.832025+0000","last_became_active":"2023-09-12T16:02:05.027582+0000","last_became_peered":"2023-09-12T16:02:05.027582+0000","last_unstale":"2023-09-12T16:02:50.832025+0000","last_undegraded":"2023-09-12T16:02:50.832025+0000","last_fullsized":"2023-09-12T16:02:50.832025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:47.835790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835647+0000","last_change":"2023-09-12T16:02:04.024800+0000","last_active":"2023-09-12T16:02:50.835647+0000","last_peered":"2023-09-12T16:02:50.835647+0000","last_clean":"2023-09-12T16:02:50.835647+0000","last_became_active":"2023-09-12T16:02:04.024436+0000","last_became_peered":"2023-09-12T16:02:04.024436+0000","last_unstale":"2023-09-12T16:02:50.835647+0000","last_undegraded":"2023-09-12T16:02:50.835647+0000","last_fullsized":"2023-09-12T16:02:50.835647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:57:35.093322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829225+0000","last_change":"2023-09-12T16:01:04.599706+0000","last_active":"2023-09-12T16:02:50.829225+0000","last_peered":"2023-09-12T16:02:50.829225+0000","last_clean":"2023-09-12T16:02:50.829225+0000","last_became_active":"2023-09-12T16:01:04.599328+0000","last_became_peered":"2023-09-12T16:01:04.599328+0000","last_unstale":"2023-09-12T16:02:50.829225+0000","last_undegraded":"2023-09-12T16:02:50.829225+0000","last_fullsized":"2023-09-12T16:02:50.829225+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T15:58:15.250648+0000","last_clean_scrub_stamp":"2023-09-12T15:58:15.250648+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:57:56.846488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833462+0000","last_change":"2023-09-12T16:02:09.841960+0000","last_active":"2023-09-12T16:02:50.833462+0000","last_peered":"2023-09-12T16:02:50.833462+0000","last_clean":"2023-09-12T16:02:50.833462+0000","last_became_active":"2023-09-12T16:02:09.841279+0000","last_became_peered":"2023-09-12T16:02:09.841279+0000","last_unstale":"2023-09-12T16:02:50.833462+0000","last_undegraded":"2023-09-12T16:02:50.833462+0000","last_fullsized":"2023-09-12T16:02:50.833462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:17:19.419350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.847600+0000","last_change":"2023-09-12T16:02:51.847600+0000","last_active":"2023-09-12T16:02:51.847600+0000","last_peered":"2023-09-12T16:02:51.847600+0000","last_clean":"2023-09-12T16:02:51.847600+0000","last_became_active":"2023-09-12T16:02:51.847292+0000","last_became_peered":"2023-09-12T16:02:51.847292+0000","last_unstale":"2023-09-12T16:02:51.847600+0000","last_undegraded":"2023-09-12T16:02:51.847600+0000","last_fullsized":"2023-09-12T16:02:51.847600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:25:58.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829344+0000","last_change":"2023-09-12T16:02:04.029790+0000","last_active":"2023-09-12T16:02:50.829344+0000","last_peered":"2023-09-12T16:02:50.829344+0000","last_clean":"2023-09-12T16:02:50.829344+0000","last_became_active":"2023-09-12T16:02:04.029440+0000","last_became_peered":"2023-09-12T16:02:04.029440+0000","last_unstale":"2023-09-12T16:02:50.829344+0000","last_undegraded":"2023-09-12T16:02:50.829344+0000","last_fullsized":"2023-09-12T16:02:50.829344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:50:33.690628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025552+0000","last_change":"2023-09-12T16:02:09.289757+0000","last_active":"2023-09-12T16:02:51.025552+0000","last_peered":"2023-09-12T16:02:51.025552+0000","last_clean":"2023-09-12T16:02:51.025552+0000","last_became_active":"2023-09-12T16:02:09.289348+0000","last_became_peered":"2023-09-12T16:02:09.289348+0000","last_unstale":"2023-09-12T16:02:51.025552+0000","last_undegraded":"2023-09-12T16:02:51.025552+0000","last_fullsized":"2023-09-12T16:02:51.025552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:15:14.695521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831296+0000","last_change":"2023-09-12T16:02:04.033213+0000","last_active":"2023-09-12T16:02:50.831296+0000","last_peered":"2023-09-12T16:02:50.831296+0000","last_clean":"2023-09-12T16:02:50.831296+0000","last_became_active":"2023-09-12T16:02:04.032978+0000","last_became_peered":"2023-09-12T16:02:04.032978+0000","last_unstale":"2023-09-12T16:02:50.831296+0000","last_undegraded":"2023-09-12T16:02:50.831296+0000","last_fullsized":"2023-09-12T16:02:50.831296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:25:33.718954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835250+0000","last_change":"2023-09-12T16:02:05.043252+0000","last_active":"2023-09-12T16:02:50.835250+0000","last_peered":"2023-09-12T16:02:50.835250+0000","last_clean":"2023-09-12T16:02:50.835250+0000","last_became_active":"2023-09-12T16:02:05.042975+0000","last_became_peered":"2023-09-12T16:02:05.042975+0000","last_unstale":"2023-09-12T16:02:50.835250+0000","last_undegraded":"2023-09-12T16:02:50.835250+0000","last_fullsized":"2023-09-12T16:02:50.835250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:46:31.549547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831861+0000","last_change":"2023-09-12T16:02:09.841728+0000","last_active":"2023-09-12T16:02:50.831861+0000","last_peered":"2023-09-12T16:02:50.831861+0000","last_clean":"2023-09-12T16:02:50.831861+0000","last_became_active":"2023-09-12T16:02:09.841351+0000","last_became_peered":"2023-09-12T16:02:09.841351+0000","last_unstale":"2023-09-12T16:02:50.831861+0000","last_undegraded":"2023-09-12T16:02:50.831861+0000","last_fullsized":"2023-09-12T16:02:50.831861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:00:50.637272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833531+0000","last_change":"2023-09-12T16:02:04.025624+0000","last_active":"2023-09-12T16:02:50.833531+0000","last_peered":"2023-09-12T16:02:50.833531+0000","last_clean":"2023-09-12T16:02:50.833531+0000","last_became_active":"2023-09-12T16:02:04.025330+0000","last_became_peered":"2023-09-12T16:02:04.025330+0000","last_unstale":"2023-09-12T16:02:50.833531+0000","last_undegraded":"2023-09-12T16:02:50.833531+0000","last_fullsized":"2023-09-12T16:02:50.833531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:00:57.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833174+0000","last_change":"2023-09-12T16:02:05.028414+0000","last_active":"2023-09-12T16:02:50.833174+0000","last_peered":"2023-09-12T16:02:50.833174+0000","last_clean":"2023-09-12T16:02:50.833174+0000","last_became_active":"2023-09-12T16:02:05.028081+0000","last_became_peered":"2023-09-12T16:02:05.028081+0000","last_unstale":"2023-09-12T16:02:50.833174+0000","last_undegraded":"2023-09-12T16:02:50.833174+0000","last_fullsized":"2023-09-12T16:02:50.833174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:57:58.245105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834801+0000","last_change":"2023-09-12T16:02:09.283222+0000","last_active":"2023-09-12T16:02:50.834801+0000","last_peered":"2023-09-12T16:02:50.834801+0000","last_clean":"2023-09-12T16:02:50.834801+0000","last_became_active":"2023-09-12T16:02:09.282814+0000","last_became_peered":"2023-09-12T16:02:09.282814+0000","last_unstale":"2023-09-12T16:02:50.834801+0000","last_undegraded":"2023-09-12T16:02:50.834801+0000","last_fullsized":"2023-09-12T16:02:50.834801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:18.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836561+0000","last_change":"2023-09-12T16:02:04.026694+0000","last_active":"2023-09-12T16:02:50.836561+0000","last_peered":"2023-09-12T16:02:50.836561+0000","last_clean":"2023-09-12T16:02:50.836561+0000","last_became_active":"2023-09-12T16:02:04.026368+0000","last_became_peered":"2023-09-12T16:02:04.026368+0000","last_unstale":"2023-09-12T16:02:50.836561+0000","last_undegraded":"2023-09-12T16:02:50.836561+0000","last_fullsized":"2023-09-12T16:02:50.836561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:14:10.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829216+0000","last_change":"2023-09-12T16:02:05.039037+0000","last_active":"2023-09-12T16:02:50.829216+0000","last_peered":"2023-09-12T16:02:50.829216+0000","last_clean":"2023-09-12T16:02:50.829216+0000","last_became_active":"2023-09-12T16:02:05.038708+0000","last_became_peered":"2023-09-12T16:02:05.038708+0000","last_unstale":"2023-09-12T16:02:50.829216+0000","last_undegraded":"2023-09-12T16:02:50.829216+0000","last_fullsized":"2023-09-12T16:02:50.829216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:13:09.805646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836160+0000","last_change":"2023-09-12T16:02:09.841757+0000","last_active":"2023-09-12T16:02:50.836160+0000","last_peered":"2023-09-12T16:02:50.836160+0000","last_clean":"2023-09-12T16:02:50.836160+0000","last_became_active":"2023-09-12T16:02:09.841173+0000","last_became_peered":"2023-09-12T16:02:09.841173+0000","last_unstale":"2023-09-12T16:02:50.836160+0000","last_undegraded":"2023-09-12T16:02:50.836160+0000","last_fullsized":"2023-09-12T16:02:50.836160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:57:06.626828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836147+0000","last_change":"2023-09-12T16:02:04.031009+0000","last_active":"2023-09-12T16:02:50.836147+0000","last_peered":"2023-09-12T16:02:50.836147+0000","last_clean":"2023-09-12T16:02:50.836147+0000","last_became_active":"2023-09-12T16:02:04.030714+0000","last_became_peered":"2023-09-12T16:02:04.030714+0000","last_unstale":"2023-09-12T16:02:50.836147+0000","last_undegraded":"2023-09-12T16:02:50.836147+0000","last_fullsized":"2023-09-12T16:02:50.836147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:12.578686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830789+0000","last_change":"2023-09-12T16:02:05.030778+0000","last_active":"2023-09-12T16:02:50.830789+0000","last_peered":"2023-09-12T16:02:50.830789+0000","last_clean":"2023-09-12T16:02:50.830789+0000","last_became_active":"2023-09-12T16:02:05.030454+0000","last_became_peered":"2023-09-12T16:02:05.030454+0000","last_unstale":"2023-09-12T16:02:50.830789+0000","last_undegraded":"2023-09-12T16:02:50.830789+0000","last_fullsized":"2023-09-12T16:02:50.830789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:06:52.951246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834030+0000","last_change":"2023-09-12T16:02:09.289668+0000","last_active":"2023-09-12T16:02:50.834030+0000","last_peered":"2023-09-12T16:02:50.834030+0000","last_clean":"2023-09-12T16:02:50.834030+0000","last_became_active":"2023-09-12T16:02:09.289301+0000","last_became_peered":"2023-09-12T16:02:09.289301+0000","last_unstale":"2023-09-12T16:02:50.834030+0000","last_undegraded":"2023-09-12T16:02:50.834030+0000","last_fullsized":"2023-09-12T16:02:50.834030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:20:04.528146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832522+0000","last_change":"2023-09-12T16:02:04.036175+0000","last_active":"2023-09-12T16:02:50.832522+0000","last_peered":"2023-09-12T16:02:50.832522+0000","last_clean":"2023-09-12T16:02:50.832522+0000","last_became_active":"2023-09-12T16:02:04.035879+0000","last_became_peered":"2023-09-12T16:02:04.035879+0000","last_unstale":"2023-09-12T16:02:50.832522+0000","last_undegraded":"2023-09-12T16:02:50.832522+0000","last_fullsized":"2023-09-12T16:02:50.832522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:29:20.798716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025379+0000","last_change":"2023-09-12T16:02:05.034195+0000","last_active":"2023-09-12T16:02:51.025379+0000","last_peered":"2023-09-12T16:02:51.025379+0000","last_clean":"2023-09-12T16:02:51.025379+0000","last_became_active":"2023-09-12T16:02:05.033534+0000","last_became_peered":"2023-09-12T16:02:05.033534+0000","last_unstale":"2023-09-12T16:02:51.025379+0000","last_undegraded":"2023-09-12T16:02:51.025379+0000","last_fullsized":"2023-09-12T16:02:51.025379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:34:04.180218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834899+0000","last_change":"2023-09-12T16:02:09.287543+0000","last_active":"2023-09-12T16:02:50.834899+0000","last_peered":"2023-09-12T16:02:50.834899+0000","last_clean":"2023-09-12T16:02:50.834899+0000","last_became_active":"2023-09-12T16:02:09.287055+0000","last_became_peered":"2023-09-12T16:02:09.287055+0000","last_unstale":"2023-09-12T16:02:50.834899+0000","last_undegraded":"2023-09-12T16:02:50.834899+0000","last_fullsized":"2023-09-12T16:02:50.834899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:58.214381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834994+0000","last_change":"2023-09-12T16:02:04.030206+0000","last_active":"2023-09-12T16:02:50.834994+0000","last_peered":"2023-09-12T16:02:50.834994+0000","last_clean":"2023-09-12T16:02:50.834994+0000","last_became_active":"2023-09-12T16:02:04.029765+0000","last_became_peered":"2023-09-12T16:02:04.029765+0000","last_unstale":"2023-09-12T16:02:50.834994+0000","last_undegraded":"2023-09-12T16:02:50.834994+0000","last_fullsized":"2023-09-12T16:02:50.834994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:56:07.551681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832663+0000","last_change":"2023-09-12T16:02:05.028300+0000","last_active":"2023-09-12T16:02:50.832663+0000","last_peered":"2023-09-12T16:02:50.832663+0000","last_clean":"2023-09-12T16:02:50.832663+0000","last_became_active":"2023-09-12T16:02:05.027970+0000","last_became_peered":"2023-09-12T16:02:05.027970+0000","last_unstale":"2023-09-12T16:02:50.832663+0000","last_undegraded":"2023-09-12T16:02:50.832663+0000","last_fullsized":"2023-09-12T16:02:50.832663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:39:44.571593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835970+0000","last_change":"2023-09-12T16:02:09.841661+0000","last_active":"2023-09-12T16:02:50.835970+0000","last_peered":"2023-09-12T16:02:50.835970+0000","last_clean":"2023-09-12T16:02:50.835970+0000","last_became_active":"2023-09-12T16:02:09.841039+0000","last_became_peered":"2023-09-12T16:02:09.841039+0000","last_unstale":"2023-09-12T16:02:50.835970+0000","last_undegraded":"2023-09-12T16:02:50.835970+0000","last_fullsized":"2023-09-12T16:02:50.835970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:01:34.529098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830867+0000","last_change":"2023-09-12T16:02:04.031927+0000","last_active":"2023-09-12T16:02:50.830867+0000","last_peered":"2023-09-12T16:02:50.830867+0000","last_clean":"2023-09-12T16:02:50.830867+0000","last_became_active":"2023-09-12T16:02:04.031614+0000","last_became_peered":"2023-09-12T16:02:04.031614+0000","last_unstale":"2023-09-12T16:02:50.830867+0000","last_undegraded":"2023-09-12T16:02:50.830867+0000","last_fullsized":"2023-09-12T16:02:50.830867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:00:23.423933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829483+0000","last_change":"2023-09-12T16:02:05.032647+0000","last_active":"2023-09-12T16:02:50.829483+0000","last_peered":"2023-09-12T16:02:50.829483+0000","last_clean":"2023-09-12T16:02:50.829483+0000","last_became_active":"2023-09-12T16:02:05.031701+0000","last_became_peered":"2023-09-12T16:02:05.031701+0000","last_unstale":"2023-09-12T16:02:50.829483+0000","last_undegraded":"2023-09-12T16:02:50.829483+0000","last_fullsized":"2023-09-12T16:02:50.829483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:02:19.596913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837136+0000","last_change":"2023-09-12T16:02:09.291043+0000","last_active":"2023-09-12T16:02:50.837136+0000","last_peered":"2023-09-12T16:02:50.837136+0000","last_clean":"2023-09-12T16:02:50.837136+0000","last_became_active":"2023-09-12T16:02:09.290666+0000","last_became_peered":"2023-09-12T16:02:09.290666+0000","last_unstale":"2023-09-12T16:02:50.837136+0000","last_undegraded":"2023-09-12T16:02:50.837136+0000","last_fullsized":"2023-09-12T16:02:50.837136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:10:47.354479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835888+0000","last_change":"2023-09-12T16:02:04.030501+0000","last_active":"2023-09-12T16:02:50.835888+0000","last_peered":"2023-09-12T16:02:50.835888+0000","last_clean":"2023-09-12T16:02:50.835888+0000","last_became_active":"2023-09-12T16:02:04.030196+0000","last_became_peered":"2023-09-12T16:02:04.030196+0000","last_unstale":"2023-09-12T16:02:50.835888+0000","last_undegraded":"2023-09-12T16:02:50.835888+0000","last_fullsized":"2023-09-12T16:02:50.835888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:58:34.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830617+0000","last_change":"2023-09-12T16:02:05.030466+0000","last_active":"2023-09-12T16:02:50.830617+0000","last_peered":"2023-09-12T16:02:50.830617+0000","last_clean":"2023-09-12T16:02:50.830617+0000","last_became_active":"2023-09-12T16:02:05.029988+0000","last_became_peered":"2023-09-12T16:02:05.029988+0000","last_unstale":"2023-09-12T16:02:50.830617+0000","last_undegraded":"2023-09-12T16:02:50.830617+0000","last_fullsized":"2023-09-12T16:02:50.830617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:08:56.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.846403+0000","last_change":"2023-09-12T16:02:51.846403+0000","last_active":"2023-09-12T16:02:51.846403+0000","last_peered":"2023-09-12T16:02:51.846403+0000","last_clean":"2023-09-12T16:02:51.846403+0000","last_became_active":"2023-09-12T16:02:51.846047+0000","last_became_peered":"2023-09-12T16:02:51.846047+0000","last_unstale":"2023-09-12T16:02:51.846403+0000","last_undegraded":"2023-09-12T16:02:51.846403+0000","last_fullsized":"2023-09-12T16:02:51.846403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:58:11.458339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832945+0000","last_change":"2023-09-12T16:02:04.027380+0000","last_active":"2023-09-12T16:02:50.832945+0000","last_peered":"2023-09-12T16:02:50.832945+0000","last_clean":"2023-09-12T16:02:50.832945+0000","last_became_active":"2023-09-12T16:02:04.027098+0000","last_became_peered":"2023-09-12T16:02:04.027098+0000","last_unstale":"2023-09-12T16:02:50.832945+0000","last_undegraded":"2023-09-12T16:02:50.832945+0000","last_fullsized":"2023-09-12T16:02:50.832945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:34:12.260747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.829993+0000","last_change":"2023-09-12T16:02:05.029249+0000","last_active":"2023-09-12T16:02:50.829993+0000","last_peered":"2023-09-12T16:02:50.829993+0000","last_clean":"2023-09-12T16:02:50.829993+0000","last_became_active":"2023-09-12T16:02:05.028955+0000","last_became_peered":"2023-09-12T16:02:05.028955+0000","last_unstale":"2023-09-12T16:02:50.829993+0000","last_undegraded":"2023-09-12T16:02:50.829993+0000","last_fullsized":"2023-09-12T16:02:50.829993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:02:10.382712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831923+0000","last_change":"2023-09-12T16:02:09.288724+0000","last_active":"2023-09-12T16:02:50.831923+0000","last_peered":"2023-09-12T16:02:50.831923+0000","last_clean":"2023-09-12T16:02:50.831923+0000","last_became_active":"2023-09-12T16:02:09.288206+0000","last_became_peered":"2023-09-12T16:02:09.288206+0000","last_unstale":"2023-09-12T16:02:50.831923+0000","last_undegraded":"2023-09-12T16:02:50.831923+0000","last_fullsized":"2023-09-12T16:02:50.831923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:08.257697+0000","last_clean_scrub_stamp":"2023-09-12T16:02:08.257697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:45:10.154195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835225+0000","last_change":"2023-09-12T16:02:04.031219+0000","last_active":"2023-09-12T16:02:50.835225+0000","last_peered":"2023-09-12T16:02:50.835225+0000","last_clean":"2023-09-12T16:02:50.835225+0000","last_became_active":"2023-09-12T16:02:04.030867+0000","last_became_peered":"2023-09-12T16:02:04.030867+0000","last_unstale":"2023-09-12T16:02:50.835225+0000","last_undegraded":"2023-09-12T16:02:50.835225+0000","last_fullsized":"2023-09-12T16:02:50.835225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:02.995592+0000","last_clean_scrub_stamp":"2023-09-12T16:02:02.995592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:16.885094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.831081+0000","last_change":"2023-09-12T16:02:05.028004+0000","last_active":"2023-09-12T16:02:50.831081+0000","last_peered":"2023-09-12T16:02:50.831081+0000","last_clean":"2023-09-12T16:02:50.831081+0000","last_became_active":"2023-09-12T16:02:05.027668+0000","last_became_peered":"2023-09-12T16:02:05.027668+0000","last_unstale":"2023-09-12T16:02:50.831081+0000","last_undegraded":"2023-09-12T16:02:50.831081+0000","last_fullsized":"2023-09-12T16:02:50.831081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:41:23.019243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025479+0000","last_change":"2023-09-12T16:02:05.034380+0000","last_active":"2023-09-12T16:02:51.025479+0000","last_peered":"2023-09-12T16:02:51.025479+0000","last_clean":"2023-09-12T16:02:51.025479+0000","last_became_active":"2023-09-12T16:02:05.034022+0000","last_became_peered":"2023-09-12T16:02:05.034022+0000","last_unstale":"2023-09-12T16:02:51.025479+0000","last_undegraded":"2023-09-12T16:02:51.025479+0000","last_fullsized":"2023-09-12T16:02:51.025479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:55:56.404742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835155+0000","last_change":"2023-09-12T16:02:05.031723+0000","last_active":"2023-09-12T16:02:50.835155+0000","last_peered":"2023-09-12T16:02:50.835155+0000","last_clean":"2023-09-12T16:02:50.835155+0000","last_became_active":"2023-09-12T16:02:05.030954+0000","last_became_peered":"2023-09-12T16:02:05.030954+0000","last_unstale":"2023-09-12T16:02:50.835155+0000","last_undegraded":"2023-09-12T16:02:50.835155+0000","last_fullsized":"2023-09-12T16:02:50.835155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.319954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.834938+0000","last_change":"2023-09-12T16:02:05.034851+0000","last_active":"2023-09-12T16:02:50.834938+0000","last_peered":"2023-09-12T16:02:50.834938+0000","last_clean":"2023-09-12T16:02:50.834938+0000","last_became_active":"2023-09-12T16:02:05.030733+0000","last_became_peered":"2023-09-12T16:02:05.030733+0000","last_unstale":"2023-09-12T16:02:50.834938+0000","last_undegraded":"2023-09-12T16:02:50.834938+0000","last_fullsized":"2023-09-12T16:02:50.834938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:33.605216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.830111+0000","last_change":"2023-09-12T16:02:05.037331+0000","last_active":"2023-09-12T16:02:50.830111+0000","last_peered":"2023-09-12T16:02:50.830111+0000","last_clean":"2023-09-12T16:02:50.830111+0000","last_became_active":"2023-09-12T16:02:05.037015+0000","last_became_peered":"2023-09-12T16:02:05.037015+0000","last_unstale":"2023-09-12T16:02:50.830111+0000","last_undegraded":"2023-09-12T16:02:50.830111+0000","last_fullsized":"2023-09-12T16:02:50.830111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:29:14.035891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835319+0000","last_change":"2023-09-12T16:02:05.039803+0000","last_active":"2023-09-12T16:02:50.835319+0000","last_peered":"2023-09-12T16:02:50.835319+0000","last_clean":"2023-09-12T16:02:50.835319+0000","last_became_active":"2023-09-12T16:02:05.039390+0000","last_became_peered":"2023-09-12T16:02:05.039390+0000","last_unstale":"2023-09-12T16:02:50.835319+0000","last_undegraded":"2023-09-12T16:02:50.835319+0000","last_fullsized":"2023-09-12T16:02:50.835319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:12:22.422090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025700+0000","last_change":"2023-09-12T16:02:05.033153+0000","last_active":"2023-09-12T16:02:51.025700+0000","last_peered":"2023-09-12T16:02:51.025700+0000","last_clean":"2023-09-12T16:02:51.025700+0000","last_became_active":"2023-09-12T16:02:05.032800+0000","last_became_peered":"2023-09-12T16:02:05.032800+0000","last_unstale":"2023-09-12T16:02:51.025700+0000","last_undegraded":"2023-09-12T16:02:51.025700+0000","last_fullsized":"2023-09-12T16:02:51.025700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:31:43.090031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.836428+0000","last_change":"2023-09-12T16:02:05.031519+0000","last_active":"2023-09-12T16:02:50.836428+0000","last_peered":"2023-09-12T16:02:50.836428+0000","last_clean":"2023-09-12T16:02:50.836428+0000","last_became_active":"2023-09-12T16:02:05.030227+0000","last_became_peered":"2023-09-12T16:02:05.030227+0000","last_unstale":"2023-09-12T16:02:50.836428+0000","last_undegraded":"2023-09-12T16:02:50.836428+0000","last_fullsized":"2023-09-12T16:02:50.836428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:30:00.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.833413+0000","last_change":"2023-09-12T16:02:05.031763+0000","last_active":"2023-09-12T16:02:50.833413+0000","last_peered":"2023-09-12T16:02:50.833413+0000","last_clean":"2023-09-12T16:02:50.833413+0000","last_became_active":"2023-09-12T16:02:05.031441+0000","last_became_peered":"2023-09-12T16:02:05.031441+0000","last_unstale":"2023-09-12T16:02:50.833413+0000","last_undegraded":"2023-09-12T16:02:50.833413+0000","last_fullsized":"2023-09-12T16:02:50.833413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:55.867251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832249+0000","last_change":"2023-09-12T16:02:05.033786+0000","last_active":"2023-09-12T16:02:50.832249+0000","last_peered":"2023-09-12T16:02:50.832249+0000","last_clean":"2023-09-12T16:02:50.832249+0000","last_became_active":"2023-09-12T16:02:05.033497+0000","last_became_peered":"2023-09-12T16:02:05.033497+0000","last_unstale":"2023-09-12T16:02:50.832249+0000","last_undegraded":"2023-09-12T16:02:50.832249+0000","last_fullsized":"2023-09-12T16:02:50.832249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:01:58.912488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.832560+0000","last_change":"2023-09-12T16:02:05.031144+0000","last_active":"2023-09-12T16:02:50.832560+0000","last_peered":"2023-09-12T16:02:50.832560+0000","last_clean":"2023-09-12T16:02:50.832560+0000","last_became_active":"2023-09-12T16:02:05.030827+0000","last_became_peered":"2023-09-12T16:02:05.030827+0000","last_unstale":"2023-09-12T16:02:50.832560+0000","last_undegraded":"2023-09-12T16:02:50.832560+0000","last_fullsized":"2023-09-12T16:02:50.832560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:40:48.014344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.837378+0000","last_change":"2023-09-12T16:02:05.021273+0000","last_active":"2023-09-12T16:02:50.837378+0000","last_peered":"2023-09-12T16:02:50.837378+0000","last_clean":"2023-09-12T16:02:50.837378+0000","last_became_active":"2023-09-12T16:02:05.020953+0000","last_became_peered":"2023-09-12T16:02:05.020953+0000","last_unstale":"2023-09-12T16:02:50.837378+0000","last_undegraded":"2023-09-12T16:02:50.837378+0000","last_fullsized":"2023-09-12T16:02:50.837378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:33:25.462412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:50.835081+0000","last_change":"2023-09-12T16:02:05.039019+0000","last_active":"2023-09-12T16:02:50.835081+0000","last_peered":"2023-09-12T16:02:50.835081+0000","last_clean":"2023-09-12T16:02:50.835081+0000","last_became_active":"2023-09-12T16:02:05.038621+0000","last_became_peered":"2023-09-12T16:02:05.038621+0000","last_unstale":"2023-09-12T16:02:50.835081+0000","last_undegraded":"2023-09-12T16:02:50.835081+0000","last_fullsized":"2023-09-12T16:02:50.835081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:08:08.934375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-12T16:02:51.025761+0000","last_change":"2023-09-12T16:02:05.039737+0000","last_active":"2023-09-12T16:02:51.025761+0000","last_peered":"2023-09-12T16:02:51.025761+0000","last_clean":"2023-09-12T16:02:51.025761+0000","last_became_active":"2023-09-12T16:02:05.039304+0000","last_became_peered":"2023-09-12T16:02:05.039304+0000","last_unstale":"2023-09-12T16:02:51.025761+0000","last_undegraded":"2023-09-12T16:02:51.025761+0000","last_fullsized":"2023-09-12T16:02:51.025761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T16:02:04.000649+0000","last_clean_scrub_stamp":"2023-09-12T16:02:04.000649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:50:13.141915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244717,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69913,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":489,"num_read_kb":318,"num_write":158,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":59815,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":201417,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":489432,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.876}]},{"osd":1,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.755,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80600000000000005}]},{"osd":2,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.5,"5min":0.496,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77600000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78200000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80400000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.85699999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.54900000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89900000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71899999999999997,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.84799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.89100000000000001}]},{"osd":8,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75900000000000001}]},{"osd":9,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.65200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.621,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.84199999999999997}]},{"osd":10,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.61399999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.98199999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.93000000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776163,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":1142931,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.5,"15min":0.501},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.503},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80000000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.752},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82899999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.58999999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.90800000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.79300000000000004}]},{"osd":6,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95299999999999996,"5min":1.042,"15min":1.042},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.56499999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.85099999999999998,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.80700000000000005}]},{"osd":8,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.63700000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.008,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.875},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93400000000000005,"5min":1.073,"15min":1.073},"last":0.86599999999999999}]},{"osd":9,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96299999999999997,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.099,"15min":1.099},"last":0.81999999999999995}]},{"osd":11,"last update":"Tue Sep 12 16:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.79100000000000004,"15min":0.78400000000000003},"min":{"1min":0.68899999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.66000000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.89500000000000002}]}]},{"osd":9,"up_from":63,"seq":270582939688,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":493764,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.48599999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.42799999999999999,"15min":0.436},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.82499999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.372}]},{"osd":1,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.312}]},{"osd":2,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70499999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.46500000000000002,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.72099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81799999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996}]},{"osd":6,"last update":"Tue Sep 12 16:01:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66300000000000003,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.495},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88700000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.41799999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.70099999999999996,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.80800000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.621}]},{"osd":10,"last update":"Tue Sep 12 16:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.69999999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46000000000000002}]},{"osd":11,"last update":"Tue Sep 12 16:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61199999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168620,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":488675,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66100000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55100000000000005}]},{"osd":1,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.63,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.57399999999999995,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71499999999999997}]},{"osd":2,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.44400000000000001},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.48399999999999999,"15min":0.45400000000000001},"min":{"1min":0.34799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.47399999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":2.5710000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.95999999999999996}]},{"osd":4,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68400000000000005,"15min":0.65500000000000003},"min":{"1min":0.58199999999999996,"5min":0.373,"15min":0.373},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67600000000000005,"15min":0.65400000000000003},"min":{"1min":0.54000000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.012,"15min":1.012},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71399999999999997,"15min":0.69099999999999995},"min":{"1min":0.50600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0469999999999999,"5min":1.075,"15min":1.075},"last":0.80500000000000005}]},{"osd":7,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.76300000000000001,"15min":0.73599999999999999},"min":{"1min":0.69199999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.872},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.56499999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.75800000000000001}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.84799999999999998}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71899999999999997,"15min":0.70299999999999996},"min":{"1min":0.60199999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.67500000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92600000000000005}]},{"osd":11,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.88600000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1085873,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.40200000000000002,"15min":0.38600000000000001},"min":{"1min":0.34200000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63600000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.433,"15min":0.40699999999999997},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.505}]},{"osd":2,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59099999999999997,"15min":0.55800000000000005},"min":{"1min":0.439,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1639999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0640000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.78200000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.623}]},{"osd":4,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.91200000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.35099999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72599999999999998,"5min":1.069,"15min":1.069},"last":0.749}]},{"osd":5,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65100000000000002,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93999999999999995,"5min":1.909,"15min":1.909},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65000000000000002,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.139,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.90500000000000003}]},{"osd":6,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96299999999999997,"5min":1.119,"15min":1.119},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84699999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.85099999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0169999999999999,"5min":1.879,"15min":1.879},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98499999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.998}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96499999999999997,"5min":1.329,"15min":1.329},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.0309999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.59799999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.872}]},{"osd":11,"last update":"Tue Sep 12 16:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.97599999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":484844,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.39200000000000002,"15min":0.39000000000000001},"min":{"1min":0.25600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56999999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.35899999999999999}]},{"osd":1,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70099999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.69799999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71899999999999997,"5min":1.079,"15min":1.079},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":1.129,"15min":1.129},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.75,"5min":1.083,"15min":1.083},"last":0.439},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.68300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72699999999999998,"5min":1.161,"15min":1.161},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67800000000000005}]},{"osd":5,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81999999999999995,"5min":1.022,"15min":1.022},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.65300000000000002}]},{"osd":8,"last update":"Tue Sep 12 16:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68400000000000005,"15min":0.70399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83399999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.628}]},{"osd":9,"last update":"Tue Sep 12 16:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71799999999999997,"15min":0.72899999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.495}]},{"osd":10,"last update":"Tue Sep 12 16:03:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90800000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60499999999999998}]},{"osd":11,"last update":"Tue Sep 12 16:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":545948,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.38100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":6.3369999999999997}]},{"osd":2,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.61099999999999999,"15min":0.54200000000000004},"min":{"1min":0.55000000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.628}]},{"osd":3,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57099999999999995,"15min":0.51900000000000002},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":1.002,"15min":1.002},"last":6.5170000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.498,"15min":0.45600000000000002},"min":{"1min":0.40899999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.435},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.57499999999999996,"15min":0.51900000000000002},"min":{"1min":0.52400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.69999999999999996,"15min":0.65200000000000002},"min":{"1min":0.52100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.036,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Sep 12 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.66900000000000004,"15min":0.623},"min":{"1min":0.63,"5min":0.317,"15min":0.317},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.68100000000000005,"15min":0.63700000000000001},"min":{"1min":0.70499999999999996,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60099999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.77300000000000002,"15min":0.73699999999999999},"min":{"1min":0.59599999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1399999999999999,"5min":1.167,"15min":1.167},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69499999999999995,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73299999999999998}]},{"osd":9,"last update":"Tue Sep 12 16:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81000000000000005,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.81999999999999995,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.88900000000000001}]},{"osd":10,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.67300000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77000000000000002}]},{"osd":11,"last update":"Tue Sep 12 16:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.81499999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":486232,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58099999999999996,"15min":0.60599999999999998},"min":{"1min":0.38300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82999999999999996,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.48699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":2.113,"15min":2.113},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.52100000000000002},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79300000000000004,"5min":0.874,"15min":0.874},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76600000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.441}]},{"osd":3,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.308},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.33100000000000002}]},{"osd":4,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76600000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.74399999999999999,"5min":0.879,"15min":0.879},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":1.004,"15min":1.004},"last":0.41699999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59699999999999998}]},{"osd":8,"last update":"Tue Sep 12 16:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.53500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.66000000000000003,"15min":0.67600000000000005},"min":{"1min":0.47099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.501}]},{"osd":9,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.873,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58099999999999996}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.028,"15min":1.028},"last":0.65700000000000003}]},{"osd":11,"last update":"Tue Sep 12 16:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.59599999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485552,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.377,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79600000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.437},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.499,"15min":0.47899999999999998},"min":{"1min":0.42299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.57999999999999996,"15min":0.51200000000000001},"min":{"1min":0.432,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90600000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.61299999999999999,"15min":0.52400000000000002},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.878}]},{"osd":3,"last update":"Tue Sep 12 16:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.89700000000000002}]},{"osd":5,"last update":"Tue Sep 12 16:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.53300000000000003,"15min":0.48699999999999999},"min":{"1min":0.35399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.875,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.56200000000000006,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85099999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Sep 12 16:03:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73599999999999999}]},{"osd":8,"last update":"Tue Sep 12 16:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":1.05,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65100000000000002,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.747}]},{"osd":9,"last update":"Tue Sep 12 16:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.81399999999999995}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.60799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.747,"15min":0.72099999999999997},"min":{"1min":0.64400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.95599999999999996}]},{"osd":11,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.93100000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":485038,"data_compressed":20605,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.34999999999999998},"min":{"1min":0.27000000000000002,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.167,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.26700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.317}]},{"osd":1,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82299999999999995,"5min":3.552,"15min":3.552},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.752,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56599999999999995,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86399999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76300000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.44500000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78100000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72799999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56699999999999995}]},{"osd":5,"last update":"Tue Sep 12 16:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82899999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.46999999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47599999999999998},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65000000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.36599999999999999}]},{"osd":7,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.71599999999999997,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":7.6929999999999996,"5min":7.6929999999999996,"15min":7.6929999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.50900000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Sep 12 16:01:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66100000000000003}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.083,"15min":1.083},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.75,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.45400000000000001}]},{"osd":10,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.84899999999999998,"5min":0.875,"15min":0.875},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53900000000000003}]},{"osd":11,"last update":"Tue Sep 12 16:03:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58899999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":550496,"data_compressed":30118,"data_compressed_allocated":40960,"data_compressed_original":114688,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:01:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.46999999999999997},"min":{"1min":0.38500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.45800000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88800000000000001,"5min":1.458,"15min":1.458},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.36299999999999999},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.41799999999999998,"15min":0.374},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.436}]},{"osd":2,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Sep 12 16:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58199999999999996,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.316,"15min":0.316},"max":{"1min":1.0249999999999999,"5min":1.383,"15min":1.383},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52900000000000003,"15min":0.49399999999999999},"min":{"1min":0.38600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41199999999999998}]},{"osd":5,"last update":"Tue Sep 12 16:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.66300000000000003,"15min":0.626},"min":{"1min":0.55100000000000005,"5min":0.379,"15min":0.379},"max":{"1min":0.92100000000000004,"5min":1.54,"15min":1.54},"last":0.752},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.52600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90800000000000003,"5min":1.502,"15min":1.502},"last":0.84599999999999997}]},{"osd":6,"last update":"Tue Sep 12 16:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.496,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93999999999999995,"5min":1.639,"15min":1.639},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53400000000000003}]},{"osd":8,"last update":"Tue Sep 12 16:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.58799999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.80800000000000005,"15min":0.79600000000000004},"min":{"1min":0.67400000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.018,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.87}]},{"osd":9,"last update":"Tue Sep 12 16:02:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97499999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.622}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.755,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.93000000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1,"5min":1,"15min":1},"last":0.79100000000000004}]},{"osd":11,"last update":"Tue Sep 12 16:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.82299999999999995}]}]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1080753,"data_compressed":56267,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.48099999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.376},"min":{"1min":0.29599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.38},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.32400000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40300000000000002}]},{"osd":3,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53000000000000003,"15min":0.54700000000000004},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.495},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.41499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.47399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.433,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81200000000000006}]},{"osd":6,"last update":"Tue Sep 12 16:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.62},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.879,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.57499999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72099999999999997}]},{"osd":8,"last update":"Tue Sep 12 16:03:24 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48899999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58699999999999997}]},{"osd":9,"last update":"Tue Sep 12 16:02:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.752}]},{"osd":10,"last update":"Tue Sep 12 16:03:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.56899999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35662,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":61919,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T16:03:48.794 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:48.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.946+0000 7fcf5beee700 1 -- 172.21.15.130:0/3108322518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf541000f0 msgr2=0x7fcf54107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.946+0000 7fcf5beee700 1 --2- 172.21.15.130:0/3108322518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf541000f0 0x7fcf54107990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00b0a0 tx=0x7fcf4c00b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.946+0000 7fcf5beee700 1 -- 172.21.15.130:0/3108322518 shutdown_connections 2023-09-12T16:03:48.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.946+0000 7fcf5beee700 1 --2- 172.21.15.130:0/3108322518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf541000f0 0x7fcf54107990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.946+0000 7fcf5beee700 1 --2- 172.21.15.130:0/3108322518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf54100b70 0x7fcf54100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.946+0000 7fcf5beee700 1 --2- 172.21.15.130:0/3108322518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf540ff730 0x7fcf540ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.946+0000 7fcf5beee700 1 -- 172.21.15.130:0/3108322518 >> 172.21.15.130:0/3108322518 conn(0x7fcf540fa1c0 msgr2=0x7fcf540fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:48.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.946+0000 7fcf5beee700 1 -- 172.21.15.130:0/3108322518 shutdown_connections 2023-09-12T16:03:48.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.946+0000 7fcf5beee700 1 -- 172.21.15.130:0/3108322518 wait complete. 2023-09-12T16:03:48.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5beee700 1 Processor -- start 2023-09-12T16:03:48.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5beee700 1 -- start start 2023-09-12T16:03:48.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5beee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf540ff730 0x7fcf54195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5beee700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf541000f0 0x7fcf541957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5beee700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf54100b70 0x7fcf5419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5beee700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5419cdf0 con 0x7fcf54100b70 2023-09-12T16:03:48.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5beee700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5419cf30 con 0x7fcf541000f0 2023-09-12T16:03:48.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5beee700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5419d0c0 con 0x7fcf540ff730 2023-09-12T16:03:48.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5a48b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf54100b70 0x7fcf5419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5a48b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf54100b70 0x7fcf5419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55230/0 (socket says 172.21.15.130:55230) 2023-09-12T16:03:48.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5a48b700 1 -- 172.21.15.130:0/2551761034 learned_addr learned my addr 172.21.15.130:0/2551761034 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:48.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf59489700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf541000f0 0x7fcf541957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf59c8a700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf540ff730 0x7fcf54195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5a48b700 1 -- 172.21.15.130:0/2551761034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf540ff730 msgr2=0x7fcf54195270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5a48b700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf540ff730 0x7fcf54195270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5a48b700 1 -- 172.21.15.130:0/2551761034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf541000f0 msgr2=0x7fcf541957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:48.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.947+0000 7fcf5a48b700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf541000f0 0x7fcf541957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:48.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.948+0000 7fcf5a48b700 1 -- 172.21.15.130:0/2551761034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c009d00 con 0x7fcf54100b70 2023-09-12T16:03:48.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.948+0000 7fcf5a48b700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf54100b70 0x7fcf5419c820 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0062a0 tx=0x7fcf4c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:48.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.948+0000 7fcf4affd700 1 -- 172.21.15.130:0/2551761034 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00e040 con 0x7fcf54100b70 2023-09-12T16:03:48.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.948+0000 7fcf4affd700 1 -- 172.21.15.130:0/2551761034 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf4c00cec0 con 0x7fcf54100b70 2023-09-12T16:03:48.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.948+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5419d340 con 0x7fcf54100b70 2023-09-12T16:03:48.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.948+0000 7fcf4affd700 1 -- 172.21.15.130:0/2551761034 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004020 con 0x7fcf54100b70 2023-09-12T16:03:48.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.948+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5419d7c0 con 0x7fcf54100b70 2023-09-12T16:03:48.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.949+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5410ad60 con 0x7fcf54100b70 2023-09-12T16:03:48.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.950+0000 7fcf4affd700 1 -- 172.21.15.130:0/2551761034 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fcf4c01d690 con 0x7fcf54100b70 2023-09-12T16:03:48.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.950+0000 7fcf4affd700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf400a0710 0x7fcf400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:48.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.951+0000 7fcf4affd700 1 -- 172.21.15.130:0/2551761034 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcf4c0cdf60 con 0x7fcf54100b70 2023-09-12T16:03:48.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.951+0000 7fcf59c8a700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf400a0710 0x7fcf400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:48.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.951+0000 7fcf59c8a700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf400a0710 0x7fcf400a2bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcf44005950 tx=0x7fcf440058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:48.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:48.954+0000 7fcf4affd700 1 -- 172.21.15.130:0/2551761034 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcf4c0976b0 con 0x7fcf54100b70 2023-09-12T16:03:49.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.154+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf54106390 con 0x7fcf54100b70 2023-09-12T16:03:49.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.155+0000 7fcf4affd700 1 -- 172.21.15.130:0/2551761034 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24473 (secure 0 0 0) 0x7fcf4c09acd0 con 0x7fcf54100b70 2023-09-12T16:03:49.157 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:49.157 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":88,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:02:51.826597+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:49.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf400a0710 msgr2=0x7fcf400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf400a0710 0x7fcf400a2bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcf44005950 tx=0x7fcf440058e0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf54100b70 msgr2=0x7fcf5419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf54100b70 0x7fcf5419c820 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0062a0 tx=0x7fcf4c004d00 comp rx=0 tx=0).stop 2023-09-12T16:03:49.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 shutdown_connections 2023-09-12T16:03:49.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf400a0710 0x7fcf400a2bd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf54100b70 0x7fcf5419c820 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf540ff730 0x7fcf54195270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 --2- 172.21.15.130:0/2551761034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf541000f0 0x7fcf541957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.157+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 >> 172.21.15.130:0/2551761034 conn(0x7fcf540fa1c0 msgr2=0x7fcf540fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:49.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.158+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 shutdown_connections 2023-09-12T16:03:49.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.158+0000 7fcf5beee700 1 -- 172.21.15.130:0/2551761034 wait complete. 2023-09-12T16:03:49.169 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:48 smithi130 ceph-mon[98670]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:49.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:03:49.172 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:03:49.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:48 smithi159 ceph-mon[102522]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:49.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:48 smithi203 ceph-mon[103431]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:49.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.296+0000 7f6f37af8700 1 -- 172.21.15.130:0/4057131243 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30106370 msgr2=0x7f6f30108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.296+0000 7f6f37af8700 1 --2- 172.21.15.130:0/4057131243 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30106370 0x7f6f30108760 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800b0a0 tx=0x7f6f2800b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.296+0000 7f6f37af8700 1 -- 172.21.15.130:0/4057131243 shutdown_connections 2023-09-12T16:03:49.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.296+0000 7f6f37af8700 1 --2- 172.21.15.130:0/4057131243 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30106370 0x7f6f30108760 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.296+0000 7f6f37af8700 1 --2- 172.21.15.130:0/4057131243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f30101110 0x7f6f30103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.296+0000 7f6f37af8700 1 --2- 172.21.15.130:0/4057131243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f30103a40 0x7f6f30105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.296+0000 7f6f37af8700 1 -- 172.21.15.130:0/4057131243 >> 172.21.15.130:0/4057131243 conn(0x7f6f300faa60 msgr2=0x7f6f300fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:49.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.296+0000 7f6f37af8700 1 -- 172.21.15.130:0/4057131243 shutdown_connections 2023-09-12T16:03:49.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.296+0000 7f6f37af8700 1 -- 172.21.15.130:0/4057131243 wait complete. 2023-09-12T16:03:49.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f37af8700 1 Processor -- start 2023-09-12T16:03:49.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f37af8700 1 -- start start 2023-09-12T16:03:49.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f37af8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30101110 0x7f6f30196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f37af8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f30103a40 0x7f6f30196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f37af8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f30106370 0x7f6f3019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f37af8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3019dea0 con 0x7f6f30101110 2023-09-12T16:03:49.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f37af8700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3019dfe0 con 0x7f6f30103a40 2023-09-12T16:03:49.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f37af8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3019e170 con 0x7f6f30106370 2023-09-12T16:03:49.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35894700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30101110 0x7f6f30196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35894700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30101110 0x7f6f30196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55240/0 (socket says 172.21.15.130:55240) 2023-09-12T16:03:49.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35894700 1 -- 172.21.15.130:0/2072032873 learned_addr learned my addr 172.21.15.130:0/2072032873 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:49.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f36095700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f30106370 0x7f6f3019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35093700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f30103a40 0x7f6f30196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35894700 1 -- 172.21.15.130:0/2072032873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f30106370 msgr2=0x7f6f3019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35894700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f30106370 0x7f6f3019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35894700 1 -- 172.21.15.130:0/2072032873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f30103a40 msgr2=0x7f6f30196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35894700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f30103a40 0x7f6f30196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35894700 1 -- 172.21.15.130:0/2072032873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28009d00 con 0x7f6f30101110 2023-09-12T16:03:49.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35093700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f30103a40 0x7f6f30196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:49.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.297+0000 7f6f35894700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30101110 0x7f6f30196320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000e9a0 tx=0x7f6f2000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:49.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.298+0000 7f6f26ffd700 1 -- 172.21.15.130:0/2072032873 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20009790 con 0x7f6f30101110 2023-09-12T16:03:49.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.298+0000 7f6f26ffd700 1 -- 172.21.15.130:0/2072032873 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f2000cbd0 con 0x7f6f30101110 2023-09-12T16:03:49.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.298+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3019e450 con 0x7f6f30101110 2023-09-12T16:03:49.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.298+0000 7f6f26ffd700 1 -- 172.21.15.130:0/2072032873 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20018680 con 0x7f6f30101110 2023-09-12T16:03:49.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.298+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3019e960 con 0x7f6f30101110 2023-09-12T16:03:49.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.299+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f300fc5e0 con 0x7f6f30101110 2023-09-12T16:03:49.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.300+0000 7f6f26ffd700 1 -- 172.21.15.130:0/2072032873 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f6f2000cd40 con 0x7f6f30101110 2023-09-12T16:03:49.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.300+0000 7f6f26ffd700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6f1c0a06d0 0x7f6f1c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.310 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.300+0000 7f6f26ffd700 1 -- 172.21.15.130:0/2072032873 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6f20014070 con 0x7f6f30101110 2023-09-12T16:03:49.311 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.300+0000 7f6f35093700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6f1c0a06d0 0x7f6f1c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.311 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.301+0000 7f6f35093700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6f1c0a06d0 0x7f6f1c0a2b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005fd0 tx=0x7f6f2c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:49.312 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.304+0000 7f6f26ffd700 1 -- 172.21.15.130:0/2072032873 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6f20095c20 con 0x7f6f30101110 2023-09-12T16:03:49.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.494+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f3010c000 con 0x7f6f30101110 2023-09-12T16:03:49.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.494+0000 7f6f26ffd700 1 -- 172.21.15.130:0/2072032873 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f20099240 con 0x7f6f30101110 2023-09-12T16:03:49.495 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:03:49.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.496+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6f1c0a06d0 msgr2=0x7f6f1c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.496+0000 7f6f37af8700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6f1c0a06d0 0x7f6f1c0a2b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005fd0 tx=0x7f6f2c005e30 comp rx=0 tx=0).stop 2023-09-12T16:03:49.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.496+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30101110 msgr2=0x7f6f30196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.496+0000 7f6f37af8700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30101110 0x7f6f30196320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000e9a0 tx=0x7f6f2000ecb0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.497+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 shutdown_connections 2023-09-12T16:03:49.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.497+0000 7f6f37af8700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6f1c0a06d0 0x7f6f1c0a2b90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.497+0000 7f6f37af8700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f30106370 0x7f6f3019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.497+0000 7f6f37af8700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f30101110 0x7f6f30196320 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.497+0000 7f6f37af8700 1 --2- 172.21.15.130:0/2072032873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f30103a40 0x7f6f30196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.497+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 >> 172.21.15.130:0/2072032873 conn(0x7f6f300faa60 msgr2=0x7f6f30101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:49.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.497+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 shutdown_connections 2023-09-12T16:03:49.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.497+0000 7f6f37af8700 1 -- 172.21.15.130:0/2072032873 wait complete. 2023-09-12T16:03:49.509 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:03:49.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.634+0000 7f2ed6016700 1 -- 172.21.15.130:0/3701247790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 msgr2=0x7f2ed0107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.634+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3701247790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 0x7f2ed0107940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00099c0 tx=0x7f2ec0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.634+0000 7f2ed6016700 1 -- 172.21.15.130:0/3701247790 shutdown_connections 2023-09-12T16:03:49.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.634+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3701247790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 0x7f2ed0107940 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.634+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3701247790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ed00ff500 0x7f2ed00ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.634+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3701247790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed01006f0 0x7f2ed0100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.634+0000 7f2ed6016700 1 -- 172.21.15.130:0/3701247790 >> 172.21.15.130:0/3701247790 conn(0x7f2ed00faa90 msgr2=0x7f2ed00fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:49.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.634+0000 7f2ed6016700 1 -- 172.21.15.130:0/3701247790 shutdown_connections 2023-09-12T16:03:49.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.634+0000 7f2ed6016700 1 -- 172.21.15.130:0/3701247790 wait complete. 2023-09-12T16:03:49.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ed6016700 1 Processor -- start 2023-09-12T16:03:49.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ed6016700 1 -- start start 2023-09-12T16:03:49.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ed6016700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed00ff500 0x7f2ed0195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ed6016700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ed01006f0 0x7f2ed0195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ed6016700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 0x7f2ed019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ed6016700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed019cc80 con 0x7f2ed0101090 2023-09-12T16:03:49.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ed6016700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed019cdc0 con 0x7f2ed00ff500 2023-09-12T16:03:49.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ed6016700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed019cf50 con 0x7f2ed01006f0 2023-09-12T16:03:49.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ecffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 0x7f2ed019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ecffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 0x7f2ed019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55258/0 (socket says 172.21.15.130:55258) 2023-09-12T16:03:49.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ecffff700 1 -- 172.21.15.130:0/3478469787 learned_addr learned my addr 172.21.15.130:0/3478469787 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:49.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ecf7fe700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed00ff500 0x7f2ed0195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2eceffd700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ed01006f0 0x7f2ed0195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ecffff700 1 -- 172.21.15.130:0/3478469787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ed01006f0 msgr2=0x7f2ed0195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ecffff700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ed01006f0 0x7f2ed0195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ecffff700 1 -- 172.21.15.130:0/3478469787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed00ff500 msgr2=0x7f2ed0195100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ecffff700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed00ff500 0x7f2ed0195100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.635+0000 7f2ecffff700 1 -- 172.21.15.130:0/3478469787 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec00095f0 con 0x7f2ed0101090 2023-09-12T16:03:49.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.636+0000 7f2ecffff700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 0x7f2ed019c6b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009990 tx=0x7f2ec000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:49.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.636+0000 7f2eccff9700 1 -- 172.21.15.130:0/3478469787 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec001d070 con 0x7f2ed0101090 2023-09-12T16:03:49.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.636+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed019d1d0 con 0x7f2ed0101090 2023-09-12T16:03:49.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.636+0000 7f2eccff9700 1 -- 172.21.15.130:0/3478469787 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ec0005030 con 0x7f2ed0101090 2023-09-12T16:03:49.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.636+0000 7f2eccff9700 1 -- 172.21.15.130:0/3478469787 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0004ad0 con 0x7f2ed0101090 2023-09-12T16:03:49.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.636+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed019d5a0 con 0x7f2ed0101090 2023-09-12T16:03:49.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.637+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed01063e0 con 0x7f2ed0101090 2023-09-12T16:03:49.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.638+0000 7f2eccff9700 1 -- 172.21.15.130:0/3478469787 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f2ec0004c70 con 0x7f2ed0101090 2023-09-12T16:03:49.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.638+0000 7f2eccff9700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2eb80a06d0 0x7f2eb80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.638+0000 7f2eccff9700 1 -- 172.21.15.130:0/3478469787 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2ec00cceb0 con 0x7f2ed0101090 2023-09-12T16:03:49.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.638+0000 7f2ecf7fe700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2eb80a06d0 0x7f2eb80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.639+0000 7f2ecf7fe700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2eb80a06d0 0x7f2eb80a2b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0100550 tx=0x7f2ec800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:49.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.643+0000 7f2eccff9700 1 -- 172.21.15.130:0/3478469787 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2ec0096600 con 0x7f2ed0101090 2023-09-12T16:03:49.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.832+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ed002cf90 con 0x7f2ed0101090 2023-09-12T16:03:49.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.832+0000 7f2eccff9700 1 -- 172.21.15.130:0/3478469787 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ec0099c20 con 0x7f2ed0101090 2023-09-12T16:03:49.833 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:49.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2eb80a06d0 msgr2=0x7f2eb80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2eb80a06d0 0x7f2eb80a2b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0100550 tx=0x7f2ec800b040 comp rx=0 tx=0).stop 2023-09-12T16:03:49.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 msgr2=0x7f2ed019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 0x7f2ed019c6b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009990 tx=0x7f2ec000f690 comp rx=0 tx=0).stop 2023-09-12T16:03:49.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 shutdown_connections 2023-09-12T16:03:49.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2eb80a06d0 0x7f2eb80a2b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ed0101090 0x7f2ed019c6b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed00ff500 0x7f2ed0195100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 --2- 172.21.15.130:0/3478469787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ed01006f0 0x7f2ed0195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 >> 172.21.15.130:0/3478469787 conn(0x7f2ed00faa90 msgr2=0x7f2ed00fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:49.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 shutdown_connections 2023-09-12T16:03:49.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.834+0000 7f2ed6016700 1 -- 172.21.15.130:0/3478469787 wait complete. 2023-09-12T16:03:49.847 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:03:49.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.972+0000 7f377b90e700 1 -- 172.21.15.130:0/158920063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 msgr2=0x7f37740ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.972+0000 7f377b90e700 1 --2- 172.21.15.130:0/158920063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 0x7f37740ffa80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3768009990 tx=0x7f3768009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.972+0000 7f377b90e700 1 -- 172.21.15.130:0/158920063 shutdown_connections 2023-09-12T16:03:49.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.972+0000 7f377b90e700 1 --2- 172.21.15.130:0/158920063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37740fffc0 0x7f3774104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.972+0000 7f377b90e700 1 --2- 172.21.15.130:0/158920063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37740fe8a0 0x7f37740fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.972+0000 7f377b90e700 1 --2- 172.21.15.130:0/158920063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 0x7f37740ffa80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.972+0000 7f377b90e700 1 -- 172.21.15.130:0/158920063 >> 172.21.15.130:0/158920063 conn(0x7f37740fa1d0 msgr2=0x7f37740fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:49.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.973+0000 7f377b90e700 1 -- 172.21.15.130:0/158920063 shutdown_connections 2023-09-12T16:03:49.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.973+0000 7f377b90e700 1 -- 172.21.15.130:0/158920063 wait complete. 2023-09-12T16:03:49.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.973+0000 7f377b90e700 1 Processor -- start 2023-09-12T16:03:49.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.973+0000 7f377b90e700 1 -- start start 2023-09-12T16:03:49.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.973+0000 7f377b90e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37740fe8a0 0x7f377419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f377b90e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 0x7f377419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f377b90e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37740fffc0 0x7f37741a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f377b90e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37741a4450 con 0x7f37740ff600 2023-09-12T16:03:49.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f377b90e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37741a4590 con 0x7f37740fffc0 2023-09-12T16:03:49.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f377b90e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37741a4720 con 0x7f37740fe8a0 2023-09-12T16:03:49.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3778ea9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 0x7f377419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3778ea9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 0x7f377419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55268/0 (socket says 172.21.15.130:55268) 2023-09-12T16:03:49.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3778ea9700 1 -- 172.21.15.130:0/4021674340 learned_addr learned my addr 172.21.15.130:0/4021674340 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:49.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f37796aa700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37740fe8a0 0x7f377419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3778ea9700 1 -- 172.21.15.130:0/4021674340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37740fe8a0 msgr2=0x7f377419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3778ea9700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37740fe8a0 0x7f377419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3779eab700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37740fffc0 0x7f37741a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3778ea9700 1 -- 172.21.15.130:0/4021674340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37740fffc0 msgr2=0x7f37741a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:49.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3778ea9700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37740fffc0 0x7f37741a3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:49.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3778ea9700 1 -- 172.21.15.130:0/4021674340 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680095f0 con 0x7f37740ff600 2023-09-12T16:03:49.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f3778ea9700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 0x7f377419ce10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3768005910 tx=0x7f376800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:49.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f37667fc700 1 -- 172.21.15.130:0/4021674340 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376801d070 con 0x7f37740ff600 2023-09-12T16:03:49.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37741a49a0 con 0x7f37740ff600 2023-09-12T16:03:49.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f37667fc700 1 -- 172.21.15.130:0/4021674340 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376800fb80 con 0x7f37740ff600 2023-09-12T16:03:49.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f37667fc700 1 -- 172.21.15.130:0/4021674340 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768017650 con 0x7f37740ff600 2023-09-12T16:03:49.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.974+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37741a4dc0 con 0x7f37740ff600 2023-09-12T16:03:49.985 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.975+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3774103fd0 con 0x7f37740ff600 2023-09-12T16:03:49.986 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.976+0000 7f37667fc700 1 -- 172.21.15.130:0/4021674340 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f376800fcf0 con 0x7f37740ff600 2023-09-12T16:03:49.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.977+0000 7f37667fc700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f37600a06d0 0x7f37600a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:49.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.977+0000 7f37796aa700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f37600a06d0 0x7f37600a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:49.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.977+0000 7f37667fc700 1 -- 172.21.15.130:0/4021674340 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f37680cd7d0 con 0x7f37740ff600 2023-09-12T16:03:49.987 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.977+0000 7f37796aa700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f37600a06d0 0x7f37600a2b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f377000ac50 tx=0x7f377000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:49.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:49.980+0000 7f37667fc700 1 -- 172.21.15.130:0/4021674340 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3768096ec0 con 0x7f37740ff600 2023-09-12T16:03:50.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.173+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37740008d0 con 0x7f37740ff600 2023-09-12T16:03:50.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.173+0000 7f37667fc700 1 -- 172.21.15.130:0/4021674340 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f376809a4e0 con 0x7f37740ff600 2023-09-12T16:03:50.174 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:03:50.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f37600a06d0 msgr2=0x7f37600a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f37600a06d0 0x7f37600a2b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f377000ac50 tx=0x7f377000a380 comp rx=0 tx=0).stop 2023-09-12T16:03:50.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 msgr2=0x7f377419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 0x7f377419ce10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3768005910 tx=0x7f376800f690 comp rx=0 tx=0).stop 2023-09-12T16:03:50.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 shutdown_connections 2023-09-12T16:03:50.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f37600a06d0 0x7f37600a2b90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37740fffc0 0x7f37741a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37740fe8a0 0x7f377419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 --2- 172.21.15.130:0/4021674340 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37740ff600 0x7f377419ce10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.175+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 >> 172.21.15.130:0/4021674340 conn(0x7f37740fa1d0 msgr2=0x7f37740fae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:50.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.176+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 shutdown_connections 2023-09-12T16:03:50.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.176+0000 7f377b90e700 1 -- 172.21.15.130:0/4021674340 wait complete. 2023-09-12T16:03:50.187 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:49 smithi130 ceph-mon[98670]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:50.187 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:49 smithi130 ceph-mon[98670]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:50.188 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:49 smithi130 ceph-mon[98670]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2023-09-12T16:03:50.188 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:49 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2551761034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:50.188 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:49 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2072032873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:50.188 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:49 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3478469787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:50.189 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:03:50.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:49 smithi159 ceph-mon[102522]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:50.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:49 smithi159 ceph-mon[102522]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:50.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:49 smithi159 ceph-mon[102522]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2023-09-12T16:03:50.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:49 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2551761034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:50.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:49 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2072032873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:50.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:49 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3478469787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:49 smithi203 ceph-mon[103431]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:50.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:49 smithi203 ceph-mon[103431]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-12T16:03:50.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:49 smithi203 ceph-mon[103431]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2023-09-12T16:03:50.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:49 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2551761034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:50.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:49 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2072032873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:03:50.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:49 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3478469787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:03:50.316 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 -- 172.21.15.130:0/1551118503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1280101080 msgr2=0x7f1280107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 --2- 172.21.15.130:0/1551118503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1280101080 0x7f1280107930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f127800b0d0 tx=0x7f127800b3e0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 -- 172.21.15.130:0/1551118503 shutdown_connections 2023-09-12T16:03:50.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 --2- 172.21.15.130:0/1551118503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1280101080 0x7f1280107930 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 --2- 172.21.15.130:0/1551118503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12800ff4f0 0x7f12800ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 --2- 172.21.15.130:0/1551118503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12801006e0 0x7f1280100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 -- 172.21.15.130:0/1551118503 >> 172.21.15.130:0/1551118503 conn(0x7f12800faa60 msgr2=0x7f12800fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:50.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 -- 172.21.15.130:0/1551118503 shutdown_connections 2023-09-12T16:03:50.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 -- 172.21.15.130:0/1551118503 wait complete. 2023-09-12T16:03:50.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 Processor -- start 2023-09-12T16:03:50.318 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.314+0000 7f1286560700 1 -- start start 2023-09-12T16:03:50.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f1286560700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12800ff4f0 0x7f1280103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:50.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f1286560700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12801006e0 0x7f1280101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:50.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f1286560700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1280101080 0x7f1280102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:50.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f1286560700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801027a0 con 0x7f12800ff4f0 2023-09-12T16:03:50.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f1286560700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1280102910 con 0x7f12801006e0 2023-09-12T16:03:50.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f1286560700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801a0430 con 0x7f1280101080 2023-09-12T16:03:50.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12800ff4f0 0x7f1280103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:50.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12800ff4f0 0x7f1280103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55280/0 (socket says 172.21.15.130:55280) 2023-09-12T16:03:50.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127ffff700 1 -- 172.21.15.130:0/3052744028 learned_addr learned my addr 172.21.15.130:0/3052744028 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:50.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127f7fe700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12801006e0 0x7f1280101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:50.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f1284afd700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1280101080 0x7f1280102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:50.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127ffff700 1 -- 172.21.15.130:0/3052744028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1280101080 msgr2=0x7f1280102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127ffff700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1280101080 0x7f1280102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127ffff700 1 -- 172.21.15.130:0/3052744028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12801006e0 msgr2=0x7f1280101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127ffff700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12801006e0 0x7f1280101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127ffff700 1 -- 172.21.15.130:0/3052744028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278009d00 con 0x7f12800ff4f0 2023-09-12T16:03:50.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127ffff700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12800ff4f0 0x7f1280103480 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f127400c8f0 tx=0x7f127400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:50.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127d7fa700 1 -- 172.21.15.130:0/3052744028 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274004220 con 0x7f12800ff4f0 2023-09-12T16:03:50.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127d7fa700 1 -- 172.21.15.130:0/3052744028 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1274004380 con 0x7f12800ff4f0 2023-09-12T16:03:50.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12801a0710 con 0x7f12800ff4f0 2023-09-12T16:03:50.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.315+0000 7f127d7fa700 1 -- 172.21.15.130:0/3052744028 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274003b40 con 0x7f12800ff4f0 2023-09-12T16:03:50.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.316+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12801a0c20 con 0x7f12800ff4f0 2023-09-12T16:03:50.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.316+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128010ad20 con 0x7f12800ff4f0 2023-09-12T16:03:50.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.318+0000 7f127d7fa700 1 -- 172.21.15.130:0/3052744028 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f1274003ce0 con 0x7f12800ff4f0 2023-09-12T16:03:50.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.318+0000 7f127d7fa700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f12680a0750 0x7f12680a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:50.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.319+0000 7f127f7fe700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f12680a0750 0x7f12680a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:50.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.319+0000 7f127d7fa700 1 -- 172.21.15.130:0/3052744028 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f12740cbf20 con 0x7f12800ff4f0 2023-09-12T16:03:50.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.319+0000 7f127f7fe700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f12680a0750 0x7f12680a2c10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1270005950 tx=0x7f127000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:50.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.322+0000 7f127d7fa700 1 -- 172.21.15.130:0/3052744028 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1274095660 con 0x7f12800ff4f0 2023-09-12T16:03:50.514 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.513+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f12801a0ea0 con 0x7f12800ff4f0 2023-09-12T16:03:50.514 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.513+0000 7f127d7fa700 1 -- 172.21.15.130:0/3052744028 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f127401d0a0 con 0x7f12800ff4f0 2023-09-12T16:03:50.515 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:03:50.516 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.515+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f12680a0750 msgr2=0x7f12680a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.515+0000 7f1286560700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f12680a0750 0x7f12680a2c10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1270005950 tx=0x7f127000b410 comp rx=0 tx=0).stop 2023-09-12T16:03:50.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.515+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12800ff4f0 msgr2=0x7f1280103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.515+0000 7f1286560700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12800ff4f0 0x7f1280103480 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f127400c8f0 tx=0x7f127400cc00 comp rx=0 tx=0).stop 2023-09-12T16:03:50.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.516+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 shutdown_connections 2023-09-12T16:03:50.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.516+0000 7f1286560700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f12680a0750 0x7f12680a2c10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.516+0000 7f1286560700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1280101080 0x7f1280102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.516+0000 7f1286560700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12800ff4f0 0x7f1280103480 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.516+0000 7f1286560700 1 --2- 172.21.15.130:0/3052744028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12801006e0 0x7f1280101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.516+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 >> 172.21.15.130:0/3052744028 conn(0x7f12800faa60 msgr2=0x7f1280067760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:50.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.516+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 shutdown_connections 2023-09-12T16:03:50.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.516+0000 7f1286560700 1 -- 172.21.15.130:0/3052744028 wait complete. 2023-09-12T16:03:50.529 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:03:50.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 -- 172.21.15.130:0/108257571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc0ff4e0 msgr2=0x7f48cc0ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 --2- 172.21.15.130:0/108257571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc0ff4e0 0x7f48cc0ff900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f48c00099c0 tx=0x7f48c0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 -- 172.21.15.130:0/108257571 shutdown_connections 2023-09-12T16:03:50.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 --2- 172.21.15.130:0/108257571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48cc101070 0x7f48cc107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 --2- 172.21.15.130:0/108257571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc0ff4e0 0x7f48cc0ff900 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 --2- 172.21.15.130:0/108257571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48cc1006d0 0x7f48cc100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 -- 172.21.15.130:0/108257571 >> 172.21.15.130:0/108257571 conn(0x7f48cc0faa30 msgr2=0x7f48cc0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:50.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 -- 172.21.15.130:0/108257571 shutdown_connections 2023-09-12T16:03:50.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 -- 172.21.15.130:0/108257571 wait complete. 2023-09-12T16:03:50.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 Processor -- start 2023-09-12T16:03:50.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.649+0000 7f48d29da700 1 -- start start 2023-09-12T16:03:50.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48d29da700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48cc0ff4e0 0x7f48cc068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:50.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48d29da700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc1006d0 0x7f48cc19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:50.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48d29da700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48cc101070 0x7f48cc19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:50.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48d29da700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc19dd70 con 0x7f48cc1006d0 2023-09-12T16:03:50.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48d29da700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc19dee0 con 0x7f48cc101070 2023-09-12T16:03:50.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48d29da700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc19e0a0 con 0x7f48cc0ff4e0 2023-09-12T16:03:50.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cb7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc1006d0 0x7f48cc19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:50.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cb7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc1006d0 0x7f48cc19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55296/0 (socket says 172.21.15.130:55296) 2023-09-12T16:03:50.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cb7fe700 1 -- 172.21.15.130:0/787666033 learned_addr learned my addr 172.21.15.130:0/787666033 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:50.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48d0f77700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48cc101070 0x7f48cc19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:50.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cbfff700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48cc0ff4e0 0x7f48cc068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:50.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cb7fe700 1 -- 172.21.15.130:0/787666033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48cc0ff4e0 msgr2=0x7f48cc068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cb7fe700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48cc0ff4e0 0x7f48cc068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cb7fe700 1 -- 172.21.15.130:0/787666033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48cc101070 msgr2=0x7f48cc19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cb7fe700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48cc101070 0x7f48cc19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cb7fe700 1 -- 172.21.15.130:0/787666033 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00095f0 con 0x7f48cc1006d0 2023-09-12T16:03:50.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cbfff700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48cc0ff4e0 0x7f48cc068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:50.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.650+0000 7f48cb7fe700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc1006d0 0x7f48cc19af40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d950 tx=0x7f48bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:50.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.651+0000 7f48c97fa700 1 -- 172.21.15.130:0/787666033 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc009940 con 0x7f48cc1006d0 2023-09-12T16:03:50.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.651+0000 7f48c97fa700 1 -- 172.21.15.130:0/787666033 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48bc010460 con 0x7f48cc1006d0 2023-09-12T16:03:50.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.651+0000 7f48c97fa700 1 -- 172.21.15.130:0/787666033 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00f620 con 0x7f48cc1006d0 2023-09-12T16:03:50.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.651+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc19e380 con 0x7f48cc1006d0 2023-09-12T16:03:50.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.651+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc19e840 con 0x7f48cc1006d0 2023-09-12T16:03:50.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.652+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc10acf0 con 0x7f48cc1006d0 2023-09-12T16:03:50.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.653+0000 7f48c97fa700 1 -- 172.21.15.130:0/787666033 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f48bc0108d0 con 0x7f48cc1006d0 2023-09-12T16:03:50.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.653+0000 7f48c97fa700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48b40a0710 0x7f48b40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:50.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.653+0000 7f48cbfff700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48b40a0710 0x7f48b40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:50.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.653+0000 7f48c97fa700 1 -- 172.21.15.130:0/787666033 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f48bc0cbe90 con 0x7f48cc1006d0 2023-09-12T16:03:50.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.654+0000 7f48cbfff700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48b40a0710 0x7f48b40a2bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f48c000b580 tx=0x7f48c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:50.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.658+0000 7f48c97fa700 1 -- 172.21.15.130:0/787666033 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f48bc095550 con 0x7f48cc1006d0 2023-09-12T16:03:50.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.921+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48cc106450 con 0x7f48cc1006d0 2023-09-12T16:03:50.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.922+0000 7f48c97fa700 1 -- 172.21.15.130:0/787666033 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11342 (secure 0 0 0) 0x7f48bc016090 con 0x7f48cc1006d0 2023-09-12T16:03:50.923 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:50.923 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:02:25.646573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:03:50.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48b40a0710 msgr2=0x7f48b40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48b40a0710 0x7f48b40a2bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f48c000b580 tx=0x7f48c0005fb0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc1006d0 msgr2=0x7f48cc19af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:50.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc1006d0 0x7f48cc19af40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d950 tx=0x7f48bc00dc60 comp rx=0 tx=0).stop 2023-09-12T16:03:50.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 shutdown_connections 2023-09-12T16:03:50.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f48b40a0710 0x7f48b40a2bd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48cc101070 0x7f48cc19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48cc0ff4e0 0x7f48cc068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 --2- 172.21.15.130:0/787666033 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48cc1006d0 0x7f48cc19af40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:50.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 >> 172.21.15.130:0/787666033 conn(0x7f48cc0faa30 msgr2=0x7f48cc0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:50.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 shutdown_connections 2023-09-12T16:03:50.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:50.925+0000 7f48d29da700 1 -- 172.21.15.130:0/787666033 wait complete. 2023-09-12T16:03:50.928 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 24 2023-09-12T16:03:50.937 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:03:51.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.061+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/3250641808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8100620 msgr2=0x7fb0b8100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.061+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/3250641808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8100620 0x7fb0b8100a40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80099c0 tx=0x7fb0a8009cd0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.061+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/3250641808 shutdown_connections 2023-09-12T16:03:51.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.061+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/3250641808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81021b0 0x7fb0b8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.061+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/3250641808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8100620 0x7fb0b8100a40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.061+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/3250641808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b8101810 0x7fb0b8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.061+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/3250641808 >> 172.21.15.130:0/3250641808 conn(0x7fb0b80fbb70 msgr2=0x7fb0b80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.061+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/3250641808 shutdown_connections 2023-09-12T16:03:51.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.061+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/3250641808 wait complete. 2023-09-12T16:03:51.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bfe5d700 1 Processor -- start 2023-09-12T16:03:51.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bfe5d700 1 -- start start 2023-09-12T16:03:51.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bfe5d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b8100620 0x7fb0b8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bfe5d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8101810 0x7fb0b81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bfe5d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81021b0 0x7fb0b819fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bfe5d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b819fff0 con 0x7fb0b8101810 2023-09-12T16:03:51.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bfe5d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b81a0130 con 0x7fb0b8100620 2023-09-12T16:03:51.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bfe5d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b81a02c0 con 0x7fb0b81021b0 2023-09-12T16:03:51.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bd3f8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8101810 0x7fb0b81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bd3f8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8101810 0x7fb0b81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55310/0 (socket says 172.21.15.130:55310) 2023-09-12T16:03:51.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bd3f8700 1 -- 172.21.15.130:0/875463315 learned_addr learned my addr 172.21.15.130:0/875463315 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bdbf9700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b8100620 0x7fb0b8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0be3fa700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81021b0 0x7fb0b819fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bd3f8700 1 -- 172.21.15.130:0/875463315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81021b0 msgr2=0x7fb0b819fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bd3f8700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81021b0 0x7fb0b819fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bd3f8700 1 -- 172.21.15.130:0/875463315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b8100620 msgr2=0x7fb0b8198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bd3f8700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b8100620 0x7fb0b8198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bd3f8700 1 -- 172.21.15.130:0/875463315 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a80095f0 con 0x7fb0b8101810 2023-09-12T16:03:51.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.062+0000 7fb0bd3f8700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8101810 0x7fb0b81989b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c9b0 tx=0x7fb0b400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.063+0000 7fb0aeffd700 1 -- 172.21.15.130:0/875463315 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4007b00 con 0x7fb0b8101810 2023-09-12T16:03:51.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.063+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b81a05a0 con 0x7fb0b8101810 2023-09-12T16:03:51.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.063+0000 7fb0aeffd700 1 -- 172.21.15.130:0/875463315 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0b400ce50 con 0x7fb0b8101810 2023-09-12T16:03:51.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.063+0000 7fb0aeffd700 1 -- 172.21.15.130:0/875463315 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4018660 con 0x7fb0b8101810 2023-09-12T16:03:51.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.063+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b81a0b00 con 0x7fb0b8101810 2023-09-12T16:03:51.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.063+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b810be50 con 0x7fb0b8101810 2023-09-12T16:03:51.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.065+0000 7fb0aeffd700 1 -- 172.21.15.130:0/875463315 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7fb0b400f450 con 0x7fb0b8101810 2023-09-12T16:03:51.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.065+0000 7fb0aeffd700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb0a40a0710 0x7fb0a40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.065+0000 7fb0bdbf9700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb0a40a0710 0x7fb0a40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.065+0000 7fb0aeffd700 1 -- 172.21.15.130:0/875463315 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb0b40cc210 con 0x7fb0b8101810 2023-09-12T16:03:51.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.066+0000 7fb0bdbf9700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb0a40a0710 0x7fb0a40a2bd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009990 tx=0x7fb0a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.069+0000 7fb0aeffd700 1 -- 172.21.15.130:0/875463315 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb0b4095900 con 0x7fb0b8101810 2023-09-12T16:03:51.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:50 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4021674340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:51.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:50 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3052744028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:51.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-09-12T16:03:51.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2023-09-12T16:03:51.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-09-12T16:03:51.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:50 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4021674340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:51.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:50 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3052744028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:51.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-09-12T16:03:51.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2023-09-12T16:03:51.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-09-12T16:03:51.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:50 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4021674340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:03:51.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:50 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3052744028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:03:51.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-09-12T16:03:51.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2023-09-12T16:03:51.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-09-12T16:03:51.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.262+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0b81a0d80 con 0x7fb0b8101810 2023-09-12T16:03:51.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.264+0000 7fb0aeffd700 1 -- 172.21.15.130:0/875463315 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24671 (secure 0 0 0) 0x7fb0b4007e60 con 0x7fb0b8101810 2023-09-12T16:03:51.265 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:51.265 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":89,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:03:50.883409+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:03:51.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb0a40a0710 msgr2=0x7fb0a40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb0a40a0710 0x7fb0a40a2bd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009990 tx=0x7fb0a8005fb0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8101810 msgr2=0x7fb0b81989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8101810 0x7fb0b81989b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c9b0 tx=0x7fb0b400ccc0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 shutdown_connections 2023-09-12T16:03:51.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb0a40a0710 0x7fb0a40a2bd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81021b0 0x7fb0b819fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b8100620 0x7fb0b8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 --2- 172.21.15.130:0/875463315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0b8101810 0x7fb0b81989b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 >> 172.21.15.130:0/875463315 conn(0x7fb0b80fbb70 msgr2=0x7fb0b8107040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 shutdown_connections 2023-09-12T16:03:51.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.266+0000 7fb0bfe5d700 1 -- 172.21.15.130:0/875463315 wait complete. 2023-09-12T16:03:51.280 DEBUG:tasks.fwd_scrub:Yielding 2023-09-12T16:03:51.280 INFO:teuthology.run_tasks:Running task exec... 2023-09-12T16:03:51.291 INFO:teuthology.task.exec:Executing custom commands... 2023-09-12T16:03:51.291 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi130.front.sepia.ceph.com 2023-09-12T16:03:51.291 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-09-12T16:03:51.292 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-09-12T16:03:51.292 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-12T16:03:51.293 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-12T16:03:51.330 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:51.348 INFO:teuthology.orchestra.run.smithi130.stdout:Filesystem Size Used Avail Use% Mounted on 2023-09-12T16:03:51.348 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:3300,172.21.15.159:3300,172.21.15.203:3300:/volumes/_nogroup/sv_1/47d66d2e-a22e-4bfa-ac90-69257256aa66 1.1T 332M 1.1T 1% /tmp/cephfs 2023-09-12T16:03:51.350 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-09-12T16:03:51.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.424+0000 7f31e1ea0700 1 -- 172.21.15.130:0/987941999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1062d0 msgr2=0x7f31dc108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.424+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/987941999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1062d0 0x7f31dc108740 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f31d400b0a0 tx=0x7f31d400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.424+0000 7f31e1ea0700 1 -- 172.21.15.130:0/987941999 shutdown_connections 2023-09-12T16:03:51.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.424+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/987941999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1062d0 0x7f31dc108740 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.424+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/987941999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31dc101070 0x7f31dc103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.424+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/987941999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31dc1039a0 0x7f31dc105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.424+0000 7f31e1ea0700 1 -- 172.21.15.130:0/987941999 >> 172.21.15.130:0/987941999 conn(0x7f31dc0faa60 msgr2=0x7f31dc0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.424+0000 7f31e1ea0700 1 -- 172.21.15.130:0/987941999 shutdown_connections 2023-09-12T16:03:51.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.424+0000 7f31e1ea0700 1 -- 172.21.15.130:0/987941999 wait complete. 2023-09-12T16:03:51.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31e1ea0700 1 Processor -- start 2023-09-12T16:03:51.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31e1ea0700 1 -- start start 2023-09-12T16:03:51.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31e1ea0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31dc101070 0x7f31dc196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31e1ea0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1039a0 0x7f31dc196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31e1ea0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31dc1062d0 0x7f31dc19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31e1ea0700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc19df80 con 0x7f31dc1039a0 2023-09-12T16:03:51.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31e1ea0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc19e0c0 con 0x7f31dc101070 2023-09-12T16:03:51.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31e1ea0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc19e250 con 0x7f31dc1062d0 2023-09-12T16:03:51.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1039a0 0x7f31dc196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1039a0 0x7f31dc196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55324/0 (socket says 172.21.15.130:55324) 2023-09-12T16:03:51.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31daffd700 1 -- 172.21.15.130:0/435968552 learned_addr learned my addr 172.21.15.130:0/435968552 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31db7fe700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31dc101070 0x7f31dc196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.425+0000 7f31dbfff700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31dc1062d0 0x7f31dc19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31daffd700 1 -- 172.21.15.130:0/435968552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31dc1062d0 msgr2=0x7f31dc19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31daffd700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31dc1062d0 0x7f31dc19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31daffd700 1 -- 172.21.15.130:0/435968552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31dc101070 msgr2=0x7f31dc196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31daffd700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31dc101070 0x7f31dc196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31daffd700 1 -- 172.21.15.130:0/435968552 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0009520 con 0x7f31dc1039a0 2023-09-12T16:03:51.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31daffd700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1039a0 0x7f31dc196940 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f31d000eb30 tx=0x7f31d000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.445 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31d8ff9700 1 -- 172.21.15.130:0/435968552 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0009880 con 0x7f31dc1039a0 2023-09-12T16:03:51.445 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31d8ff9700 1 -- 172.21.15.130:0/435968552 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31d00044e0 con 0x7f31dc1039a0 2023-09-12T16:03:51.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31d8ff9700 1 -- 172.21.15.130:0/435968552 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0003d80 con 0x7f31dc1039a0 2023-09-12T16:03:51.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31e1ea0700 1 -- 172.21.15.130:0/435968552 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d4009d00 con 0x7f31dc1039a0 2023-09-12T16:03:51.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.426+0000 7f31e1ea0700 1 -- 172.21.15.130:0/435968552 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc19e860 con 0x7f31dc1039a0 2023-09-12T16:03:51.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.428+0000 7f31d8ff9700 1 -- 172.21.15.130:0/435968552 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f31d0003f20 con 0x7f31dc1039a0 2023-09-12T16:03:51.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.428+0000 7f31d8ff9700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f31c40a06d0 0x7f31c40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.429+0000 7f31db7fe700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f31c40a06d0 0x7f31c40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.429+0000 7f31d8ff9700 1 -- 172.21.15.130:0/435968552 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f31d0014070 con 0x7f31dc1039a0 2023-09-12T16:03:51.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.429+0000 7f31db7fe700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f31c40a06d0 0x7f31c40a2b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009c60 tx=0x7f31cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.429+0000 7f31ee066b80 1 Processor -- start 2023-09-12T16:03:51.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31ee066b80 1 -- start start 2023-09-12T16:03:51.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e9890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 0x55c12e987880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e988330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31ee066b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e8e07b0 con 0x55c12e987e80 2023-09-12T16:03:51.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31ee066b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e8de0d0 con 0x55c12e989620 2023-09-12T16:03:51.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31ee066b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e8d99c0 con 0x55c12e988c80 2023-09-12T16:03:51.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31dbfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e988330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31dbfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e988330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55334/0 (socket says 172.21.15.130:55334) 2023-09-12T16:03:51.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31dbfff700 1 -- 172.21.15.130:0/993871865 learned_addr learned my addr 172.21.15.130:0/993871865 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31daffd700 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 0x55c12e987880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31db7fe700 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e9890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31dbfff700 1 -- 172.21.15.130:0/993871865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 msgr2=0x55c12e9890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31dbfff700 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e9890a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31dbfff700 1 -- 172.21.15.130:0/993871865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 msgr2=0x55c12e987880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31dbfff700 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 0x55c12e987880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31dbfff700 1 -- 172.21.15.130:0/993871865 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c12e705e60 con 0x55c12e987e80 2023-09-12T16:03:51.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31db7fe700 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e9890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:51.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31daffd700 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 0x55c12e987880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:51.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.430+0000 7f31dbfff700 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e988330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f31d400b0a0 tx=0x7f31d4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf58cf9061e65672 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31c1ffb700 1 -- 172.21.15.130:0/993871865 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d400e040 con 0x55c12e987e80 2023-09-12T16:03:51.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31c1ffb700 1 -- 172.21.15.130:0/993871865 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31d40076e0 con 0x55c12e987e80 2023-09-12T16:03:51.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31c1ffb700 1 -- 172.21.15.130:0/993871865 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d401c6a0 con 0x55c12e987e80 2023-09-12T16:03:51.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31ee066b80 1 -- 172.21.15.130:0/993871865 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 msgr2=0x55c12e988330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31ee066b80 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e988330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f31d400b0a0 tx=0x7f31d4009c20 comp rx=0 tx=0).stop 2023-09-12T16:03:51.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31ee066b80 1 -- 172.21.15.130:0/993871865 shutdown_connections 2023-09-12T16:03:51.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31ee066b80 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e988330 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31ee066b80 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e9890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31ee066b80 1 --2- 172.21.15.130:0/993871865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 0x55c12e987880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31ee066b80 1 -- 172.21.15.130:0/993871865 >> 172.21.15.130:0/993871865 conn(0x55c12e8fc110 msgr2=0x55c12e9827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31ee066b80 1 -- 172.21.15.130:0/993871865 shutdown_connections 2023-09-12T16:03:51.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.431+0000 7f31ee066b80 1 -- 172.21.15.130:0/993871865 wait complete. 2023-09-12T16:03:51.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31ee066b80 1 -- start start 2023-09-12T16:03:51.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31ee066b80 1 Processor -- start 2023-09-12T16:03:51.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e991580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e991ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 0x55c12e998b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31ee066b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e877bf0 con 0x55c12e987e80 2023-09-12T16:03:51.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31ee066b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e8778f0 con 0x55c12e989620 2023-09-12T16:03:51.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31ee066b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e75b9a0 con 0x55c12e988c80 2023-09-12T16:03:51.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31ee066b80 10 client.0 _ll_register_callbacks cb 0x55c12e7c1b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:51.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31db7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e991580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31db7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e991580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55336/0 (socket says 172.21.15.130:55336) 2023-09-12T16:03:51.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31db7fe700 1 -- 172.21.15.130:0/2313557630 learned_addr learned my addr 172.21.15.130:0/2313557630 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.432+0000 7f31dbfff700 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 0x55c12e998b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31daffd700 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e991ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31db7fe700 1 -- 172.21.15.130:0/2313557630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 msgr2=0x55c12e991ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31db7fe700 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e991ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31db7fe700 1 -- 172.21.15.130:0/2313557630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 msgr2=0x55c12e998b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31db7fe700 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 0x55c12e998b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31db7fe700 1 -- 172.21.15.130:0/2313557630 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc109330 con 0x55c12e987e80 2023-09-12T16:03:51.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31daffd700 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e991ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:51.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31db7fe700 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e991580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009b20 tx=0x7f31cc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67090939688b196 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31baffd700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:51.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31baffd700 1 -- 172.21.15.130:0/2313557630 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc01de70 con 0x55c12e987e80 2023-09-12T16:03:51.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31baffd700 1 -- 172.21.15.130:0/2313557630 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f31cc004260 con 0x55c12e987e80 2023-09-12T16:03:51.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31ee066b80 1 -- 172.21.15.130:0/2313557630 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c12e831570 con 0x55c12e987e80 2023-09-12T16:03:51.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31baffd700 1 -- 172.21.15.130:0/2313557630 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31cc004420 con 0x55c12e987e80 2023-09-12T16:03:51.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31ee066b80 10 client.15735 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:51.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.433+0000 7f31ee066b80 1 -- 172.21.15.130:0/2313557630 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c12e6eece0 con 0x55c12e987e80 2023-09-12T16:03:51.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31baffd700 1 -- 172.21.15.130:0/2313557630 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c12e831570 con 0x55c12e987e80 2023-09-12T16:03:51.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31baffd700 1 -- 172.21.15.130:0/2313557630 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f31cc003b60 con 0x55c12e987e80 2023-09-12T16:03:51.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31ee066b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:51.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31ee066b80 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:03:51.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31ee066b80 20 client.15735 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:51.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c12e99d2d0 0x55c12e99f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31ee066b80 4 client.15735 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:03:51.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31ee066b80 1 -- 172.21.15.130:0/2313557630 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c12e8ee920 con 0x55c12e99d2d0 2023-09-12T16:03:51.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31daffd700 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c12e99d2d0 0x55c12e99f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31baffd700 10 client.15735 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:51.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.434+0000 7f31daffd700 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c12e99d2d0 0x55c12e99f6f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d49b1d18d86e58 server_cookie=36bbf70c81c18a8e in_seq=0 out_seq=0 2023-09-12T16:03:51.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31baffd700 1 -- 172.21.15.130:0/2313557630 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c12e8ee920 con 0x55c12e99d2d0 2023-09-12T16:03:51.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31baffd700 10 client.15735 handle_command_reply: tid=0 2023-09-12T16:03:51.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 client.15735 shutdown 2023-09-12T16:03:51.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 -- 172.21.15.130:0/2313557630 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 msgr2=0x55c12e991580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e991580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009b20 tx=0x7f31cc0047c0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 -- 172.21.15.130:0/2313557630 shutdown_connections 2023-09-12T16:03:51.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c12e99d2d0 0x55c12e99f6f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e989620 0x55c12e998b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e987e80 0x55c12e991580 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2313557630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e988c80 0x55c12e991ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 -- 172.21.15.130:0/2313557630 >> 172.21.15.130:0/2313557630 conn(0x55c12e8fc110 msgr2=0x55c12e981e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31baffd700 0 client.15735 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:51.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 -- 172.21.15.130:0/2313557630 shutdown_connections 2023-09-12T16:03:51.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.435+0000 7f31ee066b80 1 -- 172.21.15.130:0/2313557630 wait complete. 2023-09-12T16:03:51.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.436+0000 7f31ee066b80 20 client.15735 trim_cache size 0 max 16384 2023-09-12T16:03:51.470 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-09-12T16:03:51.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31ee066b80 1 Processor -- start 2023-09-12T16:03:51.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31ee066b80 1 -- start start 2023-09-12T16:03:51.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 0x55c12e987790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e998ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 0x55c12e98f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31ee066b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e987cd0 con 0x55c12e988c80 2023-09-12T16:03:51.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31ee066b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e9836e0 con 0x55c12e987e80 2023-09-12T16:03:51.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31ee066b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e9910d0 con 0x55c12e989620 2023-09-12T16:03:51.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e998ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e998ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55346/0 (socket says 172.21.15.130:55346) 2023-09-12T16:03:51.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31daffd700 1 -- 172.21.15.130:0/2002189489 learned_addr learned my addr 172.21.15.130:0/2002189489 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31dbfff700 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 0x55c12e98f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31db7fe700 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 0x55c12e987790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31daffd700 1 -- 172.21.15.130:0/2002189489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 msgr2=0x55c12e98f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31daffd700 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 0x55c12e98f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31daffd700 1 -- 172.21.15.130:0/2002189489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 msgr2=0x55c12e987790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31daffd700 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 0x55c12e987790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31daffd700 1 -- 172.21.15.130:0/2002189489 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c12e9a4360 con 0x55c12e988c80 2023-09-12T16:03:51.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31db7fe700 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 0x55c12e987790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:51.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.456+0000 7f31daffd700 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e998ac0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f31d0096370 tx=0x7f31d00b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfe8dce4e26ffcb server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31c27fc700 1 -- 172.21.15.130:0/2002189489 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0099900 con 0x55c12e988c80 2023-09-12T16:03:51.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31c27fc700 1 -- 172.21.15.130:0/2002189489 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31d009d730 con 0x55c12e988c80 2023-09-12T16:03:51.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31c27fc700 1 -- 172.21.15.130:0/2002189489 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00cc310 con 0x55c12e988c80 2023-09-12T16:03:51.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31ee066b80 1 -- 172.21.15.130:0/2002189489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 msgr2=0x55c12e998ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e998ac0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f31d0096370 tx=0x7f31d00b9040 comp rx=0 tx=0).stop 2023-09-12T16:03:51.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31ee066b80 1 -- 172.21.15.130:0/2002189489 shutdown_connections 2023-09-12T16:03:51.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 0x55c12e98f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 0x55c12e987790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2002189489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e998ac0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31ee066b80 1 -- 172.21.15.130:0/2002189489 >> 172.21.15.130:0/2002189489 conn(0x55c12e8fc110 msgr2=0x55c12e99f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31ee066b80 1 -- 172.21.15.130:0/2002189489 shutdown_connections 2023-09-12T16:03:51.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.457+0000 7f31ee066b80 1 -- 172.21.15.130:0/2002189489 wait complete. 2023-09-12T16:03:51.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31ee066b80 1 -- start start 2023-09-12T16:03:51.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31ee066b80 1 Processor -- start 2023-09-12T16:03:51.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 0x55c12e998990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e995430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31ee066b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 0x55c12e995970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31ee066b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e996080 con 0x55c12e988c80 2023-09-12T16:03:51.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31ee066b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e9f1130 con 0x55c12e987e80 2023-09-12T16:03:51.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31ee066b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12e9f12f0 con 0x55c12e989620 2023-09-12T16:03:51.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31ee066b80 10 client.0 _ll_register_callbacks cb 0x7f31cc01d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:51.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e995430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e995430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55352/0 (socket says 172.21.15.130:55352) 2023-09-12T16:03:51.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31daffd700 1 -- 172.21.15.130:0/2953435806 learned_addr learned my addr 172.21.15.130:0/2953435806 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31dbfff700 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 0x55c12e995970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31daffd700 1 -- 172.21.15.130:0/2953435806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 msgr2=0x55c12e995970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31daffd700 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 0x55c12e995970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31daffd700 1 -- 172.21.15.130:0/2953435806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 msgr2=0x55c12e998990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.458+0000 7f31daffd700 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 0x55c12e998990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31daffd700 1 -- 172.21.15.130:0/2953435806 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc19e860 con 0x55c12e988c80 2023-09-12T16:03:51.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31dbfff700 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 0x55c12e995970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:51.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31daffd700 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e995430 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f31d0095e00 tx=0x7f31d00a1a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10220204111b43f server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31baffd700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:51.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31baffd700 1 -- 172.21.15.130:0/2953435806 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00cc310 con 0x55c12e988c80 2023-09-12T16:03:51.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31baffd700 1 -- 172.21.15.130:0/2953435806 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f31d0096010 con 0x55c12e988c80 2023-09-12T16:03:51.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31ee066b80 1 -- 172.21.15.130:0/2953435806 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31d40076e0 con 0x55c12e988c80 2023-09-12T16:03:51.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31baffd700 1 -- 172.21.15.130:0/2953435806 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31d00cc470 con 0x55c12e988c80 2023-09-12T16:03:51.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31ee066b80 10 client.15741 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:51.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31ee066b80 1 -- 172.21.15.130:0/2953435806 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c12e72c2d0 con 0x55c12e988c80 2023-09-12T16:03:51.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31baffd700 1 -- 172.21.15.130:0/2953435806 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31d40076e0 con 0x55c12e988c80 2023-09-12T16:03:51.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31baffd700 1 -- 172.21.15.130:0/2953435806 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f31d0098df0 con 0x55c12e988c80 2023-09-12T16:03:51.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31ee066b80 10 client.15741 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:51.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31ee066b80 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:03:51.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31ee066b80 20 client.15741 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:51.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c12e9f28f0 0x55c12e9f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31ee066b80 4 client.15741 mds_command: new command op to 24593 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-12T16:03:51.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.459+0000 7f31ee066b80 1 -- 172.21.15.130:0/2953435806 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c12e8d29a0 con 0x55c12e9f28f0 2023-09-12T16:03:51.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.460+0000 7f31db7fe700 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c12e9f28f0 0x55c12e9f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.460+0000 7f31db7fe700 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c12e9f28f0 0x55c12e9f4d10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959caa34ea64b754 server_cookie=353806030d480d0 in_seq=0 out_seq=0 2023-09-12T16:03:51.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.460+0000 7f31baffd700 10 client.15741 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:51.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31baffd700 1 -- 172.21.15.130:0/2953435806 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c12e8d29a0 con 0x55c12e9f28f0 2023-09-12T16:03:51.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31baffd700 10 client.15741 handle_command_reply: tid=0 2023-09-12T16:03:51.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 client.15741 shutdown 2023-09-12T16:03:51.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 -- 172.21.15.130:0/2953435806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 msgr2=0x55c12e995430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e995430 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f31d0095e00 tx=0x7f31d00a1a20 comp rx=0 tx=0).stop 2023-09-12T16:03:51.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 -- 172.21.15.130:0/2953435806 shutdown_connections 2023-09-12T16:03:51.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c12e9f28f0 0x55c12e9f4d10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c12e989620 0x55c12e995970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c12e987e80 0x55c12e998990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31baffd700 0 client.15741 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:51.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 --2- 172.21.15.130:0/2953435806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c12e988c80 0x55c12e995430 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 -- 172.21.15.130:0/2953435806 >> 172.21.15.130:0/2953435806 conn(0x55c12e8fc110 msgr2=0x55c12e98f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 -- 172.21.15.130:0/2953435806 shutdown_connections 2023-09-12T16:03:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.461+0000 7f31ee066b80 1 -- 172.21.15.130:0/2953435806 wait complete. 2023-09-12T16:03:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31ee066b80 20 client.15741 trim_cache size 0 max 16384 2023-09-12T16:03:51.493 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:03:51.494 INFO:teuthology.orchestra.run.smithi130.stdout: "return_code": 0, 2023-09-12T16:03:51.494 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_tag": "29863276-3a08-45e4-b8db-54081891787c", 2023-09-12T16:03:51.494 INFO:teuthology.orchestra.run.smithi130.stdout: "mode": "asynchronous" 2023-09-12T16:03:51.494 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:03:51.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 -- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f31c40a06d0 msgr2=0x7f31c40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f31c40a06d0 0x7f31c40a2b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009c60 tx=0x7f31cc009260 comp rx=0 tx=0).stop 2023-09-12T16:03:51.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 -- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1039a0 msgr2=0x7f31dc196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1039a0 0x7f31dc196940 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f31d000eb30 tx=0x7f31d000ee40 comp rx=0 tx=0).stop 2023-09-12T16:03:51.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 -- 172.21.15.130:0/435968552 shutdown_connections 2023-09-12T16:03:51.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f31c40a06d0 0x7f31c40a2b90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31dc1062d0 0x7f31dc19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31dc101070 0x7f31dc196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 --2- 172.21.15.130:0/435968552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31dc1039a0 0x7f31dc196940 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 -- 172.21.15.130:0/435968552 >> 172.21.15.130:0/435968552 conn(0x7f31dc0faa60 msgr2=0x7f31dc101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 -- 172.21.15.130:0/435968552 shutdown_connections 2023-09-12T16:03:51.498 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.462+0000 7f31e1ea0700 1 -- 172.21.15.130:0/435968552 wait complete. 2023-09-12T16:03:51.498 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:03:51.578 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-09-12T16:03:51.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 -- 172.21.15.130:0/3591046399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a4100af0 msgr2=0x7f88a4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 --2- 172.21.15.130:0/3591046399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a4100af0 0x7f88a4100f10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009990 tx=0x7f88a0009ca0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 -- 172.21.15.130:0/3591046399 shutdown_connections 2023-09-12T16:03:51.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 --2- 172.21.15.130:0/3591046399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88a40ff010 0x7f88a40ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 --2- 172.21.15.130:0/3591046399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a4100af0 0x7f88a4100f10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 --2- 172.21.15.130:0/3591046399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a40fe670 0x7f88a40fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 -- 172.21.15.130:0/3591046399 >> 172.21.15.130:0/3591046399 conn(0x7f88a40fa1d0 msgr2=0x7f88a40fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 -- 172.21.15.130:0/3591046399 shutdown_connections 2023-09-12T16:03:51.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 -- 172.21.15.130:0/3591046399 wait complete. 2023-09-12T16:03:51.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 Processor -- start 2023-09-12T16:03:51.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 -- start start 2023-09-12T16:03:51.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a40fe670 0x7f88a4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a40ff010 0x7f88a4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88a4100af0 0x7f88a419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41a00e0 con 0x7f88a40fe670 2023-09-12T16:03:51.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41a0220 con 0x7f88a4100af0 2023-09-12T16:03:51.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88ab487700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41a03b0 con 0x7f88a40ff010 2023-09-12T16:03:51.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a9223700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a40fe670 0x7f88a4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a9223700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a40fe670 0x7f88a4198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55364/0 (socket says 172.21.15.130:55364) 2023-09-12T16:03:51.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a9223700 1 -- 172.21.15.130:0/1621853391 learned_addr learned my addr 172.21.15.130:0/1621853391 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a8a22700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a40ff010 0x7f88a4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a9a24700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88a4100af0 0x7f88a419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a9223700 1 -- 172.21.15.130:0/1621853391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a40ff010 msgr2=0x7f88a4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a9223700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a40ff010 0x7f88a4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a9223700 1 -- 172.21.15.130:0/1621853391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88a4100af0 msgr2=0x7f88a419fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a9223700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88a4100af0 0x7f88a419fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.665+0000 7f88a9223700 1 -- 172.21.15.130:0/1621853391 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a00095f0 con 0x7f88a40fe670 2023-09-12T16:03:51.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.667+0000 7f88a9223700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a40fe670 0x7f88a4198560 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f88a00095c0 tx=0x7f88a000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.667+0000 7f889a7fc700 1 -- 172.21.15.130:0/1621853391 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a001d070 con 0x7f88a40fe670 2023-09-12T16:03:51.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.667+0000 7f88ab487700 1 -- 172.21.15.130:0/1621853391 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a41a0630 con 0x7f88a40fe670 2023-09-12T16:03:51.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.667+0000 7f889a7fc700 1 -- 172.21.15.130:0/1621853391 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88a0022470 con 0x7f88a40fe670 2023-09-12T16:03:51.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.667+0000 7f889a7fc700 1 -- 172.21.15.130:0/1621853391 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0017650 con 0x7f88a40fe670 2023-09-12T16:03:51.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.667+0000 7f88ab487700 1 -- 172.21.15.130:0/1621853391 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a41a0ab0 con 0x7f88a40fe670 2023-09-12T16:03:51.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.669+0000 7f889a7fc700 1 -- 172.21.15.130:0/1621853391 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) v1 ==== 135311+0+0 (secure 0 0 0) 0x7f88a000fad0 con 0x7f88a40fe670 2023-09-12T16:03:51.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.670+0000 7f889a7fc700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88900a06d0 0x7f88900a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.670+0000 7f88a8a22700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88900a06d0 0x7f88900a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.670+0000 7f889a7fc700 1 -- 172.21.15.130:0/1621853391 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f88a00cd350 con 0x7f88a40fe670 2023-09-12T16:03:51.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.670+0000 7f88a8a22700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88900a06d0 0x7f88900a2b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8894005950 tx=0x7f889400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88b764db80 1 Processor -- start 2023-09-12T16:03:51.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88b764db80 1 -- start start 2023-09-12T16:03:51.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88b764db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 0x555cce69e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88b764db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce69c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88b764db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce69d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88b764db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce5f5730 con 0x555cce69e5a0 2023-09-12T16:03:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88b764db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce5f3050 con 0x555cce69ce00 2023-09-12T16:03:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88b764db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce5ee940 con 0x555cce69dc00 2023-09-12T16:03:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a8a22700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce69c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a8a22700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce69c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55372/0 (socket says 172.21.15.130:55372) 2023-09-12T16:03:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a8a22700 1 -- 172.21.15.130:0/3182887286 learned_addr learned my addr 172.21.15.130:0/3182887286 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a9223700 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 0x555cce69e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a9a24700 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce69d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a8a22700 1 -- 172.21.15.130:0/3182887286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 msgr2=0x555cce69e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a8a22700 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 0x555cce69e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a8a22700 1 -- 172.21.15.130:0/3182887286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 msgr2=0x555cce69d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a8a22700 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce69d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a8a22700 1 -- 172.21.15.130:0/3182887286 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cce41ade0 con 0x555cce69e5a0 2023-09-12T16:03:51.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a9223700 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 0x555cce69e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:51.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f88a8a22700 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce69c800 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8894010fa0 tx=0x7f8894013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a37ee35f3433e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f888f7fe700 1 -- 172.21.15.130:0/3182887286 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889401e620 con 0x555cce69e5a0 2023-09-12T16:03:51.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f888f7fe700 1 -- 172.21.15.130:0/3182887286 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8894015070 con 0x555cce69e5a0 2023-09-12T16:03:51.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.671+0000 7f888f7fe700 1 -- 172.21.15.130:0/3182887286 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889401ca20 con 0x555cce69e5a0 2023-09-12T16:03:51.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- 172.21.15.130:0/3182887286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 msgr2=0x555cce69c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce69c800 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8894010fa0 tx=0x7f8894013000 comp rx=0 tx=0).stop 2023-09-12T16:03:51.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- 172.21.15.130:0/3182887286 shutdown_connections 2023-09-12T16:03:51.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce69d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 0x555cce69e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 --2- 172.21.15.130:0/3182887286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce69c800 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- 172.21.15.130:0/3182887286 >> 172.21.15.130:0/3182887286 conn(0x555cce611090 msgr2=0x555cce697720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- 172.21.15.130:0/3182887286 shutdown_connections 2023-09-12T16:03:51.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- 172.21.15.130:0/3182887286 wait complete. 2023-09-12T16:03:51.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- start start 2023-09-12T16:03:51.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 Processor -- start 2023-09-12T16:03:51.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce6a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69dc00 0x555cce6a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69e5a0 0x555cce6adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce58cb70 con 0x555cce69dc00 2023-09-12T16:03:51.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce58c870 con 0x555cce69ce00 2023-09-12T16:03:51.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce470920 con 0x555cce69e5a0 2023-09-12T16:03:51.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 10 client.0 _ll_register_callbacks cb 0x555cce4d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:51.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a8a22700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69dc00 0x555cce6a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a8a22700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69dc00 0x555cce6a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55378/0 (socket says 172.21.15.130:55378) 2023-09-12T16:03:51.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a8a22700 1 -- 172.21.15.130:0/1963712831 learned_addr learned my addr 172.21.15.130:0/1963712831 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a9223700 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce6a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a9a24700 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69e5a0 0x555cce6adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a8a22700 1 -- 172.21.15.130:0/1963712831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69e5a0 msgr2=0x555cce6adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a8a22700 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69e5a0 0x555cce6adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a8a22700 1 -- 172.21.15.130:0/1963712831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 msgr2=0x555cce6a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a8a22700 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce6a65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a8a22700 1 -- 172.21.15.130:0/1963712831 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8894010c50 con 0x555cce69dc00 2023-09-12T16:03:51.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a9a24700 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69e5a0 0x555cce6adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:03:51.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88a8a22700 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69dc00 0x555cce6a6b00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8894010f70 tx=0x7f8894005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f329e3d0bb8a2a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f888cff9700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:51.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f888cff9700 1 -- 172.21.15.130:0/1963712831 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889402d070 con 0x555cce69dc00 2023-09-12T16:03:51.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f888cff9700 1 -- 172.21.15.130:0/1963712831 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8894017070 con 0x555cce69dc00 2023-09-12T16:03:51.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.673+0000 7f88b764db80 1 -- 172.21.15.130:0/1963712831 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cce5464f0 con 0x555cce69dc00 2023-09-12T16:03:51.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f888cff9700 1 -- 172.21.15.130:0/1963712831 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889400be60 con 0x555cce69dc00 2023-09-12T16:03:51.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f88b764db80 10 client.15753 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:51.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f88b764db80 1 -- 172.21.15.130:0/1963712831 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555cce403c60 con 0x555cce69dc00 2023-09-12T16:03:51.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f888cff9700 1 -- 172.21.15.130:0/1963712831 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555cce5464f0 con 0x555cce69dc00 2023-09-12T16:03:51.694 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f888cff9700 1 -- 172.21.15.130:0/1963712831 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8894039070 con 0x555cce69dc00 2023-09-12T16:03:51.694 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f88b764db80 10 client.15753 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:51.694 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f88b764db80 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:03:51.694 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f88b764db80 20 client.15753 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:51.694 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f88b764db80 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x555cce6b22d0 0x555cce6b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f88b764db80 4 client.15753 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:03:51.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f88b764db80 1 -- 172.21.15.130:0/1963712831 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cce6038a0 con 0x555cce6b22d0 2023-09-12T16:03:51.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.675+0000 7f88a9223700 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x555cce6b22d0 0x555cce6b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.676+0000 7f888cff9700 10 client.15753 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:51.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.676+0000 7f88a9223700 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x555cce6b22d0 0x555cce6b46f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23483f592b4d8237 server_cookie=a3878def22d30904 in_seq=0 out_seq=0 2023-09-12T16:03:51.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.676+0000 7f888cff9700 1 -- 172.21.15.130:0/1963712831 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555cce6038a0 con 0x555cce6b22d0 2023-09-12T16:03:51.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.676+0000 7f888cff9700 10 client.15753 handle_command_reply: tid=0 2023-09-12T16:03:51.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 client.15753 shutdown 2023-09-12T16:03:51.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 -- 172.21.15.130:0/1963712831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69dc00 msgr2=0x555cce6a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69dc00 0x555cce6a6b00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8894010f70 tx=0x7f8894005270 comp rx=0 tx=0).stop 2023-09-12T16:03:51.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 -- 172.21.15.130:0/1963712831 shutdown_connections 2023-09-12T16:03:51.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x555cce6b22d0 0x555cce6b46f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69e5a0 0x555cce6adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce6a65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f888cff9700 0 client.15753 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:51.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 --2- 172.21.15.130:0/1963712831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69dc00 0x555cce6a6b00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.698 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 -- 172.21.15.130:0/1963712831 >> 172.21.15.130:0/1963712831 conn(0x555cce611090 msgr2=0x555cce696dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.698 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 -- 172.21.15.130:0/1963712831 shutdown_connections 2023-09-12T16:03:51.698 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 1 -- 172.21.15.130:0/1963712831 wait complete. 2023-09-12T16:03:51.698 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.677+0000 7f88b764db80 20 client.15753 trim_cache size 0 max 16384 2023-09-12T16:03:51.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.693+0000 7f88b764db80 1 Processor -- start 2023-09-12T16:03:51.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88b764db80 1 -- start start 2023-09-12T16:03:51.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88b764db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce6a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88b764db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 0x555cce6adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88b764db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88b764db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce698fe0 con 0x555cce69e5a0 2023-09-12T16:03:51.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88b764db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce6a4f00 con 0x555cce69ce00 2023-09-12T16:03:51.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88b764db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce6a4160 con 0x555cce69dc00 2023-09-12T16:03:51.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9a24700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9a24700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55388/0 (socket says 172.21.15.130:55388) 2023-09-12T16:03:51.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9a24700 1 -- 172.21.15.130:0/2216624045 learned_addr learned my addr 172.21.15.130:0/2216624045 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a8a22700 1 --2- 172.21.15.130:0/2216624045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 0x555cce6adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9223700 1 --2- 172.21.15.130:0/2216624045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce6a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9a24700 1 -- 172.21.15.130:0/2216624045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 msgr2=0x555cce6adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9a24700 1 --2- 172.21.15.130:0/2216624045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 0x555cce6adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9a24700 1 -- 172.21.15.130:0/2216624045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 msgr2=0x555cce6a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9a24700 1 --2- 172.21.15.130:0/2216624045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce6a65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9a24700 1 -- 172.21.15.130:0/2216624045 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cce441250 con 0x555cce69e5a0 2023-09-12T16:03:51.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.694+0000 7f88a9a24700 1 --2- 172.21.15.130:0/2216624045 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6a43b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f889c00aed0 tx=0x7f889c00af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77fe1847b61cdcf server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f888ffff700 1 -- 172.21.15.130:0/2216624045 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c00f050 con 0x555cce69e5a0 2023-09-12T16:03:51.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f888ffff700 1 -- 172.21.15.130:0/2216624045 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f889c010040 con 0x555cce69e5a0 2023-09-12T16:03:51.708 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f888ffff700 1 -- 172.21.15.130:0/2216624045 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c014620 con 0x555cce69e5a0 2023-09-12T16:03:51.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f88b764db80 1 -- 172.21.15.130:0/2216624045 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 msgr2=0x555cce6a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f88b764db80 1 --2- 172.21.15.130:0/2216624045 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6a43b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f889c00aed0 tx=0x7f889c00af00 comp rx=0 tx=0).stop 2023-09-12T16:03:51.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f88b764db80 1 -- 172.21.15.130:0/2216624045 shutdown_connections 2023-09-12T16:03:51.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f88b764db80 1 --2- 172.21.15.130:0/2216624045 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6a43b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f88b764db80 1 --2- 172.21.15.130:0/2216624045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69ce00 0x555cce6a65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f88b764db80 1 --2- 172.21.15.130:0/2216624045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69dc00 0x555cce6adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f88b764db80 1 -- 172.21.15.130:0/2216624045 >> 172.21.15.130:0/2216624045 conn(0x555cce611090 msgr2=0x555cce69d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f88b764db80 1 -- 172.21.15.130:0/2216624045 shutdown_connections 2023-09-12T16:03:51.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.695+0000 7f88b764db80 1 -- 172.21.15.130:0/2216624045 wait complete. 2023-09-12T16:03:51.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88b764db80 1 -- start start 2023-09-12T16:03:51.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88b764db80 1 Processor -- start 2023-09-12T16:03:51.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88b764db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69ce00 0x555cce6ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88b764db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69dc00 0x555cce6ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88b764db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88b764db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce6aa7a0 con 0x555cce69e5a0 2023-09-12T16:03:51.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88b764db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce6aa910 con 0x555cce69dc00 2023-09-12T16:03:51.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88b764db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cce6aaad0 con 0x555cce69ce00 2023-09-12T16:03:51.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88b764db80 10 client.0 _ll_register_callbacks cb 0x7f8894010e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:03:51.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88a9a24700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88a9a24700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6aa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55394/0 (socket says 172.21.15.130:55394) 2023-09-12T16:03:51.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88a9a24700 1 -- 172.21.15.130:0/2949360481 learned_addr learned my addr 172.21.15.130:0/2949360481 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:03:51.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88a8a22700 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69dc00 0x555cce6ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.696+0000 7f88a9223700 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69ce00 0x555cce6ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88a9a24700 1 -- 172.21.15.130:0/2949360481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69ce00 msgr2=0x555cce6ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88a9a24700 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69ce00 0x555cce6ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88a9a24700 1 -- 172.21.15.130:0/2949360481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69dc00 msgr2=0x555cce6ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:03:51.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88a9a24700 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69dc00 0x555cce6ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88a9a24700 1 -- 172.21.15.130:0/2949360481 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c00ab00 con 0x555cce69e5a0 2023-09-12T16:03:51.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88a9223700 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69ce00 0x555cce6ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:03:51.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88a9a24700 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6aa0c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f889c0096b0 tx=0x7f889c0156f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d964f226ff1b63c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:03:51.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f888cff9700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:03:51.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f888cff9700 1 -- 172.21.15.130:0/2949360481 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c028630 con 0x555cce69e5a0 2023-09-12T16:03:51.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88b764db80 1 -- 172.21.15.130:0/2949360481 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8894015070 con 0x555cce69e5a0 2023-09-12T16:03:51.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f888cff9700 1 -- 172.21.15.130:0/2949360481 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f889c006e40 con 0x555cce69e5a0 2023-09-12T16:03:51.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f888cff9700 1 -- 172.21.15.130:0/2949360481 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889c00f030 con 0x555cce69e5a0 2023-09-12T16:03:51.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88b764db80 10 client.15759 fetch_fsmap learned FSMap version 24 2023-09-12T16:03:51.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88b764db80 1 -- 172.21.15.130:0/2949360481 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555cce4192b0 con 0x555cce69e5a0 2023-09-12T16:03:51.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f888cff9700 1 -- 172.21.15.130:0/2949360481 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f889c031020 con 0x555cce69e5a0 2023-09-12T16:03:51.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f888cff9700 1 -- 172.21.15.130:0/2949360481 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f889c01f400 con 0x555cce69e5a0 2023-09-12T16:03:51.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88b764db80 10 client.15759 fetch_fsmap finished waiting for FSMap version 24 2023-09-12T16:03:51.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88b764db80 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:03:51.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.697+0000 7f88b764db80 20 client.15759 populate_metadata read hostname 'smithi130' 2023-09-12T16:03:51.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x555cce707870 0x555cce709c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:03:51.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 4 client.15759 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:03:51.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 -- 172.21.15.130:0/2949360481 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555cce5e7920 con 0x555cce707870 2023-09-12T16:03:51.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88a9223700 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x555cce707870 0x555cce709c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:03:51.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f888cff9700 10 client.15759 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:51.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88a9223700 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x555cce707870 0x555cce709c90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ccfbb67c129313 server_cookie=458bea103339268f in_seq=0 out_seq=0 2023-09-12T16:03:51.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f888cff9700 1 -- 172.21.15.130:0/2949360481 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555cce5e7920 con 0x555cce707870 2023-09-12T16:03:51.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f888cff9700 10 client.15759 handle_command_reply: tid=0 2023-09-12T16:03:51.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 client.15759 shutdown 2023-09-12T16:03:51.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 -- 172.21.15.130:0/2949360481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 msgr2=0x555cce6aa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6aa0c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f889c0096b0 tx=0x7f889c0156f0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 -- 172.21.15.130:0/2949360481 shutdown_connections 2023-09-12T16:03:51.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x555cce707870 0x555cce709c90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cce69e5a0 0x555cce6aa0c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555cce69ce00 0x555cce6ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f888cff9700 0 client.15759 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:03:51.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 --2- 172.21.15.130:0/2949360481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cce69dc00 0x555cce6ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.698+0000 7f88b764db80 1 -- 172.21.15.130:0/2949360481 >> 172.21.15.130:0/2949360481 conn(0x555cce611090 msgr2=0x555cce6a43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88b764db80 1 -- 172.21.15.130:0/2949360481 shutdown_connections 2023-09-12T16:03:51.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88b764db80 1 -- 172.21.15.130:0/2949360481 wait complete. 2023-09-12T16:03:51.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88b764db80 20 client.15759 trim_cache size 0 max 16384 2023-09-12T16:03:51.721 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:03:51.721 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (0 inodes in the stack)", 2023-09-12T16:03:51.722 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:03:51.722 INFO:teuthology.orchestra.run.smithi130.stdout: "29863276-3a08-45e4-b8db-54081891787c": { 2023-09-12T16:03:51.722 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:03:51.722 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "29863276-3a08-45e4-b8db-54081891787c", 2023-09-12T16:03:51.722 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:03:51.723 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:03:51.723 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:03:51.723 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:03:51.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 -- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88900a06d0 msgr2=0x7f88900a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88900a06d0 0x7f88900a2b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8894005950 tx=0x7f889400b410 comp rx=0 tx=0).stop 2023-09-12T16:03:51.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 -- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a40fe670 msgr2=0x7f88a4198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:03:51.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a40fe670 0x7f88a4198560 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f88a00095c0 tx=0x7f88a000f690 comp rx=0 tx=0).stop 2023-09-12T16:03:51.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 -- 172.21.15.130:0/1621853391 shutdown_connections 2023-09-12T16:03:51.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f88900a06d0 0x7f88900a2b90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88a4100af0 0x7f88a419fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88a40fe670 0x7f88a4198560 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 --2- 172.21.15.130:0/1621853391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88a40ff010 0x7f88a4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:03:51.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 -- 172.21.15.130:0/1621853391 >> 172.21.15.130:0/1621853391 conn(0x7f88a40fa1d0 msgr2=0x7f88a40fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:03:51.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 -- 172.21.15.130:0/1621853391 shutdown_connections 2023-09-12T16:03:51.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:03:51.699+0000 7f88ab487700 1 -- 172.21.15.130:0/1621853391 wait complete. 2023-09-12T16:03:51.727 INFO:tasks.cephfs.filesystem:scrub status for tag:29863276-3a08-45e4-b8db-54081891787c - {'path': '/', 'tag': '29863276-3a08-45e4-b8db-54081891787c', 'options': 'recursive,force'} 2023-09-12T16:03:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2023-09-12T16:03:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-09-12T16:03:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2023-09-12T16:03:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-09-12T16:03:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: osdmap e89: 12 total, 12 up, 12 in 2023-09-12T16:03:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/787666033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/875463315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: scrub queued for path: / 2023-09-12T16:03:52.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: scrub summary: idle+waiting paths [/] 2023-09-12T16:03:52.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:51 smithi130 ceph-mon[98670]: scrub summary: active paths [/] 2023-09-12T16:03:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2023-09-12T16:03:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-09-12T16:03:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2023-09-12T16:03:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-09-12T16:03:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: osdmap e89: 12 total, 12 up, 12 in 2023-09-12T16:03:52.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/787666033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:52.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/875463315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:52.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: scrub queued for path: / 2023-09-12T16:03:52.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: scrub summary: idle+waiting paths [/] 2023-09-12T16:03:52.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:51 smithi159 ceph-mon[102522]: scrub summary: active paths [/] 2023-09-12T16:03:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2023-09-12T16:03:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-09-12T16:03:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2023-09-12T16:03:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-09-12T16:03:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: osdmap e89: 12 total, 12 up, 12 in 2023-09-12T16:03:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/787666033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:03:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/875463315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:03:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: scrub queued for path: / 2023-09-12T16:03:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: scrub summary: idle+waiting paths [/] 2023-09-12T16:03:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:51 smithi203 ceph-mon[103431]: scrub summary: active paths [/] 2023-09-12T16:03:53.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:52 smithi130 ceph-mon[98670]: osdmap e90: 12 total, 12 up, 12 in 2023-09-12T16:03:53.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:52 smithi130 ceph-mon[98670]: mgrmap e18: x(active, since 8m), standbys: y, z 2023-09-12T16:03:53.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:52 smithi130 ceph-mon[98670]: scrub summary: idle+waiting paths [/] 2023-09-12T16:03:53.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:52 smithi159 ceph-mon[102522]: osdmap e90: 12 total, 12 up, 12 in 2023-09-12T16:03:53.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:52 smithi159 ceph-mon[102522]: mgrmap e18: x(active, since 8m), standbys: y, z 2023-09-12T16:03:53.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:52 smithi159 ceph-mon[102522]: scrub summary: idle+waiting paths [/] 2023-09-12T16:03:53.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:52 smithi203 ceph-mon[103431]: osdmap e90: 12 total, 12 up, 12 in 2023-09-12T16:03:53.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:52 smithi203 ceph-mon[103431]: mgrmap e18: x(active, since 8m), standbys: y, z 2023-09-12T16:03:53.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:52 smithi203 ceph-mon[103431]: scrub summary: idle+waiting paths [/] 2023-09-12T16:03:53.951 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:53 smithi130 ceph-mon[98670]: pgmap v321: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2023-09-12T16:03:54.047 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-09-12T16:03:54.087 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-09-12T16:03:54.089 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-09-12T16:03:54.170 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-12T16:03:54.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:53 smithi159 ceph-mon[102522]: pgmap v321: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2023-09-12T16:03:54.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:53 smithi203 ceph-mon[103431]: pgmap v321: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2023-09-12T16:03:54.250 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/: 2023-09-12T16:03:54.250 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:54.250 INFO:teuthology.orchestra.run.smithi130.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:03:54.250 INFO:teuthology.orchestra.run.smithi130.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 .. 2023-09-12T16:03:54.250 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 data 2023-09-12T16:03:54.251 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:54.251 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data: 2023-09-12T16:03:54.251 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:54.251 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:54.251 INFO:teuthology.orchestra.run.smithi130.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 .. 2023-09-12T16:03:54.252 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-09-12T16:03:54.338 INFO:teuthology.orchestra.run.smithi130.stdout:[Service] 2023-09-12T16:03:54.339 INFO:teuthology.orchestra.run.smithi130.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-09-12T16:03:54.339 INFO:teuthology.orchestra.run.smithi130.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-09-12T16:03:54.340 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-09-12T16:03:54.420 INFO:teuthology.orchestra.run.smithi130.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-09-12T16:03:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:55 smithi130 ceph-mon[98670]: pgmap v322: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 639 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:03:56.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:55 smithi159 ceph-mon[102522]: pgmap v322: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 639 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:03:56.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:55 smithi203 ceph-mon[103431]: pgmap v322: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 639 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:03:56.968 INFO:teuthology.orchestra.run.smithi130.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-09-12T16:03:56.971 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-12T16:03:57.016 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/: 2023-09-12T16:03:57.016 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.017 INFO:teuthology.orchestra.run.smithi130.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:03:57.017 INFO:teuthology.orchestra.run.smithi130.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 .. 2023-09-12T16:03:57.017 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 data 2023-09-12T16:03:57.017 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.017 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data: 2023-09-12T16:03:57.018 INFO:teuthology.orchestra.run.smithi130.stdout:total 30 2023-09-12T16:03:57.018 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 . 2023-09-12T16:03:57.018 INFO:teuthology.orchestra.run.smithi130.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 .. 2023-09-12T16:03:57.018 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 base 2023-09-12T16:03:57.018 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 12 16:03 global 2023-09-12T16:03:57.019 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 log 2023-09-12T16:03:57.019 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_commit_ts 2023-09-12T16:03:57.019 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_dynshmem 2023-09-12T16:03:57.019 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Sep 12 16:03 pg_hba.conf 2023-09-12T16:03:57.019 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 12 16:03 pg_ident.conf 2023-09-12T16:03:57.019 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 pg_logical 2023-09-12T16:03:57.020 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 pg_multixact 2023-09-12T16:03:57.020 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 pg_notify 2023-09-12T16:03:57.020 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_replslot 2023-09-12T16:03:57.020 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_serial 2023-09-12T16:03:57.020 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_snapshots 2023-09-12T16:03:57.021 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_stat 2023-09-12T16:03:57.021 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_stat_tmp 2023-09-12T16:03:57.021 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 pg_subtrans 2023-09-12T16:03:57.021 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_tblspc 2023-09-12T16:03:57.021 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_twophase 2023-09-12T16:03:57.021 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 PG_VERSION 2023-09-12T16:03:57.022 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 pg_wal 2023-09-12T16:03:57.022 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 pg_xact 2023-09-12T16:03:57.022 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 12 16:03 postgresql.auto.conf 2023-09-12T16:03:57.022 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Sep 12 16:03 postgresql.conf 2023-09-12T16:03:57.022 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.023 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/base: 2023-09-12T16:03:57.023 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.023 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 . 2023-09-12T16:03:57.023 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.023 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 1 2023-09-12T16:03:57.024 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 12346 2023-09-12T16:03:57.024 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 12347 2023-09-12T16:03:57.024 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.024 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/base/1: 2023-09-12T16:03:57.024 INFO:teuthology.orchestra.run.smithi130.stdout:total 7209 2023-09-12T16:03:57.025 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 . 2023-09-12T16:03:57.025 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 .. 2023-09-12T16:03:57.025 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 112 2023-09-12T16:03:57.025 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 113 2023-09-12T16:03:57.025 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 12 16:03 12182 2023-09-12T16:03:57.025 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12182_fsm 2023-09-12T16:03:57.026 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12182_vm 2023-09-12T16:03:57.026 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12184 2023-09-12T16:03:57.026 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12186 2023-09-12T16:03:57.026 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187 2023-09-12T16:03:57.026 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12187_fsm 2023-09-12T16:03:57.027 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187_vm 2023-09-12T16:03:57.028 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12189 2023-09-12T16:03:57.028 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12191 2023-09-12T16:03:57.028 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192 2023-09-12T16:03:57.029 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12192_fsm 2023-09-12T16:03:57.029 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192_vm 2023-09-12T16:03:57.029 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12194 2023-09-12T16:03:57.029 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12196 2023-09-12T16:03:57.030 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197 2023-09-12T16:03:57.030 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12197_fsm 2023-09-12T16:03:57.030 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197_vm 2023-09-12T16:03:57.030 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12199 2023-09-12T16:03:57.030 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12201 2023-09-12T16:03:57.031 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202 2023-09-12T16:03:57.031 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12202_fsm 2023-09-12T16:03:57.031 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202_vm 2023-09-12T16:03:57.031 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12204 2023-09-12T16:03:57.031 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12206 2023-09-12T16:03:57.032 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207 2023-09-12T16:03:57.032 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12207_fsm 2023-09-12T16:03:57.032 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207_vm 2023-09-12T16:03:57.032 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12209 2023-09-12T16:03:57.032 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12211 2023-09-12T16:03:57.033 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12212 2023-09-12T16:03:57.033 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12214 2023-09-12T16:03:57.033 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12216 2023-09-12T16:03:57.033 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 12 16:03 1247 2023-09-12T16:03:57.033 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1247_fsm 2023-09-12T16:03:57.034 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1247_vm 2023-09-12T16:03:57.034 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 12 16:03 1249 2023-09-12T16:03:57.034 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1249_fsm 2023-09-12T16:03:57.034 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1249_vm 2023-09-12T16:03:57.034 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 12 16:03 1255 2023-09-12T16:03:57.035 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1255_fsm 2023-09-12T16:03:57.035 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1255_vm 2023-09-12T16:03:57.035 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 12 16:03 1259 2023-09-12T16:03:57.035 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1259_fsm 2023-09-12T16:03:57.036 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1259_vm 2023-09-12T16:03:57.036 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417 2023-09-12T16:03:57.036 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417_vm 2023-09-12T16:03:57.036 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418 2023-09-12T16:03:57.036 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418_vm 2023-09-12T16:03:57.037 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 174 2023-09-12T16:03:57.037 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 175 2023-09-12T16:03:57.037 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2187 2023-09-12T16:03:57.037 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224 2023-09-12T16:03:57.037 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224_vm 2023-09-12T16:03:57.038 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328 2023-09-12T16:03:57.038 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328_vm 2023-09-12T16:03:57.038 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336 2023-09-12T16:03:57.038 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336_vm 2023-09-12T16:03:57.038 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2337 2023-09-12T16:03:57.039 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2600 2023-09-12T16:03:57.039 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2600_fsm 2023-09-12T16:03:57.039 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2600_vm 2023-09-12T16:03:57.039 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601 2023-09-12T16:03:57.039 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2601_fsm 2023-09-12T16:03:57.040 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601_vm 2023-09-12T16:03:57.040 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2602 2023-09-12T16:03:57.040 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2602_fsm 2023-09-12T16:03:57.040 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2602_vm 2023-09-12T16:03:57.040 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2603 2023-09-12T16:03:57.041 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2603_fsm 2023-09-12T16:03:57.041 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2603_vm 2023-09-12T16:03:57.041 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604 2023-09-12T16:03:57.041 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604_vm 2023-09-12T16:03:57.041 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2605 2023-09-12T16:03:57.042 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2605_fsm 2023-09-12T16:03:57.042 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2605_vm 2023-09-12T16:03:57.042 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606 2023-09-12T16:03:57.042 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2606_fsm 2023-09-12T16:03:57.042 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606_vm 2023-09-12T16:03:57.043 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607 2023-09-12T16:03:57.043 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607_fsm 2023-09-12T16:03:57.043 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2607_vm 2023-09-12T16:03:57.044 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 12 16:03 2608 2023-09-12T16:03:57.044 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2608_fsm 2023-09-12T16:03:57.044 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2608_vm 2023-09-12T16:03:57.044 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 12 16:03 2609 2023-09-12T16:03:57.044 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2609_fsm 2023-09-12T16:03:57.045 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2609_vm 2023-09-12T16:03:57.045 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610 2023-09-12T16:03:57.045 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610_fsm 2023-09-12T16:03:57.045 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2610_vm 2023-09-12T16:03:57.045 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611 2023-09-12T16:03:57.046 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611_vm 2023-09-12T16:03:57.046 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612 2023-09-12T16:03:57.046 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2612_fsm 2023-09-12T16:03:57.046 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612_vm 2023-09-12T16:03:57.046 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613 2023-09-12T16:03:57.047 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613_vm 2023-09-12T16:03:57.047 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615 2023-09-12T16:03:57.047 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2615_fsm 2023-09-12T16:03:57.047 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615_vm 2023-09-12T16:03:57.047 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616 2023-09-12T16:03:57.048 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616_fsm 2023-09-12T16:03:57.048 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2616_vm 2023-09-12T16:03:57.048 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 12 16:03 2617 2023-09-12T16:03:57.048 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2617_fsm 2023-09-12T16:03:57.048 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2617_vm 2023-09-12T16:03:57.049 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 12 16:03 2618 2023-09-12T16:03:57.049 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2618_fsm 2023-09-12T16:03:57.049 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2618_vm 2023-09-12T16:03:57.049 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 12 16:03 2619 2023-09-12T16:03:57.049 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2619_fsm 2023-09-12T16:03:57.050 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2619_vm 2023-09-12T16:03:57.050 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620 2023-09-12T16:03:57.050 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620_vm 2023-09-12T16:03:57.050 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2650 2023-09-12T16:03:57.050 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2651 2023-09-12T16:03:57.051 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2652 2023-09-12T16:03:57.051 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2653 2023-09-12T16:03:57.051 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2654 2023-09-12T16:03:57.051 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2655 2023-09-12T16:03:57.051 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2656 2023-09-12T16:03:57.052 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2657 2023-09-12T16:03:57.052 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 12 16:03 2658 2023-09-12T16:03:57.052 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2659 2023-09-12T16:03:57.052 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2660 2023-09-12T16:03:57.052 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2661 2023-09-12T16:03:57.053 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2662 2023-09-12T16:03:57.053 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2663 2023-09-12T16:03:57.053 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2664 2023-09-12T16:03:57.053 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2665 2023-09-12T16:03:57.053 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2666 2023-09-12T16:03:57.054 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2667 2023-09-12T16:03:57.054 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2668 2023-09-12T16:03:57.054 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2669 2023-09-12T16:03:57.054 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2670 2023-09-12T16:03:57.055 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Sep 12 16:03 2673 2023-09-12T16:03:57.055 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 12 16:03 2674 2023-09-12T16:03:57.055 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 12 16:03 2675 2023-09-12T16:03:57.055 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2678 2023-09-12T16:03:57.055 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2679 2023-09-12T16:03:57.056 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2680 2023-09-12T16:03:57.056 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2681 2023-09-12T16:03:57.056 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2682 2023-09-12T16:03:57.056 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2683 2023-09-12T16:03:57.056 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2684 2023-09-12T16:03:57.057 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2685 2023-09-12T16:03:57.057 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2686 2023-09-12T16:03:57.057 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2687 2023-09-12T16:03:57.057 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2688 2023-09-12T16:03:57.057 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2689 2023-09-12T16:03:57.058 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2690 2023-09-12T16:03:57.058 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 12 16:03 2691 2023-09-12T16:03:57.058 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2692 2023-09-12T16:03:57.058 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2693 2023-09-12T16:03:57.058 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2696 2023-09-12T16:03:57.059 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2699 2023-09-12T16:03:57.059 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2701 2023-09-12T16:03:57.059 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2702 2023-09-12T16:03:57.059 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2703 2023-09-12T16:03:57.059 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2704 2023-09-12T16:03:57.060 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2753 2023-09-12T16:03:57.060 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2753_fsm 2023-09-12T16:03:57.060 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2753_vm 2023-09-12T16:03:57.060 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2754 2023-09-12T16:03:57.061 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2755 2023-09-12T16:03:57.061 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2756 2023-09-12T16:03:57.061 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2757 2023-09-12T16:03:57.061 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830 2023-09-12T16:03:57.061 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830_vm 2023-09-12T16:03:57.062 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2831 2023-09-12T16:03:57.062 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832 2023-09-12T16:03:57.062 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832_vm 2023-09-12T16:03:57.062 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2833 2023-09-12T16:03:57.062 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834 2023-09-12T16:03:57.063 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834_vm 2023-09-12T16:03:57.063 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2835 2023-09-12T16:03:57.063 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836 2023-09-12T16:03:57.063 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836_vm 2023-09-12T16:03:57.063 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2837 2023-09-12T16:03:57.064 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 12 16:03 2838 2023-09-12T16:03:57.064 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2838_fsm 2023-09-12T16:03:57.064 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2838_vm 2023-09-12T16:03:57.064 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2839 2023-09-12T16:03:57.064 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840 2023-09-12T16:03:57.065 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840_fsm 2023-09-12T16:03:57.065 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2840_vm 2023-09-12T16:03:57.065 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2841 2023-09-12T16:03:57.065 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995 2023-09-12T16:03:57.065 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995_vm 2023-09-12T16:03:57.066 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2996 2023-09-12T16:03:57.066 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079 2023-09-12T16:03:57.066 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3079_fsm 2023-09-12T16:03:57.066 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079_vm 2023-09-12T16:03:57.066 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3080 2023-09-12T16:03:57.067 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3081 2023-09-12T16:03:57.067 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3085 2023-09-12T16:03:57.067 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118 2023-09-12T16:03:57.067 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118_vm 2023-09-12T16:03:57.067 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3119 2023-09-12T16:03:57.068 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3164 2023-09-12T16:03:57.068 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256 2023-09-12T16:03:57.068 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256_vm 2023-09-12T16:03:57.068 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3257 2023-09-12T16:03:57.068 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3258 2023-09-12T16:03:57.069 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350 2023-09-12T16:03:57.069 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350_vm 2023-09-12T16:03:57.069 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3351 2023-09-12T16:03:57.069 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3379 2023-09-12T16:03:57.069 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3380 2023-09-12T16:03:57.070 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381 2023-09-12T16:03:57.070 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381_vm 2023-09-12T16:03:57.070 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3394 2023-09-12T16:03:57.070 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3394_fsm 2023-09-12T16:03:57.070 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3394_vm 2023-09-12T16:03:57.071 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3395 2023-09-12T16:03:57.071 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439 2023-09-12T16:03:57.071 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439_vm 2023-09-12T16:03:57.071 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3440 2023-09-12T16:03:57.072 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 3455 2023-09-12T16:03:57.072 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3456 2023-09-12T16:03:57.072 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3456_fsm 2023-09-12T16:03:57.072 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3456_vm 2023-09-12T16:03:57.072 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466 2023-09-12T16:03:57.073 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466_vm 2023-09-12T16:03:57.073 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3467 2023-09-12T16:03:57.073 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3468 2023-09-12T16:03:57.073 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501 2023-09-12T16:03:57.073 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501_vm 2023-09-12T16:03:57.074 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3502 2023-09-12T16:03:57.074 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3503 2023-09-12T16:03:57.074 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3534 2023-09-12T16:03:57.074 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541 2023-09-12T16:03:57.074 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3541_fsm 2023-09-12T16:03:57.075 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541_vm 2023-09-12T16:03:57.075 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3542 2023-09-12T16:03:57.075 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3574 2023-09-12T16:03:57.075 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3575 2023-09-12T16:03:57.075 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576 2023-09-12T16:03:57.076 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576_vm 2023-09-12T16:03:57.076 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596 2023-09-12T16:03:57.076 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596_vm 2023-09-12T16:03:57.076 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3597 2023-09-12T16:03:57.076 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598 2023-09-12T16:03:57.077 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598_vm 2023-09-12T16:03:57.077 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3599 2023-09-12T16:03:57.077 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600 2023-09-12T16:03:57.077 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3600_fsm 2023-09-12T16:03:57.077 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600_vm 2023-09-12T16:03:57.078 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601 2023-09-12T16:03:57.078 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3601_fsm 2023-09-12T16:03:57.078 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601_vm 2023-09-12T16:03:57.078 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602 2023-09-12T16:03:57.078 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3602_fsm 2023-09-12T16:03:57.079 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602_vm 2023-09-12T16:03:57.079 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3603 2023-09-12T16:03:57.079 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3603_fsm 2023-09-12T16:03:57.079 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3603_vm 2023-09-12T16:03:57.079 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3604 2023-09-12T16:03:57.080 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3605 2023-09-12T16:03:57.080 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3606 2023-09-12T16:03:57.083 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3607 2023-09-12T16:03:57.084 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3608 2023-09-12T16:03:57.084 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 3609 2023-09-12T16:03:57.084 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3712 2023-09-12T16:03:57.084 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764 2023-09-12T16:03:57.084 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3764_fsm 2023-09-12T16:03:57.085 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764_vm 2023-09-12T16:03:57.085 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3766 2023-09-12T16:03:57.085 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3767 2023-09-12T16:03:57.085 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3997 2023-09-12T16:03:57.085 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 5002 2023-09-12T16:03:57.086 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 548 2023-09-12T16:03:57.086 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 549 2023-09-12T16:03:57.086 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102 2023-09-12T16:03:57.086 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102_vm 2023-09-12T16:03:57.086 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104 2023-09-12T16:03:57.087 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104_vm 2023-09-12T16:03:57.087 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106 2023-09-12T16:03:57.087 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106_vm 2023-09-12T16:03:57.087 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6110 2023-09-12T16:03:57.087 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6111 2023-09-12T16:03:57.088 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6112 2023-09-12T16:03:57.088 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6113 2023-09-12T16:03:57.088 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6117 2023-09-12T16:03:57.088 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826 2023-09-12T16:03:57.088 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826_vm 2023-09-12T16:03:57.089 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 827 2023-09-12T16:03:57.089 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 828 2023-09-12T16:03:57.089 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 12 16:03 pg_filenode.map 2023-09-12T16:03:57.089 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 PG_VERSION 2023-09-12T16:03:57.090 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.090 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-09-12T16:03:57.090 INFO:teuthology.orchestra.run.smithi130.stdout:total 7209 2023-09-12T16:03:57.090 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 . 2023-09-12T16:03:57.090 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 .. 2023-09-12T16:03:57.091 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 112 2023-09-12T16:03:57.091 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 113 2023-09-12T16:03:57.091 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 12 16:03 12182 2023-09-12T16:03:57.091 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12182_fsm 2023-09-12T16:03:57.091 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12182_vm 2023-09-12T16:03:57.092 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12184 2023-09-12T16:03:57.092 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12186 2023-09-12T16:03:57.092 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187 2023-09-12T16:03:57.092 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12187_fsm 2023-09-12T16:03:57.092 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187_vm 2023-09-12T16:03:57.093 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12189 2023-09-12T16:03:57.093 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12191 2023-09-12T16:03:57.093 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192 2023-09-12T16:03:57.093 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12192_fsm 2023-09-12T16:03:57.093 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192_vm 2023-09-12T16:03:57.094 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12194 2023-09-12T16:03:57.094 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12196 2023-09-12T16:03:57.094 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197 2023-09-12T16:03:57.094 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12197_fsm 2023-09-12T16:03:57.095 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197_vm 2023-09-12T16:03:57.095 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12199 2023-09-12T16:03:57.095 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12201 2023-09-12T16:03:57.095 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202 2023-09-12T16:03:57.095 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12202_fsm 2023-09-12T16:03:57.096 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202_vm 2023-09-12T16:03:57.096 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12204 2023-09-12T16:03:57.096 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12206 2023-09-12T16:03:57.096 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207 2023-09-12T16:03:57.096 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12207_fsm 2023-09-12T16:03:57.097 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207_vm 2023-09-12T16:03:57.097 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12209 2023-09-12T16:03:57.097 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12211 2023-09-12T16:03:57.097 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12212 2023-09-12T16:03:57.097 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12214 2023-09-12T16:03:57.098 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12216 2023-09-12T16:03:57.098 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 12 16:03 1247 2023-09-12T16:03:57.098 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1247_fsm 2023-09-12T16:03:57.098 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1247_vm 2023-09-12T16:03:57.098 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 12 16:03 1249 2023-09-12T16:03:57.099 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1249_fsm 2023-09-12T16:03:57.099 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1249_vm 2023-09-12T16:03:57.099 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 12 16:03 1255 2023-09-12T16:03:57.099 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1255_fsm 2023-09-12T16:03:57.099 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1255_vm 2023-09-12T16:03:57.100 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 12 16:03 1259 2023-09-12T16:03:57.100 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1259_fsm 2023-09-12T16:03:57.100 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1259_vm 2023-09-12T16:03:57.100 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417 2023-09-12T16:03:57.100 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417_vm 2023-09-12T16:03:57.101 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418 2023-09-12T16:03:57.101 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418_vm 2023-09-12T16:03:57.101 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 174 2023-09-12T16:03:57.101 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 175 2023-09-12T16:03:57.102 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2187 2023-09-12T16:03:57.102 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224 2023-09-12T16:03:57.102 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224_vm 2023-09-12T16:03:57.102 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328 2023-09-12T16:03:57.102 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328_vm 2023-09-12T16:03:57.103 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336 2023-09-12T16:03:57.103 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336_vm 2023-09-12T16:03:57.103 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2337 2023-09-12T16:03:57.103 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2600 2023-09-12T16:03:57.103 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2600_fsm 2023-09-12T16:03:57.104 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2600_vm 2023-09-12T16:03:57.104 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601 2023-09-12T16:03:57.104 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2601_fsm 2023-09-12T16:03:57.104 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601_vm 2023-09-12T16:03:57.105 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2602 2023-09-12T16:03:57.105 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2602_fsm 2023-09-12T16:03:57.105 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2602_vm 2023-09-12T16:03:57.105 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2603 2023-09-12T16:03:57.105 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2603_fsm 2023-09-12T16:03:57.106 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2603_vm 2023-09-12T16:03:57.106 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604 2023-09-12T16:03:57.106 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604_vm 2023-09-12T16:03:57.106 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2605 2023-09-12T16:03:57.106 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2605_fsm 2023-09-12T16:03:57.107 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2605_vm 2023-09-12T16:03:57.107 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606 2023-09-12T16:03:57.107 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2606_fsm 2023-09-12T16:03:57.107 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606_vm 2023-09-12T16:03:57.107 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607 2023-09-12T16:03:57.108 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607_fsm 2023-09-12T16:03:57.108 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2607_vm 2023-09-12T16:03:57.108 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 12 16:03 2608 2023-09-12T16:03:57.108 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2608_fsm 2023-09-12T16:03:57.108 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2608_vm 2023-09-12T16:03:57.109 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 12 16:03 2609 2023-09-12T16:03:57.109 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2609_fsm 2023-09-12T16:03:57.109 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2609_vm 2023-09-12T16:03:57.109 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610 2023-09-12T16:03:57.109 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610_fsm 2023-09-12T16:03:57.110 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2610_vm 2023-09-12T16:03:57.110 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611 2023-09-12T16:03:57.110 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611_vm 2023-09-12T16:03:57.110 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612 2023-09-12T16:03:57.110 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2612_fsm 2023-09-12T16:03:57.111 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612_vm 2023-09-12T16:03:57.111 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613 2023-09-12T16:03:57.111 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613_vm 2023-09-12T16:03:57.111 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615 2023-09-12T16:03:57.111 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2615_fsm 2023-09-12T16:03:57.112 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615_vm 2023-09-12T16:03:57.112 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616 2023-09-12T16:03:57.112 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616_fsm 2023-09-12T16:03:57.112 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2616_vm 2023-09-12T16:03:57.112 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 12 16:03 2617 2023-09-12T16:03:57.113 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2617_fsm 2023-09-12T16:03:57.113 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2617_vm 2023-09-12T16:03:57.113 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 12 16:03 2618 2023-09-12T16:03:57.113 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2618_fsm 2023-09-12T16:03:57.113 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2618_vm 2023-09-12T16:03:57.114 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 12 16:03 2619 2023-09-12T16:03:57.114 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2619_fsm 2023-09-12T16:03:57.114 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2619_vm 2023-09-12T16:03:57.114 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620 2023-09-12T16:03:57.115 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620_vm 2023-09-12T16:03:57.115 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2650 2023-09-12T16:03:57.115 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2651 2023-09-12T16:03:57.115 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2652 2023-09-12T16:03:57.115 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2653 2023-09-12T16:03:57.116 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2654 2023-09-12T16:03:57.116 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2655 2023-09-12T16:03:57.116 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2656 2023-09-12T16:03:57.116 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2657 2023-09-12T16:03:57.116 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 12 16:03 2658 2023-09-12T16:03:57.117 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2659 2023-09-12T16:03:57.117 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2660 2023-09-12T16:03:57.117 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2661 2023-09-12T16:03:57.117 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2662 2023-09-12T16:03:57.117 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2663 2023-09-12T16:03:57.118 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2664 2023-09-12T16:03:57.118 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2665 2023-09-12T16:03:57.118 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2666 2023-09-12T16:03:57.118 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2667 2023-09-12T16:03:57.119 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2668 2023-09-12T16:03:57.119 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2669 2023-09-12T16:03:57.119 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2670 2023-09-12T16:03:57.119 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Sep 12 16:03 2673 2023-09-12T16:03:57.119 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 12 16:03 2674 2023-09-12T16:03:57.120 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 12 16:03 2675 2023-09-12T16:03:57.120 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2678 2023-09-12T16:03:57.120 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2679 2023-09-12T16:03:57.120 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2680 2023-09-12T16:03:57.120 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2681 2023-09-12T16:03:57.121 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2682 2023-09-12T16:03:57.121 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2683 2023-09-12T16:03:57.121 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2684 2023-09-12T16:03:57.121 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2685 2023-09-12T16:03:57.121 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2686 2023-09-12T16:03:57.122 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2687 2023-09-12T16:03:57.122 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2688 2023-09-12T16:03:57.122 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2689 2023-09-12T16:03:57.122 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2690 2023-09-12T16:03:57.122 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 12 16:03 2691 2023-09-12T16:03:57.123 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2692 2023-09-12T16:03:57.123 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2693 2023-09-12T16:03:57.123 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2696 2023-09-12T16:03:57.123 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2699 2023-09-12T16:03:57.123 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2701 2023-09-12T16:03:57.124 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2702 2023-09-12T16:03:57.124 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2703 2023-09-12T16:03:57.124 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2704 2023-09-12T16:03:57.124 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2753 2023-09-12T16:03:57.124 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2753_fsm 2023-09-12T16:03:57.125 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2753_vm 2023-09-12T16:03:57.125 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2754 2023-09-12T16:03:57.125 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2755 2023-09-12T16:03:57.125 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2756 2023-09-12T16:03:57.125 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2757 2023-09-12T16:03:57.126 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830 2023-09-12T16:03:57.126 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830_vm 2023-09-12T16:03:57.126 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2831 2023-09-12T16:03:57.126 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832 2023-09-12T16:03:57.126 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832_vm 2023-09-12T16:03:57.127 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2833 2023-09-12T16:03:57.127 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834 2023-09-12T16:03:57.127 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834_vm 2023-09-12T16:03:57.127 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2835 2023-09-12T16:03:57.127 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836 2023-09-12T16:03:57.128 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836_vm 2023-09-12T16:03:57.128 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2837 2023-09-12T16:03:57.128 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 12 16:03 2838 2023-09-12T16:03:57.128 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2838_fsm 2023-09-12T16:03:57.128 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2838_vm 2023-09-12T16:03:57.129 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2839 2023-09-12T16:03:57.129 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840 2023-09-12T16:03:57.129 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840_fsm 2023-09-12T16:03:57.129 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2840_vm 2023-09-12T16:03:57.129 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2841 2023-09-12T16:03:57.130 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995 2023-09-12T16:03:57.130 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995_vm 2023-09-12T16:03:57.130 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2996 2023-09-12T16:03:57.130 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079 2023-09-12T16:03:57.130 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3079_fsm 2023-09-12T16:03:57.131 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079_vm 2023-09-12T16:03:57.131 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3080 2023-09-12T16:03:57.131 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3081 2023-09-12T16:03:57.131 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3085 2023-09-12T16:03:57.131 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118 2023-09-12T16:03:57.132 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118_vm 2023-09-12T16:03:57.132 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3119 2023-09-12T16:03:57.132 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3164 2023-09-12T16:03:57.132 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256 2023-09-12T16:03:57.132 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256_vm 2023-09-12T16:03:57.133 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3257 2023-09-12T16:03:57.133 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3258 2023-09-12T16:03:57.133 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350 2023-09-12T16:03:57.133 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350_vm 2023-09-12T16:03:57.133 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3351 2023-09-12T16:03:57.134 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3379 2023-09-12T16:03:57.134 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3380 2023-09-12T16:03:57.134 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381 2023-09-12T16:03:57.134 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381_vm 2023-09-12T16:03:57.134 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3394 2023-09-12T16:03:57.135 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3394_fsm 2023-09-12T16:03:57.135 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3394_vm 2023-09-12T16:03:57.135 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3395 2023-09-12T16:03:57.135 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439 2023-09-12T16:03:57.135 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439_vm 2023-09-12T16:03:57.136 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3440 2023-09-12T16:03:57.136 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 3455 2023-09-12T16:03:57.136 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3456 2023-09-12T16:03:57.136 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3456_fsm 2023-09-12T16:03:57.136 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3456_vm 2023-09-12T16:03:57.137 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466 2023-09-12T16:03:57.137 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466_vm 2023-09-12T16:03:57.137 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3467 2023-09-12T16:03:57.137 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3468 2023-09-12T16:03:57.137 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501 2023-09-12T16:03:57.138 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501_vm 2023-09-12T16:03:57.138 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3502 2023-09-12T16:03:57.138 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3503 2023-09-12T16:03:57.138 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3534 2023-09-12T16:03:57.138 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541 2023-09-12T16:03:57.139 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3541_fsm 2023-09-12T16:03:57.139 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541_vm 2023-09-12T16:03:57.139 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3542 2023-09-12T16:03:57.139 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3574 2023-09-12T16:03:57.139 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3575 2023-09-12T16:03:57.140 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576 2023-09-12T16:03:57.140 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576_vm 2023-09-12T16:03:57.140 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596 2023-09-12T16:03:57.140 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596_vm 2023-09-12T16:03:57.140 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3597 2023-09-12T16:03:57.141 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598 2023-09-12T16:03:57.141 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598_vm 2023-09-12T16:03:57.141 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3599 2023-09-12T16:03:57.141 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600 2023-09-12T16:03:57.141 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3600_fsm 2023-09-12T16:03:57.142 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600_vm 2023-09-12T16:03:57.142 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601 2023-09-12T16:03:57.142 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3601_fsm 2023-09-12T16:03:57.142 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601_vm 2023-09-12T16:03:57.142 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602 2023-09-12T16:03:57.143 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3602_fsm 2023-09-12T16:03:57.143 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602_vm 2023-09-12T16:03:57.143 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3603 2023-09-12T16:03:57.143 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3603_fsm 2023-09-12T16:03:57.143 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3603_vm 2023-09-12T16:03:57.144 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3604 2023-09-12T16:03:57.144 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3605 2023-09-12T16:03:57.144 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3606 2023-09-12T16:03:57.144 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3607 2023-09-12T16:03:57.144 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3608 2023-09-12T16:03:57.145 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 3609 2023-09-12T16:03:57.145 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3712 2023-09-12T16:03:57.145 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764 2023-09-12T16:03:57.145 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3764_fsm 2023-09-12T16:03:57.145 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764_vm 2023-09-12T16:03:57.146 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3766 2023-09-12T16:03:57.146 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3767 2023-09-12T16:03:57.147 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3997 2023-09-12T16:03:57.147 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 5002 2023-09-12T16:03:57.147 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 548 2023-09-12T16:03:57.147 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 549 2023-09-12T16:03:57.148 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102 2023-09-12T16:03:57.148 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102_vm 2023-09-12T16:03:57.148 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104 2023-09-12T16:03:57.148 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104_vm 2023-09-12T16:03:57.149 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106 2023-09-12T16:03:57.149 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106_vm 2023-09-12T16:03:57.149 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6110 2023-09-12T16:03:57.149 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6111 2023-09-12T16:03:57.149 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6112 2023-09-12T16:03:57.150 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6113 2023-09-12T16:03:57.150 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6117 2023-09-12T16:03:57.150 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826 2023-09-12T16:03:57.150 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826_vm 2023-09-12T16:03:57.150 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 827 2023-09-12T16:03:57.151 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 828 2023-09-12T16:03:57.151 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 12 16:03 pg_filenode.map 2023-09-12T16:03:57.151 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 PG_VERSION 2023-09-12T16:03:57.151 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.151 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-09-12T16:03:57.152 INFO:teuthology.orchestra.run.smithi130.stdout:total 7209 2023-09-12T16:03:57.152 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 . 2023-09-12T16:03:57.152 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 .. 2023-09-12T16:03:57.152 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 112 2023-09-12T16:03:57.152 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 113 2023-09-12T16:03:57.153 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 12 16:03 12182 2023-09-12T16:03:57.153 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12182_fsm 2023-09-12T16:03:57.153 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12182_vm 2023-09-12T16:03:57.153 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12184 2023-09-12T16:03:57.153 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12186 2023-09-12T16:03:57.154 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187 2023-09-12T16:03:57.154 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12187_fsm 2023-09-12T16:03:57.154 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187_vm 2023-09-12T16:03:57.154 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12189 2023-09-12T16:03:57.154 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12191 2023-09-12T16:03:57.155 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192 2023-09-12T16:03:57.155 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12192_fsm 2023-09-12T16:03:57.155 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192_vm 2023-09-12T16:03:57.155 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12194 2023-09-12T16:03:57.155 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12196 2023-09-12T16:03:57.156 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197 2023-09-12T16:03:57.156 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12197_fsm 2023-09-12T16:03:57.156 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197_vm 2023-09-12T16:03:57.156 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12199 2023-09-12T16:03:57.157 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12201 2023-09-12T16:03:57.157 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202 2023-09-12T16:03:57.157 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12202_fsm 2023-09-12T16:03:57.157 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202_vm 2023-09-12T16:03:57.157 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12204 2023-09-12T16:03:57.158 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12206 2023-09-12T16:03:57.158 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207 2023-09-12T16:03:57.158 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12207_fsm 2023-09-12T16:03:57.158 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207_vm 2023-09-12T16:03:57.158 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12209 2023-09-12T16:03:57.159 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12211 2023-09-12T16:03:57.159 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12212 2023-09-12T16:03:57.159 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12214 2023-09-12T16:03:57.159 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12216 2023-09-12T16:03:57.159 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 12 16:03 1247 2023-09-12T16:03:57.160 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1247_fsm 2023-09-12T16:03:57.160 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1247_vm 2023-09-12T16:03:57.160 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 12 16:03 1249 2023-09-12T16:03:57.160 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1249_fsm 2023-09-12T16:03:57.160 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1249_vm 2023-09-12T16:03:57.161 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 12 16:03 1255 2023-09-12T16:03:57.161 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1255_fsm 2023-09-12T16:03:57.161 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1255_vm 2023-09-12T16:03:57.161 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 12 16:03 1259 2023-09-12T16:03:57.161 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1259_fsm 2023-09-12T16:03:57.162 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1259_vm 2023-09-12T16:03:57.162 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417 2023-09-12T16:03:57.162 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417_vm 2023-09-12T16:03:57.162 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418 2023-09-12T16:03:57.162 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418_vm 2023-09-12T16:03:57.163 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 174 2023-09-12T16:03:57.163 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 175 2023-09-12T16:03:57.163 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2187 2023-09-12T16:03:57.163 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224 2023-09-12T16:03:57.163 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224_vm 2023-09-12T16:03:57.164 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328 2023-09-12T16:03:57.164 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328_vm 2023-09-12T16:03:57.164 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336 2023-09-12T16:03:57.164 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336_vm 2023-09-12T16:03:57.164 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2337 2023-09-12T16:03:57.165 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2600 2023-09-12T16:03:57.165 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2600_fsm 2023-09-12T16:03:57.165 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2600_vm 2023-09-12T16:03:57.165 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601 2023-09-12T16:03:57.166 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2601_fsm 2023-09-12T16:03:57.166 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601_vm 2023-09-12T16:03:57.166 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2602 2023-09-12T16:03:57.166 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2602_fsm 2023-09-12T16:03:57.166 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2602_vm 2023-09-12T16:03:57.167 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2603 2023-09-12T16:03:57.167 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2603_fsm 2023-09-12T16:03:57.167 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2603_vm 2023-09-12T16:03:57.167 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604 2023-09-12T16:03:57.167 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604_vm 2023-09-12T16:03:57.168 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2605 2023-09-12T16:03:57.168 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2605_fsm 2023-09-12T16:03:57.168 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2605_vm 2023-09-12T16:03:57.168 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606 2023-09-12T16:03:57.168 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2606_fsm 2023-09-12T16:03:57.169 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606_vm 2023-09-12T16:03:57.169 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607 2023-09-12T16:03:57.169 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607_fsm 2023-09-12T16:03:57.169 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2607_vm 2023-09-12T16:03:57.169 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 12 16:03 2608 2023-09-12T16:03:57.170 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2608_fsm 2023-09-12T16:03:57.170 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2608_vm 2023-09-12T16:03:57.170 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 12 16:03 2609 2023-09-12T16:03:57.170 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2609_fsm 2023-09-12T16:03:57.170 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2609_vm 2023-09-12T16:03:57.171 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610 2023-09-12T16:03:57.171 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610_fsm 2023-09-12T16:03:57.171 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2610_vm 2023-09-12T16:03:57.171 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611 2023-09-12T16:03:57.171 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611_vm 2023-09-12T16:03:57.172 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612 2023-09-12T16:03:57.172 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2612_fsm 2023-09-12T16:03:57.172 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612_vm 2023-09-12T16:03:57.172 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613 2023-09-12T16:03:57.172 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613_vm 2023-09-12T16:03:57.173 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615 2023-09-12T16:03:57.173 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2615_fsm 2023-09-12T16:03:57.173 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615_vm 2023-09-12T16:03:57.173 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616 2023-09-12T16:03:57.173 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616_fsm 2023-09-12T16:03:57.174 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2616_vm 2023-09-12T16:03:57.174 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 12 16:03 2617 2023-09-12T16:03:57.174 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2617_fsm 2023-09-12T16:03:57.174 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2617_vm 2023-09-12T16:03:57.174 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 12 16:03 2618 2023-09-12T16:03:57.175 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2618_fsm 2023-09-12T16:03:57.175 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2618_vm 2023-09-12T16:03:57.175 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 12 16:03 2619 2023-09-12T16:03:57.175 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2619_fsm 2023-09-12T16:03:57.175 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2619_vm 2023-09-12T16:03:57.176 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620 2023-09-12T16:03:57.176 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620_vm 2023-09-12T16:03:57.176 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2650 2023-09-12T16:03:57.176 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2651 2023-09-12T16:03:57.177 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2652 2023-09-12T16:03:57.177 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2653 2023-09-12T16:03:57.177 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2654 2023-09-12T16:03:57.177 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2655 2023-09-12T16:03:57.177 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2656 2023-09-12T16:03:57.178 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2657 2023-09-12T16:03:57.178 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 12 16:03 2658 2023-09-12T16:03:57.178 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2659 2023-09-12T16:03:57.178 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2660 2023-09-12T16:03:57.178 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2661 2023-09-12T16:03:57.179 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2662 2023-09-12T16:03:57.179 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2663 2023-09-12T16:03:57.179 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2664 2023-09-12T16:03:57.179 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2665 2023-09-12T16:03:57.179 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2666 2023-09-12T16:03:57.180 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2667 2023-09-12T16:03:57.180 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2668 2023-09-12T16:03:57.180 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2669 2023-09-12T16:03:57.180 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2670 2023-09-12T16:03:57.180 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Sep 12 16:03 2673 2023-09-12T16:03:57.181 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 12 16:03 2674 2023-09-12T16:03:57.181 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 12 16:03 2675 2023-09-12T16:03:57.181 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2678 2023-09-12T16:03:57.181 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2679 2023-09-12T16:03:57.181 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2680 2023-09-12T16:03:57.182 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2681 2023-09-12T16:03:57.182 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2682 2023-09-12T16:03:57.182 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2683 2023-09-12T16:03:57.182 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2684 2023-09-12T16:03:57.182 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2685 2023-09-12T16:03:57.183 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2686 2023-09-12T16:03:57.183 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2687 2023-09-12T16:03:57.183 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2688 2023-09-12T16:03:57.183 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2689 2023-09-12T16:03:57.183 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2690 2023-09-12T16:03:57.184 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 12 16:03 2691 2023-09-12T16:03:57.184 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2692 2023-09-12T16:03:57.184 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2693 2023-09-12T16:03:57.184 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2696 2023-09-12T16:03:57.184 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2699 2023-09-12T16:03:57.185 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2701 2023-09-12T16:03:57.185 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2702 2023-09-12T16:03:57.185 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2703 2023-09-12T16:03:57.185 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2704 2023-09-12T16:03:57.185 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2753 2023-09-12T16:03:57.186 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2753_fsm 2023-09-12T16:03:57.186 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2753_vm 2023-09-12T16:03:57.186 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2754 2023-09-12T16:03:57.186 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2755 2023-09-12T16:03:57.186 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2756 2023-09-12T16:03:57.187 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2757 2023-09-12T16:03:57.187 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830 2023-09-12T16:03:57.187 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830_vm 2023-09-12T16:03:57.187 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2831 2023-09-12T16:03:57.187 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832 2023-09-12T16:03:57.188 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832_vm 2023-09-12T16:03:57.188 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2833 2023-09-12T16:03:57.188 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834 2023-09-12T16:03:57.188 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834_vm 2023-09-12T16:03:57.188 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2835 2023-09-12T16:03:57.189 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836 2023-09-12T16:03:57.189 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836_vm 2023-09-12T16:03:57.189 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2837 2023-09-12T16:03:57.189 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 12 16:03 2838 2023-09-12T16:03:57.189 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2838_fsm 2023-09-12T16:03:57.190 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2838_vm 2023-09-12T16:03:57.190 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2839 2023-09-12T16:03:57.190 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840 2023-09-12T16:03:57.190 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840_fsm 2023-09-12T16:03:57.191 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2840_vm 2023-09-12T16:03:57.191 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2841 2023-09-12T16:03:57.191 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995 2023-09-12T16:03:57.191 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995_vm 2023-09-12T16:03:57.191 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2996 2023-09-12T16:03:57.192 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079 2023-09-12T16:03:57.192 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3079_fsm 2023-09-12T16:03:57.192 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079_vm 2023-09-12T16:03:57.192 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3080 2023-09-12T16:03:57.192 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3081 2023-09-12T16:03:57.193 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3085 2023-09-12T16:03:57.193 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118 2023-09-12T16:03:57.193 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118_vm 2023-09-12T16:03:57.193 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3119 2023-09-12T16:03:57.193 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3164 2023-09-12T16:03:57.194 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256 2023-09-12T16:03:57.194 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256_vm 2023-09-12T16:03:57.194 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3257 2023-09-12T16:03:57.194 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3258 2023-09-12T16:03:57.194 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350 2023-09-12T16:03:57.195 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350_vm 2023-09-12T16:03:57.195 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3351 2023-09-12T16:03:57.195 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3379 2023-09-12T16:03:57.195 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3380 2023-09-12T16:03:57.195 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381 2023-09-12T16:03:57.196 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381_vm 2023-09-12T16:03:57.196 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3394 2023-09-12T16:03:57.196 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3394_fsm 2023-09-12T16:03:57.196 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3394_vm 2023-09-12T16:03:57.196 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3395 2023-09-12T16:03:57.197 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439 2023-09-12T16:03:57.197 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439_vm 2023-09-12T16:03:57.197 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3440 2023-09-12T16:03:57.197 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 3455 2023-09-12T16:03:57.197 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3456 2023-09-12T16:03:57.198 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3456_fsm 2023-09-12T16:03:57.198 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3456_vm 2023-09-12T16:03:57.198 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466 2023-09-12T16:03:57.198 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466_vm 2023-09-12T16:03:57.198 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3467 2023-09-12T16:03:57.199 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3468 2023-09-12T16:03:57.199 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501 2023-09-12T16:03:57.199 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501_vm 2023-09-12T16:03:57.199 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3502 2023-09-12T16:03:57.199 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3503 2023-09-12T16:03:57.200 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3534 2023-09-12T16:03:57.200 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541 2023-09-12T16:03:57.200 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3541_fsm 2023-09-12T16:03:57.200 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541_vm 2023-09-12T16:03:57.200 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3542 2023-09-12T16:03:57.201 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3574 2023-09-12T16:03:57.201 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3575 2023-09-12T16:03:57.201 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576 2023-09-12T16:03:57.201 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576_vm 2023-09-12T16:03:57.202 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596 2023-09-12T16:03:57.202 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596_vm 2023-09-12T16:03:57.202 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3597 2023-09-12T16:03:57.202 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598 2023-09-12T16:03:57.202 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598_vm 2023-09-12T16:03:57.203 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3599 2023-09-12T16:03:57.203 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600 2023-09-12T16:03:57.203 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3600_fsm 2023-09-12T16:03:57.203 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600_vm 2023-09-12T16:03:57.203 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601 2023-09-12T16:03:57.204 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3601_fsm 2023-09-12T16:03:57.204 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601_vm 2023-09-12T16:03:57.204 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602 2023-09-12T16:03:57.204 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3602_fsm 2023-09-12T16:03:57.204 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602_vm 2023-09-12T16:03:57.205 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3603 2023-09-12T16:03:57.205 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3603_fsm 2023-09-12T16:03:57.205 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3603_vm 2023-09-12T16:03:57.205 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3604 2023-09-12T16:03:57.205 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3605 2023-09-12T16:03:57.206 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3606 2023-09-12T16:03:57.206 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3607 2023-09-12T16:03:57.206 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3608 2023-09-12T16:03:57.206 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 3609 2023-09-12T16:03:57.206 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3712 2023-09-12T16:03:57.207 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764 2023-09-12T16:03:57.207 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3764_fsm 2023-09-12T16:03:57.207 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764_vm 2023-09-12T16:03:57.207 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3766 2023-09-12T16:03:57.207 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3767 2023-09-12T16:03:57.208 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3997 2023-09-12T16:03:57.208 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 5002 2023-09-12T16:03:57.208 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 548 2023-09-12T16:03:57.208 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 549 2023-09-12T16:03:57.208 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102 2023-09-12T16:03:57.209 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102_vm 2023-09-12T16:03:57.209 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104 2023-09-12T16:03:57.209 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104_vm 2023-09-12T16:03:57.209 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106 2023-09-12T16:03:57.209 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106_vm 2023-09-12T16:03:57.210 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6110 2023-09-12T16:03:57.210 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6111 2023-09-12T16:03:57.210 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6112 2023-09-12T16:03:57.210 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6113 2023-09-12T16:03:57.211 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6117 2023-09-12T16:03:57.211 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826 2023-09-12T16:03:57.211 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826_vm 2023-09-12T16:03:57.211 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 827 2023-09-12T16:03:57.211 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 828 2023-09-12T16:03:57.212 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 12 16:03 pg_filenode.map 2023-09-12T16:03:57.212 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 PG_VERSION 2023-09-12T16:03:57.212 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.212 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/global: 2023-09-12T16:03:57.212 INFO:teuthology.orchestra.run.smithi130.stdout:total 553 2023-09-12T16:03:57.213 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 12 16:03 . 2023-09-12T16:03:57.213 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.213 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1136 2023-09-12T16:03:57.213 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1136_fsm 2023-09-12T16:03:57.213 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1136_vm 2023-09-12T16:03:57.214 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 1137 2023-09-12T16:03:57.214 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1213 2023-09-12T16:03:57.214 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1213_fsm 2023-09-12T16:03:57.214 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1213_vm 2023-09-12T16:03:57.214 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1214 2023-09-12T16:03:57.215 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1214_fsm 2023-09-12T16:03:57.215 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1214_vm 2023-09-12T16:03:57.215 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 1232 2023-09-12T16:03:57.215 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 1233 2023-09-12T16:03:57.217 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1260 2023-09-12T16:03:57.217 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1260_fsm 2023-09-12T16:03:57.217 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1260_vm 2023-09-12T16:03:57.217 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1261 2023-09-12T16:03:57.218 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1261_fsm 2023-09-12T16:03:57.218 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1261_vm 2023-09-12T16:03:57.218 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1262 2023-09-12T16:03:57.218 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1262_fsm 2023-09-12T16:03:57.218 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1262_vm 2023-09-12T16:03:57.219 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2396 2023-09-12T16:03:57.219 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2396_fsm 2023-09-12T16:03:57.219 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2396_vm 2023-09-12T16:03:57.219 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2397 2023-09-12T16:03:57.219 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2671 2023-09-12T16:03:57.220 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2672 2023-09-12T16:03:57.220 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2676 2023-09-12T16:03:57.220 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2677 2023-09-12T16:03:57.220 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2694 2023-09-12T16:03:57.220 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2695 2023-09-12T16:03:57.221 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2697 2023-09-12T16:03:57.221 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2698 2023-09-12T16:03:57.221 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2846 2023-09-12T16:03:57.221 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2846_vm 2023-09-12T16:03:57.221 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2847 2023-09-12T16:03:57.222 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2964 2023-09-12T16:03:57.222 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2964_vm 2023-09-12T16:03:57.222 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2965 2023-09-12T16:03:57.222 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2966 2023-09-12T16:03:57.223 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2966_vm 2023-09-12T16:03:57.223 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2967 2023-09-12T16:03:57.223 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3592 2023-09-12T16:03:57.223 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3592_vm 2023-09-12T16:03:57.223 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3593 2023-09-12T16:03:57.224 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 4060 2023-09-12T16:03:57.224 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 4060_vm 2023-09-12T16:03:57.224 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 4061 2023-09-12T16:03:57.224 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6000 2023-09-12T16:03:57.224 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6000_vm 2023-09-12T16:03:57.225 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6001 2023-09-12T16:03:57.225 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6002 2023-09-12T16:03:57.225 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6100 2023-09-12T16:03:57.225 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6100_vm 2023-09-12T16:03:57.225 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6114 2023-09-12T16:03:57.226 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6115 2023-09-12T16:03:57.226 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 pg_control 2023-09-12T16:03:57.226 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 12 16:03 pg_filenode.map 2023-09-12T16:03:57.226 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.226 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/log: 2023-09-12T16:03:57.227 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.227 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.227 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.227 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.227 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-09-12T16:03:57.228 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.228 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.228 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.228 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.228 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-09-12T16:03:57.229 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.229 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.229 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.229 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.229 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-09-12T16:03:57.230 INFO:teuthology.orchestra.run.smithi130.stdout:total 1 2023-09-12T16:03:57.230 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 . 2023-09-12T16:03:57.230 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.230 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 mappings 2023-09-12T16:03:57.230 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 12 16:03 replorigin_checkpoint 2023-09-12T16:03:57.231 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 snapshots 2023-09-12T16:03:57.231 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.231 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-09-12T16:03:57.231 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.231 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.232 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 .. 2023-09-12T16:03:57.232 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.232 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-09-12T16:03:57.232 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.232 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.233 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 .. 2023-09-12T16:03:57.233 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.233 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-09-12T16:03:57.233 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.233 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 . 2023-09-12T16:03:57.234 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.234 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 members 2023-09-12T16:03:57.234 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 offsets 2023-09-12T16:03:57.234 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.235 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-09-12T16:03:57.235 INFO:teuthology.orchestra.run.smithi130.stdout:total 8 2023-09-12T16:03:57.235 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:03:57.235 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 .. 2023-09-12T16:03:57.235 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 0000 2023-09-12T16:03:57.236 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.236 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-09-12T16:03:57.236 INFO:teuthology.orchestra.run.smithi130.stdout:total 8 2023-09-12T16:03:57.236 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:03:57.236 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 .. 2023-09-12T16:03:57.237 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 0000 2023-09-12T16:03:57.237 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.237 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-09-12T16:03:57.237 INFO:teuthology.orchestra.run.smithi130.stdout:total 8 2023-09-12T16:03:57.237 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:03:57.238 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.238 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 0000 2023-09-12T16:03:57.238 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.238 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-09-12T16:03:57.238 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.239 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.239 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.239 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.239 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-09-12T16:03:57.239 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.240 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.240 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.240 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.240 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-09-12T16:03:57.240 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.241 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.241 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.241 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.241 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-09-12T16:03:57.241 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.242 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.242 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.242 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.242 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-09-12T16:03:57.243 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.243 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.243 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.243 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.243 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-09-12T16:03:57.244 INFO:teuthology.orchestra.run.smithi130.stdout:total 8 2023-09-12T16:03:57.244 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:03:57.244 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.244 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 0000 2023-09-12T16:03:57.244 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.245 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-09-12T16:03:57.245 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.245 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.245 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.245 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.246 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-09-12T16:03:57.246 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.246 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.246 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.246 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.247 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-09-12T16:03:57.247 INFO:teuthology.orchestra.run.smithi130.stdout:total 16384 2023-09-12T16:03:57.247 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 . 2023-09-12T16:03:57.247 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.247 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:03 000000010000000000000001 2023-09-12T16:03:57.248 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 archive_status 2023-09-12T16:03:57.248 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.248 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-09-12T16:03:57.248 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:03:57.248 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:03:57.249 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 .. 2023-09-12T16:03:57.249 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:03:57.249 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-09-12T16:03:57.249 INFO:teuthology.orchestra.run.smithi130.stdout:total 8 2023-09-12T16:03:57.249 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:03:57.250 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 12 16:03 .. 2023-09-12T16:03:57.250 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 0000 2023-09-12T16:03:57.250 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-09-12T16:03:57.378 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i' 2023-09-12T16:03:57.423 INFO:teuthology.orchestra.run.smithi130.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-09-12T16:03:57.427 INFO:teuthology.orchestra.run.smithi130.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-09-12T16:03:57.454 INFO:teuthology.orchestra.run.smithi130.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-09-12T16:03:57.458 INFO:teuthology.orchestra.run.smithi130.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-09-12T16:03:57.633 INFO:teuthology.orchestra.run.smithi130.stderr:creating tables... 2023-09-12T16:03:57.692 INFO:teuthology.orchestra.run.smithi130.stderr:100000 of 50000000 tuples (0%) done (elapsed 0.06 s, remaining 28.93 s) 2023-09-12T16:03:57.873 INFO:teuthology.orchestra.run.smithi130.stderr:200000 of 50000000 tuples (0%) done (elapsed 0.24 s, remaining 59.41 s) 2023-09-12T16:03:57.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:57 smithi130 ceph-mon[98670]: pgmap v323: 129 pgs: 129 active+clean; 2.1 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 198 KiB/s wr, 10 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:03:57.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:57 smithi159 ceph-mon[102522]: pgmap v323: 129 pgs: 129 active+clean; 2.1 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 198 KiB/s wr, 10 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:03:57.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:57 smithi203 ceph-mon[103431]: pgmap v323: 129 pgs: 129 active+clean; 2.1 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 198 KiB/s wr, 10 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:03:58.061 INFO:teuthology.orchestra.run.smithi130.stderr:300000 of 50000000 tuples (0%) done (elapsed 0.43 s, remaining 70.73 s) 2023-09-12T16:03:58.122 INFO:teuthology.orchestra.run.smithi130.stderr:400000 of 50000000 tuples (0%) done (elapsed 0.49 s, remaining 60.48 s) 2023-09-12T16:03:58.294 INFO:teuthology.orchestra.run.smithi130.stderr:500000 of 50000000 tuples (1%) done (elapsed 0.66 s, remaining 65.32 s) 2023-09-12T16:03:58.456 INFO:teuthology.orchestra.run.smithi130.stderr:600000 of 50000000 tuples (1%) done (elapsed 0.82 s, remaining 67.63 s) 2023-09-12T16:03:58.535 INFO:teuthology.orchestra.run.smithi130.stderr:700000 of 50000000 tuples (1%) done (elapsed 0.90 s, remaining 63.39 s) 2023-09-12T16:03:58.723 INFO:teuthology.orchestra.run.smithi130.stderr:800000 of 50000000 tuples (1%) done (elapsed 1.09 s, remaining 66.94 s) 2023-09-12T16:03:58.889 INFO:teuthology.orchestra.run.smithi130.stderr:900000 of 50000000 tuples (1%) done (elapsed 1.25 s, remaining 68.45 s) 2023-09-12T16:03:58.971 INFO:teuthology.orchestra.run.smithi130.stderr:1000000 of 50000000 tuples (2%) done (elapsed 1.34 s, remaining 65.49 s) 2023-09-12T16:03:59.149 INFO:teuthology.orchestra.run.smithi130.stderr:1100000 of 50000000 tuples (2%) done (elapsed 1.51 s, remaining 67.30 s) 2023-09-12T16:03:59.213 INFO:teuthology.orchestra.run.smithi130.stderr:1200000 of 50000000 tuples (2%) done (elapsed 1.58 s, remaining 64.18 s) 2023-09-12T16:03:59.396 INFO:teuthology.orchestra.run.smithi130.stderr:1300000 of 50000000 tuples (2%) done (elapsed 1.76 s, remaining 66.01 s) 2023-09-12T16:03:59.556 INFO:teuthology.orchestra.run.smithi130.stderr:1400000 of 50000000 tuples (2%) done (elapsed 1.92 s, remaining 66.72 s) 2023-09-12T16:03:59.651 INFO:teuthology.orchestra.run.smithi130.stderr:1500000 of 50000000 tuples (3%) done (elapsed 2.02 s, remaining 65.19 s) 2023-09-12T16:03:59.825 INFO:teuthology.orchestra.run.smithi130.stderr:1600000 of 50000000 tuples (3%) done (elapsed 2.19 s, remaining 66.26 s) 2023-09-12T16:03:59.996 INFO:teuthology.orchestra.run.smithi130.stderr:1700000 of 50000000 tuples (3%) done (elapsed 2.36 s, remaining 67.11 s) 2023-09-12T16:04:00.074 INFO:teuthology.orchestra.run.smithi130.stderr:1800000 of 50000000 tuples (3%) done (elapsed 2.44 s, remaining 65.33 s) 2023-09-12T16:04:00.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:03:59 smithi130 ceph-mon[98670]: pgmap v324: 129 pgs: 129 active+clean; 22 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 62 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:04:00.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:03:59 smithi159 ceph-mon[102522]: pgmap v324: 129 pgs: 129 active+clean; 22 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 62 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:04:00.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:03:59 smithi203 ceph-mon[103431]: pgmap v324: 129 pgs: 129 active+clean; 22 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 62 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:04:00.251 INFO:teuthology.orchestra.run.smithi130.stderr:1900000 of 50000000 tuples (3%) done (elapsed 2.62 s, remaining 66.26 s) 2023-09-12T16:04:00.350 INFO:teuthology.orchestra.run.smithi130.stderr:2000000 of 50000000 tuples (4%) done (elapsed 2.71 s, remaining 65.14 s) 2023-09-12T16:04:00.499 INFO:teuthology.orchestra.run.smithi130.stderr:2100000 of 50000000 tuples (4%) done (elapsed 2.86 s, remaining 65.34 s) 2023-09-12T16:04:00.680 INFO:teuthology.orchestra.run.smithi130.stderr:2200000 of 50000000 tuples (4%) done (elapsed 3.05 s, remaining 66.17 s) 2023-09-12T16:04:00.776 INFO:teuthology.orchestra.run.smithi130.stderr:2300000 of 50000000 tuples (4%) done (elapsed 3.14 s, remaining 65.15 s) 2023-09-12T16:04:00.941 INFO:teuthology.orchestra.run.smithi130.stderr:2400000 of 50000000 tuples (4%) done (elapsed 3.31 s, remaining 65.58 s) 2023-09-12T16:04:01.113 INFO:teuthology.orchestra.run.smithi130.stderr:2500000 of 50000000 tuples (5%) done (elapsed 3.48 s, remaining 66.10 s) 2023-09-12T16:04:01.179 INFO:teuthology.orchestra.run.smithi130.stderr:2600000 of 50000000 tuples (5%) done (elapsed 3.54 s, remaining 64.62 s) 2023-09-12T16:04:01.352 INFO:teuthology.orchestra.run.smithi130.stderr:2700000 of 50000000 tuples (5%) done (elapsed 3.72 s, remaining 65.13 s) 2023-09-12T16:04:01.507 INFO:teuthology.orchestra.run.smithi130.stderr:2800000 of 50000000 tuples (5%) done (elapsed 3.87 s, remaining 65.28 s) 2023-09-12T16:04:01.585 INFO:teuthology.orchestra.run.smithi130.stderr:2900000 of 50000000 tuples (5%) done (elapsed 3.95 s, remaining 64.16 s) 2023-09-12T16:04:01.764 INFO:teuthology.orchestra.run.smithi130.stderr:3000000 of 50000000 tuples (6%) done (elapsed 4.13 s, remaining 64.70 s) 2023-09-12T16:04:01.925 INFO:teuthology.orchestra.run.smithi130.stderr:3100000 of 50000000 tuples (6%) done (elapsed 4.29 s, remaining 64.90 s) 2023-09-12T16:04:02.008 INFO:teuthology.orchestra.run.smithi130.stderr:3200000 of 50000000 tuples (6%) done (elapsed 4.37 s, remaining 63.96 s) 2023-09-12T16:04:02.187 INFO:teuthology.orchestra.run.smithi130.stderr:3300000 of 50000000 tuples (6%) done (elapsed 4.55 s, remaining 64.42 s) 2023-09-12T16:04:02.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:01 smithi130 ceph-mon[98670]: pgmap v325: 129 pgs: 129 active+clean; 103 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 16 MiB/s wr, 90 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:04:02.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:01 smithi159 ceph-mon[102522]: pgmap v325: 129 pgs: 129 active+clean; 103 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 16 MiB/s wr, 90 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:04:02.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:01 smithi203 ceph-mon[103431]: pgmap v325: 129 pgs: 129 active+clean; 103 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 16 MiB/s wr, 90 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-12T16:04:02.356 INFO:teuthology.orchestra.run.smithi130.stderr:3400000 of 50000000 tuples (6%) done (elapsed 4.72 s, remaining 64.72 s) 2023-09-12T16:04:02.440 INFO:teuthology.orchestra.run.smithi130.stderr:3500000 of 50000000 tuples (7%) done (elapsed 4.81 s, remaining 63.85 s) 2023-09-12T16:04:02.617 INFO:teuthology.orchestra.run.smithi130.stderr:3600000 of 50000000 tuples (7%) done (elapsed 4.98 s, remaining 64.22 s) 2023-09-12T16:04:02.686 INFO:teuthology.orchestra.run.smithi130.stderr:3700000 of 50000000 tuples (7%) done (elapsed 5.05 s, remaining 63.22 s) 2023-09-12T16:04:02.858 INFO:teuthology.orchestra.run.smithi130.stderr:3800000 of 50000000 tuples (7%) done (elapsed 5.22 s, remaining 63.51 s) 2023-09-12T16:04:03.024 INFO:teuthology.orchestra.run.smithi130.stderr:3900000 of 50000000 tuples (7%) done (elapsed 5.39 s, remaining 63.71 s) 2023-09-12T16:04:03.129 INFO:teuthology.orchestra.run.smithi130.stderr:4000000 of 50000000 tuples (8%) done (elapsed 5.49 s, remaining 63.19 s) 2023-09-12T16:04:03.293 INFO:teuthology.orchestra.run.smithi130.stderr:4100000 of 50000000 tuples (8%) done (elapsed 5.66 s, remaining 63.35 s) 2023-09-12T16:04:03.454 INFO:teuthology.orchestra.run.smithi130.stderr:4200000 of 50000000 tuples (8%) done (elapsed 5.82 s, remaining 63.46 s) 2023-09-12T16:04:03.537 INFO:teuthology.orchestra.run.smithi130.stderr:4300000 of 50000000 tuples (8%) done (elapsed 5.90 s, remaining 62.73 s) 2023-09-12T16:04:03.720 INFO:teuthology.orchestra.run.smithi130.stderr:4400000 of 50000000 tuples (8%) done (elapsed 6.09 s, remaining 63.07 s) 2023-09-12T16:04:03.891 INFO:teuthology.orchestra.run.smithi130.stderr:4500000 of 50000000 tuples (9%) done (elapsed 6.26 s, remaining 63.26 s) 2023-09-12T16:04:03.967 INFO:teuthology.orchestra.run.smithi130.stderr:4600000 of 50000000 tuples (9%) done (elapsed 6.33 s, remaining 62.50 s) 2023-09-12T16:04:04.155 INFO:teuthology.orchestra.run.smithi130.stderr:4700000 of 50000000 tuples (9%) done (elapsed 6.52 s, remaining 62.85 s) 2023-09-12T16:04:04.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:03 smithi130 ceph-mon[98670]: pgmap v326: 129 pgs: 129 active+clean; 263 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 45 MiB/s wr, 181 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-12T16:04:04.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:03 smithi159 ceph-mon[102522]: pgmap v326: 129 pgs: 129 active+clean; 263 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 45 MiB/s wr, 181 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-12T16:04:04.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:03 smithi203 ceph-mon[103431]: pgmap v326: 129 pgs: 129 active+clean; 263 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 45 MiB/s wr, 181 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-12T16:04:04.317 INFO:teuthology.orchestra.run.smithi130.stderr:4800000 of 50000000 tuples (9%) done (elapsed 6.68 s, remaining 62.93 s) 2023-09-12T16:04:04.396 INFO:teuthology.orchestra.run.smithi130.stderr:4900000 of 50000000 tuples (9%) done (elapsed 6.76 s, remaining 62.23 s) 2023-09-12T16:04:04.550 INFO:teuthology.orchestra.run.smithi130.stderr:5000000 of 50000000 tuples (10%) done (elapsed 6.92 s, remaining 62.24 s) 2023-09-12T16:04:04.635 INFO:teuthology.orchestra.run.smithi130.stderr:5100000 of 50000000 tuples (10%) done (elapsed 7.00 s, remaining 61.63 s) 2023-09-12T16:04:04.817 INFO:teuthology.orchestra.run.smithi130.stderr:5200000 of 50000000 tuples (10%) done (elapsed 7.18 s, remaining 61.88 s) 2023-09-12T16:04:05.000 INFO:teuthology.orchestra.run.smithi130.stderr:5300000 of 50000000 tuples (10%) done (elapsed 7.37 s, remaining 62.12 s) 2023-09-12T16:04:05.110 INFO:teuthology.orchestra.run.smithi130.stderr:5400000 of 50000000 tuples (10%) done (elapsed 7.48 s, remaining 61.75 s) 2023-09-12T16:04:05.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:04 smithi130 ceph-mon[98670]: scrub summary: idle 2023-09-12T16:04:05.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:04 smithi159 ceph-mon[102522]: scrub summary: idle 2023-09-12T16:04:05.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:04 smithi203 ceph-mon[103431]: scrub summary: idle 2023-09-12T16:04:05.269 INFO:teuthology.orchestra.run.smithi130.stderr:5500000 of 50000000 tuples (11%) done (elapsed 7.63 s, remaining 61.77 s) 2023-09-12T16:04:05.441 INFO:teuthology.orchestra.run.smithi130.stderr:5600000 of 50000000 tuples (11%) done (elapsed 7.81 s, remaining 61.90 s) 2023-09-12T16:04:05.512 INFO:teuthology.orchestra.run.smithi130.stderr:5700000 of 50000000 tuples (11%) done (elapsed 7.88 s, remaining 61.22 s) 2023-09-12T16:04:05.695 INFO:teuthology.orchestra.run.smithi130.stderr:5800000 of 50000000 tuples (11%) done (elapsed 8.06 s, remaining 61.43 s) 2023-09-12T16:04:06.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:05 smithi130 ceph-mon[98670]: pgmap v327: 129 pgs: 129 active+clean; 387 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 62 MiB/s wr, 171 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-12T16:04:06.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:05 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:06.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:05 smithi159 ceph-mon[102522]: pgmap v327: 129 pgs: 129 active+clean; 387 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 62 MiB/s wr, 171 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-12T16:04:06.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:05 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:06.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:05 smithi203 ceph-mon[103431]: pgmap v327: 129 pgs: 129 active+clean; 387 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 62 MiB/s wr, 171 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-12T16:04:06.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:05 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:06.597 INFO:teuthology.orchestra.run.smithi130.stderr:5900000 of 50000000 tuples (11%) done (elapsed 8.96 s, remaining 66.99 s) 2023-09-12T16:04:06.677 INFO:teuthology.orchestra.run.smithi130.stderr:6000000 of 50000000 tuples (12%) done (elapsed 9.04 s, remaining 66.31 s) 2023-09-12T16:04:06.850 INFO:teuthology.orchestra.run.smithi130.stderr:6100000 of 50000000 tuples (12%) done (elapsed 9.22 s, remaining 66.32 s) 2023-09-12T16:04:06.921 INFO:teuthology.orchestra.run.smithi130.stderr:6200000 of 50000000 tuples (12%) done (elapsed 9.29 s, remaining 65.61 s) 2023-09-12T16:04:07.103 INFO:teuthology.orchestra.run.smithi130.stderr:6300000 of 50000000 tuples (12%) done (elapsed 9.47 s, remaining 65.68 s) 2023-09-12T16:04:07.280 INFO:teuthology.orchestra.run.smithi130.stderr:6400000 of 50000000 tuples (12%) done (elapsed 9.65 s, remaining 65.71 s) 2023-09-12T16:04:07.351 INFO:teuthology.orchestra.run.smithi130.stderr:6500000 of 50000000 tuples (13%) done (elapsed 9.72 s, remaining 65.02 s) 2023-09-12T16:04:07.543 INFO:teuthology.orchestra.run.smithi130.stderr:6600000 of 50000000 tuples (13%) done (elapsed 9.91 s, remaining 65.15 s) 2023-09-12T16:04:07.711 INFO:teuthology.orchestra.run.smithi130.stderr:6700000 of 50000000 tuples (13%) done (elapsed 10.08 s, remaining 65.12 s) 2023-09-12T16:04:07.783 INFO:teuthology.orchestra.run.smithi130.stderr:6800000 of 50000000 tuples (13%) done (elapsed 10.15 s, remaining 64.47 s) 2023-09-12T16:04:07.971 INFO:teuthology.orchestra.run.smithi130.stderr:6900000 of 50000000 tuples (13%) done (elapsed 10.34 s, remaining 64.56 s) 2023-09-12T16:04:07.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:07 smithi159 ceph-mon[102522]: pgmap v328: 129 pgs: 129 active+clean; 538 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 82 MiB/s wr, 179 op/s 2023-09-12T16:04:07.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:07 smithi130 ceph-mon[98670]: pgmap v328: 129 pgs: 129 active+clean; 538 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 82 MiB/s wr, 179 op/s 2023-09-12T16:04:07.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:07 smithi203 ceph-mon[103431]: pgmap v328: 129 pgs: 129 active+clean; 538 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 82 MiB/s wr, 179 op/s 2023-09-12T16:04:08.148 INFO:teuthology.orchestra.run.smithi130.stderr:7000000 of 50000000 tuples (14%) done (elapsed 10.51 s, remaining 64.59 s) 2023-09-12T16:04:08.214 INFO:teuthology.orchestra.run.smithi130.stderr:7100000 of 50000000 tuples (14%) done (elapsed 10.58 s, remaining 63.92 s) 2023-09-12T16:04:08.406 INFO:teuthology.orchestra.run.smithi130.stderr:7200000 of 50000000 tuples (14%) done (elapsed 10.77 s, remaining 64.03 s) 2023-09-12T16:04:08.567 INFO:teuthology.orchestra.run.smithi130.stderr:7300000 of 50000000 tuples (14%) done (elapsed 10.93 s, remaining 63.95 s) 2023-09-12T16:04:08.640 INFO:teuthology.orchestra.run.smithi130.stderr:7400000 of 50000000 tuples (14%) done (elapsed 11.01 s, remaining 63.36 s) 2023-09-12T16:04:08.820 INFO:teuthology.orchestra.run.smithi130.stderr:7500000 of 50000000 tuples (15%) done (elapsed 11.19 s, remaining 63.39 s) 2023-09-12T16:04:08.892 INFO:teuthology.orchestra.run.smithi130.stderr:7600000 of 50000000 tuples (15%) done (elapsed 11.26 s, remaining 62.80 s) 2023-09-12T16:04:09.063 INFO:teuthology.orchestra.run.smithi130.stderr:7700000 of 50000000 tuples (15%) done (elapsed 11.43 s, remaining 62.78 s) 2023-09-12T16:04:09.221 INFO:teuthology.orchestra.run.smithi130.stderr:7800000 of 50000000 tuples (15%) done (elapsed 11.59 s, remaining 62.69 s) 2023-09-12T16:04:09.304 INFO:teuthology.orchestra.run.smithi130.stderr:7900000 of 50000000 tuples (15%) done (elapsed 11.67 s, remaining 62.19 s) 2023-09-12T16:04:09.487 INFO:teuthology.orchestra.run.smithi130.stderr:8000000 of 50000000 tuples (16%) done (elapsed 11.85 s, remaining 62.22 s) 2023-09-12T16:04:09.655 INFO:teuthology.orchestra.run.smithi130.stderr:8100000 of 50000000 tuples (16%) done (elapsed 12.02 s, remaining 62.18 s) 2023-09-12T16:04:09.741 INFO:teuthology.orchestra.run.smithi130.stderr:8200000 of 50000000 tuples (16%) done (elapsed 12.11 s, remaining 61.72 s) 2023-09-12T16:04:09.918 INFO:teuthology.orchestra.run.smithi130.stderr:8300000 of 50000000 tuples (16%) done (elapsed 12.28 s, remaining 61.72 s) 2023-09-12T16:04:10.087 INFO:teuthology.orchestra.run.smithi130.stderr:8400000 of 50000000 tuples (16%) done (elapsed 12.45 s, remaining 61.67 s) 2023-09-12T16:04:10.172 INFO:teuthology.orchestra.run.smithi130.stderr:8500000 of 50000000 tuples (17%) done (elapsed 12.54 s, remaining 61.21 s) 2023-09-12T16:04:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:09 smithi130 ceph-mon[98670]: pgmap v329: 129 pgs: 129 active+clean; 983 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 130 MiB/s wr, 208 op/s 2023-09-12T16:04:10.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:09 smithi159 ceph-mon[102522]: pgmap v329: 129 pgs: 129 active+clean; 983 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 130 MiB/s wr, 208 op/s 2023-09-12T16:04:10.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:09 smithi203 ceph-mon[103431]: pgmap v329: 129 pgs: 129 active+clean; 983 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 130 MiB/s wr, 208 op/s 2023-09-12T16:04:10.356 INFO:teuthology.orchestra.run.smithi130.stderr:8600000 of 50000000 tuples (17%) done (elapsed 12.72 s, remaining 61.24 s) 2023-09-12T16:04:10.461 INFO:teuthology.orchestra.run.smithi130.stderr:8700000 of 50000000 tuples (17%) done (elapsed 12.83 s, remaining 60.89 s) 2023-09-12T16:04:10.623 INFO:teuthology.orchestra.run.smithi130.stderr:8800000 of 50000000 tuples (17%) done (elapsed 12.99 s, remaining 60.81 s) 2023-09-12T16:04:10.798 INFO:teuthology.orchestra.run.smithi130.stderr:8900000 of 50000000 tuples (17%) done (elapsed 13.16 s, remaining 60.79 s) 2023-09-12T16:04:10.879 INFO:teuthology.orchestra.run.smithi130.stderr:9000000 of 50000000 tuples (18%) done (elapsed 13.24 s, remaining 60.34 s) 2023-09-12T16:04:11.054 INFO:teuthology.orchestra.run.smithi130.stderr:9100000 of 50000000 tuples (18%) done (elapsed 13.42 s, remaining 60.32 s) 2023-09-12T16:04:11.223 INFO:teuthology.orchestra.run.smithi130.stderr:9200000 of 50000000 tuples (18%) done (elapsed 13.59 s, remaining 60.26 s) 2023-09-12T16:04:11.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:10 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:11.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:10 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:04:11.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:10 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:11.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:10 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:04:11.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:10 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:11.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:10 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:04:11.303 INFO:teuthology.orchestra.run.smithi130.stderr:9300000 of 50000000 tuples (18%) done (elapsed 13.67 s, remaining 59.82 s) 2023-09-12T16:04:11.478 INFO:teuthology.orchestra.run.smithi130.stderr:9400000 of 50000000 tuples (18%) done (elapsed 13.84 s, remaining 59.79 s) 2023-09-12T16:04:11.587 INFO:teuthology.orchestra.run.smithi130.stderr:9500000 of 50000000 tuples (19%) done (elapsed 13.95 s, remaining 59.48 s) 2023-09-12T16:04:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:9600000 of 50000000 tuples (19%) done (elapsed 14.12 s, remaining 59.41 s) 2023-09-12T16:04:11.916 INFO:teuthology.orchestra.run.smithi130.stderr:9700000 of 50000000 tuples (19%) done (elapsed 14.28 s, remaining 59.34 s) 2023-09-12T16:04:12.053 INFO:teuthology.orchestra.run.smithi130.stderr:9800000 of 50000000 tuples (19%) done (elapsed 14.42 s, remaining 59.14 s) 2023-09-12T16:04:12.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:11 smithi159 ceph-mon[102522]: pgmap v330: 129 pgs: 129 active+clean; 1.5 GiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 183 MiB/s wr, 189 op/s 2023-09-12T16:04:12.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:11 smithi130 ceph-mon[98670]: pgmap v330: 129 pgs: 129 active+clean; 1.5 GiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 183 MiB/s wr, 189 op/s 2023-09-12T16:04:12.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:11 smithi203 ceph-mon[103431]: pgmap v330: 129 pgs: 129 active+clean; 1.5 GiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 183 MiB/s wr, 189 op/s 2023-09-12T16:04:12.708 INFO:teuthology.orchestra.run.smithi130.stderr:9900000 of 50000000 tuples (19%) done (elapsed 15.07 s, remaining 61.05 s) 2023-09-12T16:04:12.830 INFO:teuthology.orchestra.run.smithi130.stderr:10000000 of 50000000 tuples (20%) done (elapsed 15.20 s, remaining 60.78 s) 2023-09-12T16:04:12.932 INFO:teuthology.orchestra.run.smithi130.stderr:10100000 of 50000000 tuples (20%) done (elapsed 15.30 s, remaining 60.43 s) 2023-09-12T16:04:13.045 INFO:teuthology.orchestra.run.smithi130.stderr:10200000 of 50000000 tuples (20%) done (elapsed 15.41 s, remaining 60.13 s) 2023-09-12T16:04:13.174 INFO:teuthology.orchestra.run.smithi130.stderr:10300000 of 50000000 tuples (20%) done (elapsed 15.54 s, remaining 59.89 s) 2023-09-12T16:04:13.280 INFO:teuthology.orchestra.run.smithi130.stderr:10400000 of 50000000 tuples (20%) done (elapsed 15.65 s, remaining 59.57 s) 2023-09-12T16:04:13.760 INFO:teuthology.orchestra.run.smithi130.stderr:10500000 of 50000000 tuples (21%) done (elapsed 16.13 s, remaining 60.66 s) 2023-09-12T16:04:14.205 INFO:teuthology.orchestra.run.smithi130.stderr:10600000 of 50000000 tuples (21%) done (elapsed 16.57 s, remaining 61.59 s) 2023-09-12T16:04:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: pgmap v331: 129 pgs: 129 active+clean; 1.6 GiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 200 MiB/s wr, 183 op/s 2023-09-12T16:04:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:04:14.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:04:14.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:13 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: pgmap v331: 129 pgs: 129 active+clean; 1.6 GiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 200 MiB/s wr, 183 op/s 2023-09-12T16:04:14.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:04:14.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:04:14.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:13 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: pgmap v331: 129 pgs: 129 active+clean; 1.6 GiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 200 MiB/s wr, 183 op/s 2023-09-12T16:04:14.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:04:14.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:04:14.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:13 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:04:14.310 INFO:teuthology.orchestra.run.smithi130.stderr:10700000 of 50000000 tuples (21%) done (elapsed 16.68 s, remaining 61.25 s) 2023-09-12T16:04:14.391 INFO:teuthology.orchestra.run.smithi130.stderr:10800000 of 50000000 tuples (21%) done (elapsed 16.76 s, remaining 60.82 s) 2023-09-12T16:04:14.519 INFO:teuthology.orchestra.run.smithi130.stderr:10900000 of 50000000 tuples (21%) done (elapsed 16.88 s, remaining 60.57 s) 2023-09-12T16:04:14.585 INFO:teuthology.orchestra.run.smithi130.stderr:11000000 of 50000000 tuples (22%) done (elapsed 16.95 s, remaining 60.10 s) 2023-09-12T16:04:14.694 INFO:teuthology.orchestra.run.smithi130.stderr:11100000 of 50000000 tuples (22%) done (elapsed 17.06 s, remaining 59.78 s) 2023-09-12T16:04:14.795 INFO:teuthology.orchestra.run.smithi130.stderr:11200000 of 50000000 tuples (22%) done (elapsed 17.16 s, remaining 59.45 s) 2023-09-12T16:04:15.084 INFO:teuthology.orchestra.run.smithi130.stderr:11300000 of 50000000 tuples (22%) done (elapsed 17.45 s, remaining 59.76 s) 2023-09-12T16:04:15.527 INFO:teuthology.orchestra.run.smithi130.stderr:11400000 of 50000000 tuples (22%) done (elapsed 17.89 s, remaining 60.59 s) 2023-09-12T16:04:15.630 INFO:teuthology.orchestra.run.smithi130.stderr:11500000 of 50000000 tuples (23%) done (elapsed 18.00 s, remaining 60.25 s) 2023-09-12T16:04:15.735 INFO:teuthology.orchestra.run.smithi130.stderr:11600000 of 50000000 tuples (23%) done (elapsed 18.10 s, remaining 59.92 s) 2023-09-12T16:04:15.845 INFO:teuthology.orchestra.run.smithi130.stderr:11700000 of 50000000 tuples (23%) done (elapsed 18.21 s, remaining 59.61 s) 2023-09-12T16:04:15.948 INFO:teuthology.orchestra.run.smithi130.stderr:11800000 of 50000000 tuples (23%) done (elapsed 18.31 s, remaining 59.29 s) 2023-09-12T16:04:16.030 INFO:teuthology.orchestra.run.smithi130.stderr:11900000 of 50000000 tuples (23%) done (elapsed 18.40 s, remaining 58.90 s) 2023-09-12T16:04:16.125 INFO:teuthology.orchestra.run.smithi130.stderr:12000000 of 50000000 tuples (24%) done (elapsed 18.49 s, remaining 58.55 s) 2023-09-12T16:04:16.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:15 smithi130 ceph-mon[98670]: pgmap v332: 129 pgs: 129 active+clean; 1.9 GiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 204 MiB/s wr, 118 op/s 2023-09-12T16:04:16.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:15 smithi159 ceph-mon[102522]: pgmap v332: 129 pgs: 129 active+clean; 1.9 GiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 204 MiB/s wr, 118 op/s 2023-09-12T16:04:16.227 INFO:teuthology.orchestra.run.smithi130.stderr:12100000 of 50000000 tuples (24%) done (elapsed 18.59 s, remaining 58.24 s) 2023-09-12T16:04:16.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:15 smithi203 ceph-mon[103431]: pgmap v332: 129 pgs: 129 active+clean; 1.9 GiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 204 MiB/s wr, 118 op/s 2023-09-12T16:04:16.586 INFO:teuthology.orchestra.run.smithi130.stderr:12200000 of 50000000 tuples (24%) done (elapsed 18.95 s, remaining 58.72 s) 2023-09-12T16:04:17.281 INFO:teuthology.orchestra.run.smithi130.stderr:12300000 of 50000000 tuples (24%) done (elapsed 19.65 s, remaining 60.22 s) 2023-09-12T16:04:17.362 INFO:teuthology.orchestra.run.smithi130.stderr:12400000 of 50000000 tuples (24%) done (elapsed 19.73 s, remaining 59.82 s) 2023-09-12T16:04:17.494 INFO:teuthology.orchestra.run.smithi130.stderr:12500000 of 50000000 tuples (25%) done (elapsed 19.86 s, remaining 59.58 s) 2023-09-12T16:04:17.565 INFO:teuthology.orchestra.run.smithi130.stderr:12600000 of 50000000 tuples (25%) done (elapsed 19.93 s, remaining 59.16 s) 2023-09-12T16:04:17.672 INFO:teuthology.orchestra.run.smithi130.stderr:12700000 of 50000000 tuples (25%) done (elapsed 20.04 s, remaining 58.85 s) 2023-09-12T16:04:17.781 INFO:teuthology.orchestra.run.smithi130.stderr:12800000 of 50000000 tuples (25%) done (elapsed 20.15 s, remaining 58.55 s) 2023-09-12T16:04:17.878 INFO:teuthology.orchestra.run.smithi130.stderr:12900000 of 50000000 tuples (25%) done (elapsed 20.24 s, remaining 58.22 s) 2023-09-12T16:04:17.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:17 smithi130 ceph-mon[98670]: pgmap v333: 129 pgs: 129 active+clean; 2.1 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 209 MiB/s wr, 137 op/s 2023-09-12T16:04:17.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:17 smithi159 ceph-mon[102522]: pgmap v333: 129 pgs: 129 active+clean; 2.1 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 209 MiB/s wr, 137 op/s 2023-09-12T16:04:17.980 INFO:teuthology.orchestra.run.smithi130.stderr:13000000 of 50000000 tuples (26%) done (elapsed 20.35 s, remaining 57.91 s) 2023-09-12T16:04:17.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:17 smithi203 ceph-mon[103431]: pgmap v333: 129 pgs: 129 active+clean; 2.1 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 209 MiB/s wr, 137 op/s 2023-09-12T16:04:18.089 INFO:teuthology.orchestra.run.smithi130.stderr:13100000 of 50000000 tuples (26%) done (elapsed 20.45 s, remaining 57.62 s) 2023-09-12T16:04:18.185 INFO:teuthology.orchestra.run.smithi130.stderr:13200000 of 50000000 tuples (26%) done (elapsed 20.55 s, remaining 57.29 s) 2023-09-12T16:04:18.412 INFO:teuthology.orchestra.run.smithi130.stderr:13300000 of 50000000 tuples (26%) done (elapsed 20.78 s, remaining 57.33 s) 2023-09-12T16:04:19.045 INFO:teuthology.orchestra.run.smithi130.stderr:13400000 of 50000000 tuples (26%) done (elapsed 21.41 s, remaining 58.48 s) 2023-09-12T16:04:19.114 INFO:teuthology.orchestra.run.smithi130.stderr:13500000 of 50000000 tuples (27%) done (elapsed 21.48 s, remaining 58.07 s) 2023-09-12T16:04:19.234 INFO:teuthology.orchestra.run.smithi130.stderr:13600000 of 50000000 tuples (27%) done (elapsed 21.60 s, remaining 57.81 s) 2023-09-12T16:04:19.776 INFO:teuthology.orchestra.run.smithi130.stderr:13700000 of 50000000 tuples (27%) done (elapsed 22.14 s, remaining 58.66 s) 2023-09-12T16:04:19.925 INFO:teuthology.orchestra.run.smithi130.stderr:13800000 of 50000000 tuples (27%) done (elapsed 22.29 s, remaining 58.47 s) 2023-09-12T16:04:19.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:19 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:19.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:19 smithi130 ceph-mon[98670]: pgmap v334: 129 pgs: 129 active+clean; 2.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 210 MiB/s wr, 167 op/s 2023-09-12T16:04:19.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:19 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:19.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:19 smithi159 ceph-mon[102522]: pgmap v334: 129 pgs: 129 active+clean; 2.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 210 MiB/s wr, 167 op/s 2023-09-12T16:04:19.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:19 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:19.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:19 smithi203 ceph-mon[103431]: pgmap v334: 129 pgs: 129 active+clean; 2.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 210 MiB/s wr, 167 op/s 2023-09-12T16:04:20.019 INFO:teuthology.orchestra.run.smithi130.stderr:13900000 of 50000000 tuples (27%) done (elapsed 22.38 s, remaining 58.14 s) 2023-09-12T16:04:20.120 INFO:teuthology.orchestra.run.smithi130.stderr:14000000 of 50000000 tuples (28%) done (elapsed 22.49 s, remaining 57.82 s) 2023-09-12T16:04:20.207 INFO:teuthology.orchestra.run.smithi130.stderr:14100000 of 50000000 tuples (28%) done (elapsed 22.57 s, remaining 57.47 s) 2023-09-12T16:04:20.313 INFO:teuthology.orchestra.run.smithi130.stderr:14200000 of 50000000 tuples (28%) done (elapsed 22.68 s, remaining 57.17 s) 2023-09-12T16:04:20.420 INFO:teuthology.orchestra.run.smithi130.stderr:14300000 of 50000000 tuples (28%) done (elapsed 22.79 s, remaining 56.88 s) 2023-09-12T16:04:20.524 INFO:teuthology.orchestra.run.smithi130.stderr:14400000 of 50000000 tuples (28%) done (elapsed 22.89 s, remaining 56.59 s) 2023-09-12T16:04:20.625 INFO:teuthology.orchestra.run.smithi130.stderr:14500000 of 50000000 tuples (29%) done (elapsed 22.99 s, remaining 56.29 s) 2023-09-12T16:04:20.744 INFO:teuthology.orchestra.run.smithi130.stderr:14600000 of 50000000 tuples (29%) done (elapsed 23.11 s, remaining 56.03 s) 2023-09-12T16:04:20.844 INFO:teuthology.orchestra.run.smithi130.stderr:14700000 of 50000000 tuples (29%) done (elapsed 23.21 s, remaining 55.74 s) 2023-09-12T16:04:20.947 INFO:teuthology.orchestra.run.smithi130.stderr:14800000 of 50000000 tuples (29%) done (elapsed 23.31 s, remaining 55.45 s) 2023-09-12T16:04:21.018 INFO:teuthology.orchestra.run.smithi130.stderr:14900000 of 50000000 tuples (29%) done (elapsed 23.38 s, remaining 55.08 s) 2023-09-12T16:04:21.134 INFO:teuthology.orchestra.run.smithi130.stderr:15000000 of 50000000 tuples (30%) done (elapsed 23.50 s, remaining 54.83 s) 2023-09-12T16:04:21.234 INFO:teuthology.orchestra.run.smithi130.stderr:15100000 of 50000000 tuples (30%) done (elapsed 23.60 s, remaining 54.55 s) 2023-09-12T16:04:21.316 INFO:teuthology.orchestra.run.smithi130.stderr:15200000 of 50000000 tuples (30%) done (elapsed 23.68 s, remaining 54.22 s) 2023-09-12T16:04:21.435 INFO:teuthology.orchestra.run.smithi130.stderr:15300000 of 50000000 tuples (30%) done (elapsed 23.80 s, remaining 53.98 s) 2023-09-12T16:04:21.530 INFO:teuthology.orchestra.run.smithi130.stderr:15400000 of 50000000 tuples (30%) done (elapsed 23.90 s, remaining 53.69 s) 2023-09-12T16:04:21.619 INFO:teuthology.orchestra.run.smithi130.stderr:15500000 of 50000000 tuples (31%) done (elapsed 23.98 s, remaining 53.38 s) 2023-09-12T16:04:21.719 INFO:teuthology.orchestra.run.smithi130.stderr:15600000 of 50000000 tuples (31%) done (elapsed 24.09 s, remaining 53.11 s) 2023-09-12T16:04:21.729 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:04:21.831 INFO:teuthology.orchestra.run.smithi130.stderr:15700000 of 50000000 tuples (31%) done (elapsed 24.20 s, remaining 52.86 s) 2023-09-12T16:04:21.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.893+0000 7fb7dd00a700 1 -- 172.21.15.130:0/3736313224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d8103a00 msgr2=0x7fb7d8105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:21.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.893+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/3736313224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d8103a00 0x7fb7d8105df0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009990 tx=0x7fb7cc009ca0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.893+0000 7fb7dd00a700 1 -- 172.21.15.130:0/3736313224 shutdown_connections 2023-09-12T16:04:21.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.893+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/3736313224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7d8106330 0x7fb7d8108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.893+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/3736313224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d81010d0 0x7fb7d81034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.893+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/3736313224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d8103a00 0x7fb7d8105df0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.893+0000 7fb7dd00a700 1 -- 172.21.15.130:0/3736313224 >> 172.21.15.130:0/3736313224 conn(0x7fb7d80faa60 msgr2=0x7fb7d80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:21.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.893+0000 7fb7dd00a700 1 -- 172.21.15.130:0/3736313224 shutdown_connections 2023-09-12T16:04:21.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.893+0000 7fb7dd00a700 1 -- 172.21.15.130:0/3736313224 wait complete. 2023-09-12T16:04:21.898 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.894+0000 7fb7dd00a700 1 Processor -- start 2023-09-12T16:04:21.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.894+0000 7fb7dd00a700 1 -- start start 2023-09-12T16:04:21.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.894+0000 7fb7dd00a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d81010d0 0x7fb7d8196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.894+0000 7fb7dd00a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d8103a00 0x7fb7d8196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.894+0000 7fb7dd00a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7d8106330 0x7fb7d819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.894+0000 7fb7dd00a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d819de80 con 0x7fb7d81010d0 2023-09-12T16:04:21.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.894+0000 7fb7dd00a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d819dfc0 con 0x7fb7d8106330 2023-09-12T16:04:21.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.894+0000 7fb7dd00a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d819e150 con 0x7fb7d8103a00 2023-09-12T16:04:21.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d81010d0 0x7fb7d8196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d81010d0 0x7fb7d8196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55430/0 (socket says 172.21.15.130:55430) 2023-09-12T16:04:21.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d6d9d700 1 -- 172.21.15.130:0/376383854 learned_addr learned my addr 172.21.15.130:0/376383854 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:21.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d759e700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7d8106330 0x7fb7d819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d659c700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d8103a00 0x7fb7d8196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d6d9d700 1 -- 172.21.15.130:0/376383854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d8103a00 msgr2=0x7fb7d8196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:21.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d8103a00 0x7fb7d8196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d6d9d700 1 -- 172.21.15.130:0/376383854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7d8106330 msgr2=0x7fb7d819d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:21.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7d8106330 0x7fb7d819d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d6d9d700 1 -- 172.21.15.130:0/376383854 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0095f0 con 0x7fb7d81010d0 2023-09-12T16:04:21.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d659c700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d8103a00 0x7fb7d8196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:04:21.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d759e700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7d8106330 0x7fb7d819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:04:21.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d81010d0 0x7fb7d8196300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800e9a0 tx=0x7fb7c800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:21.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7bffff700 1 -- 172.21.15.130:0/376383854 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8009790 con 0x7fb7d81010d0 2023-09-12T16:04:21.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7dd00a700 1 -- 172.21.15.130:0/376383854 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d819e430 con 0x7fb7d81010d0 2023-09-12T16:04:21.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7bffff700 1 -- 172.21.15.130:0/376383854 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c800cbd0 con 0x7fb7d81010d0 2023-09-12T16:04:21.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7bffff700 1 -- 172.21.15.130:0/376383854 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8018680 con 0x7fb7d81010d0 2023-09-12T16:04:21.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.895+0000 7fb7dd00a700 1 -- 172.21.15.130:0/376383854 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d819e990 con 0x7fb7d81010d0 2023-09-12T16:04:21.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.897+0000 7fb7bffff700 1 -- 172.21.15.130:0/376383854 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fb7c800cd40 con 0x7fb7d81010d0 2023-09-12T16:04:21.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.898+0000 7fb7bffff700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7c00a06c0 0x7fb7c00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.904 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.898+0000 7fb7bffff700 1 -- 172.21.15.130:0/376383854 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb7c8014070 con 0x7fb7d81010d0 2023-09-12T16:04:21.904 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.898+0000 7fb7d659c700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7c00a06c0 0x7fb7c00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.899+0000 7fb7d659c700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7c00a06c0 0x7fb7c00a2b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b2e0 tx=0x7fb7cc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:21.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7e91d0b80 1 Processor -- start 2023-09-12T16:04:21.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7e91d0b80 1 -- start start 2023-09-12T16:04:21.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea353c00 0x5650ea354020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea352800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea352e00 0x5650ea3532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea2ab730 con 0x5650ea3545a0 2023-09-12T16:04:21.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea2a9050 con 0x5650ea352e00 2023-09-12T16:04:21.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea2a4940 con 0x5650ea353c00 2023-09-12T16:04:21.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea353c00 0x5650ea354020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea353c00 0x5650ea354020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35050/0 (socket says 172.21.15.130:35050) 2023-09-12T16:04:21.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea352800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7d6d9d700 1 -- 172.21.15.130:0/4002202537 learned_addr learned my addr 172.21.15.130:0/4002202537 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:21.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.900+0000 7fb7d759e700 1 --2- 172.21.15.130:0/4002202537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea352e00 0x5650ea3532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.901+0000 7fb7d6d9d700 1 -- 172.21.15.130:0/4002202537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea352e00 msgr2=0x5650ea3532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:21.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.901+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/4002202537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea352e00 0x5650ea3532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.901+0000 7fb7d6d9d700 1 -- 172.21.15.130:0/4002202537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 msgr2=0x5650ea352800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:21.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.901+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/4002202537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea352800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.901+0000 7fb7d6d9d700 1 -- 172.21.15.130:0/4002202537 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650ea0d0de0 con 0x5650ea353c00 2023-09-12T16:04:21.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.901+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/4002202537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea353c00 0x5650ea354020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8096120 tx=0x7fb7c8099110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edd56370f1b6efc server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:21.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.901+0000 7fb7bd7fa700 1 -- 172.21.15.130:0/4002202537 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c809d4e0 con 0x5650ea353c00 2023-09-12T16:04:21.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.901+0000 7fb7bd7fa700 1 -- 172.21.15.130:0/4002202537 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c80a1720 con 0x5650ea353c00 2023-09-12T16:04:21.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.901+0000 7fb7bd7fa700 1 -- 172.21.15.130:0/4002202537 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8098e00 con 0x5650ea353c00 2023-09-12T16:04:21.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/4002202537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea353c00 msgr2=0x5650ea354020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:21.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/4002202537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea353c00 0x5650ea354020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8096120 tx=0x7fb7c8099110 comp rx=0 tx=0).stop 2023-09-12T16:04:21.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/4002202537 shutdown_connections 2023-09-12T16:04:21.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/4002202537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea352e00 0x5650ea3532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/4002202537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea353c00 0x5650ea354020 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/4002202537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea352800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/4002202537 >> 172.21.15.130:0/4002202537 conn(0x5650ea2c7090 msgr2=0x5650ea34d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:21.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/4002202537 shutdown_connections 2023-09-12T16:04:21.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/4002202537 wait complete. 2023-09-12T16:04:21.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 -- start start 2023-09-12T16:04:21.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.902+0000 7fb7e91d0b80 1 Processor -- start 2023-09-12T16:04:21.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea352e00 0x5650ea35c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea35cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea3545a0 0x5650ea363b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea242b70 con 0x5650ea352e00 2023-09-12T16:04:21.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea242870 con 0x5650ea353c00 2023-09-12T16:04:21.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea126920 con 0x5650ea3545a0 2023-09-12T16:04:21.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7e91d0b80 10 client.0 _ll_register_callbacks cb 0x5650ea18ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:21.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea352e00 0x5650ea35c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea352e00 0x5650ea35c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55444/0 (socket says 172.21.15.130:55444) 2023-09-12T16:04:21.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d759e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea3545a0 0x5650ea363b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d6d9d700 1 -- 172.21.15.130:0/890823067 learned_addr learned my addr 172.21.15.130:0/890823067 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:21.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d659c700 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea35cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d759e700 1 -- 172.21.15.130:0/890823067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 msgr2=0x5650ea35cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:21.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d759e700 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea35cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d759e700 1 -- 172.21.15.130:0/890823067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea352e00 msgr2=0x5650ea35c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:21.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d759e700 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea352e00 0x5650ea35c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d759e700 1 -- 172.21.15.130:0/890823067 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d819e990 con 0x5650ea3545a0 2023-09-12T16:04:21.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea352e00 0x5650ea35c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:21.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7d759e700 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea3545a0 0x5650ea363b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5650ea3481d0 tx=0x7fb7d0007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d5ac75b5065a92 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:21.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7baff5700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:04:21.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7baff5700 1 -- 172.21.15.130:0/890823067 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0009e20 con 0x5650ea3545a0 2023-09-12T16:04:21.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7baff5700 1 -- 172.21.15.130:0/890823067 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb7d0010860 con 0x5650ea3545a0 2023-09-12T16:04:21.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.903+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/890823067 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650ea1fc4f0 con 0x5650ea3545a0 2023-09-12T16:04:21.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7baff5700 1 -- 172.21.15.130:0/890823067 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d0010a20 con 0x5650ea3545a0 2023-09-12T16:04:21.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7e91d0b80 10 client.25432 fetch_fsmap learned FSMap version 27 2023-09-12T16:04:21.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/890823067 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5650ea0b9c60 con 0x5650ea3545a0 2023-09-12T16:04:21.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7baff5700 1 -- 172.21.15.130:0/890823067 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650ea1fc4f0 con 0x5650ea3545a0 2023-09-12T16:04:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7baff5700 1 -- 172.21.15.130:0/890823067 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb7d0016b90 con 0x5650ea3545a0 2023-09-12T16:04:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7e91d0b80 10 client.25432 fetch_fsmap finished waiting for FSMap version 27 2023-09-12T16:04:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7e91d0b80 10 client.25432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7e91d0b80 20 client.25432 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5650ea3684f0 0x5650ea36a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7e91d0b80 4 client.25432 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:04:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.904+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/890823067 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650ea2b98a0 con 0x5650ea3684f0 2023-09-12T16:04:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.905+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5650ea3684f0 0x5650ea36a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.906+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5650ea3684f0 0x5650ea36a910 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f4759908da0cf4 server_cookie=cfdf12bbbd14c269 in_seq=0 out_seq=0 2023-09-12T16:04:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.906+0000 7fb7baff5700 10 client.25432 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:21.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.906+0000 7fb7baff5700 1 -- 172.21.15.130:0/890823067 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5650ea2b98a0 con 0x5650ea3684f0 2023-09-12T16:04:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.906+0000 7fb7baff5700 10 client.25432 handle_command_reply: tid=0 2023-09-12T16:04:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.906+0000 7fb7e91d0b80 1 client.25432 shutdown 2023-09-12T16:04:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.906+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/890823067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea3545a0 msgr2=0x5650ea363b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.906+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea3545a0 0x5650ea363b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5650ea3481d0 tx=0x7fb7d0007d30 comp rx=0 tx=0).stop 2023-09-12T16:04:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/890823067 shutdown_connections 2023-09-12T16:04:21.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5650ea3684f0 0x5650ea36a910 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea3545a0 0x5650ea363b40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7baff5700 0 client.25432 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:21.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea352e00 0x5650ea35c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/890823067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea35cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/890823067 >> 172.21.15.130:0/890823067 conn(0x5650ea2c7090 msgr2=0x5650ea34cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:21.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/890823067 shutdown_connections 2023-09-12T16:04:21.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/890823067 wait complete. 2023-09-12T16:04:21.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.907+0000 7fb7e91d0b80 20 client.25432 trim_cache size 0 max 16384 2023-09-12T16:04:21.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.928+0000 7fb7e91d0b80 1 Processor -- start 2023-09-12T16:04:21.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.928+0000 7fb7e91d0b80 1 -- start start 2023-09-12T16:04:21.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.928+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 0x5650ea351700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.928+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea363b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.928+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea351c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.928+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea349cc0 con 0x5650ea3545a0 2023-09-12T16:04:21.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.928+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea347d70 con 0x5650ea353c00 2023-09-12T16:04:21.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.928+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea345400 con 0x5650ea352e00 2023-09-12T16:04:21.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d759e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea351c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d759e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea351c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55456/0 (socket says 172.21.15.130:55456) 2023-09-12T16:04:21.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d759e700 1 -- 172.21.15.130:0/2727444726 learned_addr learned my addr 172.21.15.130:0/2727444726 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:21.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d659c700 1 --2- 172.21.15.130:0/2727444726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea363b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/2727444726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 0x5650ea351700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d759e700 1 -- 172.21.15.130:0/2727444726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 msgr2=0x5650ea351700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:21.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d759e700 1 --2- 172.21.15.130:0/2727444726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 0x5650ea351700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d759e700 1 -- 172.21.15.130:0/2727444726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 msgr2=0x5650ea363b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:21.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d759e700 1 --2- 172.21.15.130:0/2727444726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea363b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d759e700 1 -- 172.21.15.130:0/2727444726 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650ea0f7250 con 0x5650ea3545a0 2023-09-12T16:04:21.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7d759e700 1 --2- 172.21.15.130:0/2727444726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea351c40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009290 tx=0x7fb7d00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadb015bf503350a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:21.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7bdffb700 1 -- 172.21.15.130:0/2727444726 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000d070 con 0x5650ea3545a0 2023-09-12T16:04:21.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7bdffb700 1 -- 172.21.15.130:0/2727444726 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7d0010440 con 0x5650ea3545a0 2023-09-12T16:04:21.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.929+0000 7fb7bdffb700 1 -- 172.21.15.130:0/2727444726 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0010a30 con 0x5650ea3545a0 2023-09-12T16:04:21.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.930+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/2727444726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 msgr2=0x5650ea351c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:21.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.930+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/2727444726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea351c40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009290 tx=0x7fb7d00092c0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.930+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/2727444726 shutdown_connections 2023-09-12T16:04:21.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.930+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/2727444726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea351c40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.930+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/2727444726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 0x5650ea351700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.930+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/2727444726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea363b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.930+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/2727444726 >> 172.21.15.130:0/2727444726 conn(0x5650ea2c7090 msgr2=0x5650ea356ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:21.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.930+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/2727444726 shutdown_connections 2023-09-12T16:04:21.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.930+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/2727444726 wait complete. 2023-09-12T16:04:21.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7e91d0b80 1 -- start start 2023-09-12T16:04:21.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7e91d0b80 1 Processor -- start 2023-09-12T16:04:21.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 0x5650ea3607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea360cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7e91d0b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea3bc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea361230 con 0x5650ea3545a0 2023-09-12T16:04:21.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea3613a0 con 0x5650ea353c00 2023-09-12T16:04:21.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7e91d0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ea3bc810 con 0x5650ea352e00 2023-09-12T16:04:21.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7e91d0b80 10 client.0 _ll_register_callbacks cb 0x7fb7d0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:21.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d759e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea3bc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea360cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea360cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:38718/0 (socket says 172.21.15.130:38718) 2023-09-12T16:04:21.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d759e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea3bc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55462/0 (socket says 172.21.15.130:55462) 2023-09-12T16:04:21.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d659c700 1 -- 172.21.15.130:0/815828201 learned_addr learned my addr 172.21.15.130:0/815828201 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:21.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 0x5650ea3607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d659c700 1 -- 172.21.15.130:0/815828201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 msgr2=0x5650ea3607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:21.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d659c700 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 0x5650ea3607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d659c700 1 -- 172.21.15.130:0/815828201 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 msgr2=0x5650ea3bc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:21.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d659c700 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea3bc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.931+0000 7fb7d659c700 1 -- 172.21.15.130:0/815828201 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650ea0b9c60 con 0x5650ea353c00 2023-09-12T16:04:21.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7d659c700 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea360cf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009990 tx=0x7fb7cc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e262f56e14a0e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:21.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7baff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:04:21.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7baff5700 1 -- 172.21.15.130:0/815828201 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc02d030 con 0x5650ea353c00 2023-09-12T16:04:21.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7baff5700 1 -- 172.21.15.130:0/815828201 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb7cc024820 con 0x5650ea353c00 2023-09-12T16:04:21.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/815828201 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7c80a1720 con 0x5650ea353c00 2023-09-12T16:04:21.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7baff5700 1 -- 172.21.15.130:0/815828201 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7cc0059a0 con 0x5650ea353c00 2023-09-12T16:04:21.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7e91d0b80 10 client.25241 fetch_fsmap learned FSMap version 27 2023-09-12T16:04:21.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/815828201 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5650ea0cf2b0 con 0x5650ea353c00 2023-09-12T16:04:21.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7baff5700 1 -- 172.21.15.130:0/815828201 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c80a1720 con 0x5650ea353c00 2023-09-12T16:04:21.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7baff5700 1 -- 172.21.15.130:0/815828201 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb7cc034c10 con 0x5650ea353c00 2023-09-12T16:04:21.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7e91d0b80 10 client.25241 fetch_fsmap finished waiting for FSMap version 27 2023-09-12T16:04:21.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7e91d0b80 10 client.25241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:21.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7e91d0b80 20 client.25241 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:21.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5650ea3be3a0 0x5650ea3c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:21.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7e91d0b80 4 client.25241 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:04:21.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.932+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/815828201 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5650ea29d920 con 0x5650ea3be3a0 2023-09-12T16:04:21.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5650ea3be3a0 0x5650ea3c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:21.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7baff5700 10 client.25241 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:21.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7d6d9d700 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5650ea3be3a0 0x5650ea3c07c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14eee30a4b3c57a9 server_cookie=95d52de76621b13 in_seq=0 out_seq=0 2023-09-12T16:04:21.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7baff5700 1 -- 172.21.15.130:0/815828201 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5650ea29d920 con 0x5650ea3be3a0 2023-09-12T16:04:21.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7baff5700 10 client.25241 handle_command_reply: tid=0 2023-09-12T16:04:21.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 client.25241 shutdown 2023-09-12T16:04:21.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/815828201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 msgr2=0x5650ea360cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:21.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea360cf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009990 tx=0x7fb7cc009ca0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/815828201 shutdown_connections 2023-09-12T16:04:21.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5650ea3be3a0 0x5650ea3c07c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650ea3545a0 0x5650ea3bc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650ea352e00 0x5650ea3607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7baff5700 0 client.25241 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:21.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 --2- 172.21.15.130:0/815828201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ea353c00 0x5650ea360cf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/815828201 >> 172.21.15.130:0/815828201 conn(0x5650ea2c7090 msgr2=0x5650ea34cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:21.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/815828201 shutdown_connections 2023-09-12T16:04:21.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 1 -- 172.21.15.130:0/815828201 wait complete. 2023-09-12T16:04:21.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.933+0000 7fb7e91d0b80 20 client.25241 trim_cache size 0 max 16384 2023-09-12T16:04:21.950 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:04:21.950 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "no active scrubs running", 2023-09-12T16:04:21.951 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": {} 2023-09-12T16:04:21.951 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:04:21.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 -- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7c00a06c0 msgr2=0x7fb7c00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:21.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7c00a06c0 0x7fb7c00a2b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b2e0 tx=0x7fb7cc005fb0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 -- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d81010d0 msgr2=0x7fb7d8196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:21.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d81010d0 0x7fb7d8196300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800e9a0 tx=0x7fb7c800ed60 comp rx=0 tx=0).stop 2023-09-12T16:04:21.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 -- 172.21.15.130:0/376383854 shutdown_connections 2023-09-12T16:04:21.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb7c00a06c0 0x7fb7c00a2b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7d8106330 0x7fb7d819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d81010d0 0x7fb7d8196300 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 --2- 172.21.15.130:0/376383854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d8103a00 0x7fb7d8196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:21.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 -- 172.21.15.130:0/376383854 >> 172.21.15.130:0/376383854 conn(0x7fb7d80faa60 msgr2=0x7fb7d8101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:21.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 -- 172.21.15.130:0/376383854 shutdown_connections 2023-09-12T16:04:21.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:21.934+0000 7fb7dd00a700 1 -- 172.21.15.130:0/376383854 wait complete. 2023-09-12T16:04:21.955 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-12T16:04:21.955 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:04:21.955 INFO:teuthology.orchestra.run.smithi130.stderr:15800000 of 50000000 tuples (31%) done (elapsed 24.31 s, remaining 52.63 s) 2023-09-12T16:04:22.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.109+0000 7f51af806700 1 -- 172.21.15.130:0/4145375059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51a80ff4f0 msgr2=0x7f51a80ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.109+0000 7f51af806700 1 --2- 172.21.15.130:0/4145375059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51a80ff4f0 0x7f51a80ff910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51a40099c0 tx=0x7f51a4009cd0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.109+0000 7f51af806700 1 -- 172.21.15.130:0/4145375059 shutdown_connections 2023-09-12T16:04:22.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.109+0000 7f51af806700 1 --2- 172.21.15.130:0/4145375059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51a8101080 0x7f51a8107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.109+0000 7f51af806700 1 --2- 172.21.15.130:0/4145375059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51a80ff4f0 0x7f51a80ff910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.109+0000 7f51af806700 1 --2- 172.21.15.130:0/4145375059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a81006e0 0x7f51a8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.109+0000 7f51af806700 1 -- 172.21.15.130:0/4145375059 >> 172.21.15.130:0/4145375059 conn(0x7f51a80faa60 msgr2=0x7f51a80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.110+0000 7f51af806700 1 -- 172.21.15.130:0/4145375059 shutdown_connections 2023-09-12T16:04:22.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.110+0000 7f51af806700 1 -- 172.21.15.130:0/4145375059 wait complete. 2023-09-12T16:04:22.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.110+0000 7f51af806700 1 Processor -- start 2023-09-12T16:04:22.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.110+0000 7f51af806700 1 -- start start 2023-09-12T16:04:22.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51af806700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51a80ff4f0 0x7f51a8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51af806700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51a81006e0 0x7f51a80690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51af806700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a8101080 0x7f51a8070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51af806700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8069790 con 0x7f51a81006e0 2023-09-12T16:04:22.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51af806700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8069900 con 0x7f51a80ff4f0 2023-09-12T16:04:22.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51af806700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80711b0 con 0x7f51a8101080 2023-09-12T16:04:22.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51adda3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a8101080 0x7f51a8070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51adda3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a8101080 0x7f51a8070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35086/0 (socket says 172.21.15.130:35086) 2023-09-12T16:04:22.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51adda3700 1 -- 172.21.15.130:0/1495641182 learned_addr learned my addr 172.21.15.130:0/1495641182 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51ad5a2700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51a80ff4f0 0x7f51a8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51acda1700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51a81006e0 0x7f51a80690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51adda3700 1 -- 172.21.15.130:0/1495641182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51a80ff4f0 msgr2=0x7f51a8068ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51adda3700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51a80ff4f0 0x7f51a8068ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51adda3700 1 -- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51a81006e0 msgr2=0x7f51a80690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51adda3700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51a81006e0 0x7f51a80690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51adda3700 1 -- 172.21.15.130:0/1495641182 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0009c40 con 0x7f51a8101080 2023-09-12T16:04:22.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.111+0000 7f51adda3700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a8101080 0x7f51a8070c40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c370 tx=0x7f51a000c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.112+0000 7f519e7fc700 1 -- 172.21.15.130:0/1495641182 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0017070 con 0x7f51a8101080 2023-09-12T16:04:22.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.112+0000 7f519e7fc700 1 -- 172.21.15.130:0/1495641182 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a000f040 con 0x7f51a8101080 2023-09-12T16:04:22.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.112+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a40095f0 con 0x7f51a8101080 2023-09-12T16:04:22.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.112+0000 7f519e7fc700 1 -- 172.21.15.130:0/1495641182 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a000e050 con 0x7f51a8101080 2023-09-12T16:04:22.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.112+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a80717b0 con 0x7f51a8101080 2023-09-12T16:04:22.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.113+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a8106ae0 con 0x7f51a8101080 2023-09-12T16:04:22.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.114+0000 7f519e7fc700 1 -- 172.21.15.130:0/1495641182 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f51a0004bd0 con 0x7f51a8101080 2023-09-12T16:04:22.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.115+0000 7f519e7fc700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51940a0690 0x7f51940a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.115+0000 7f51ad5a2700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51940a0690 0x7f51940a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.115+0000 7f519e7fc700 1 -- 172.21.15.130:0/1495641182 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f51a00cbfc0 con 0x7f51a8101080 2023-09-12T16:04:22.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.115+0000 7f51ad5a2700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51940a0690 0x7f51940a2b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009990 tx=0x7f51a4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.119+0000 7f519e7fc700 1 -- 172.21.15.130:0/1495641182 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f51a0095780 con 0x7f51a8101080 2023-09-12T16:04:22.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:21 smithi130 ceph-mon[98670]: pgmap v335: 129 pgs: 129 active+clean; 2.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 184 MiB/s wr, 181 op/s 2023-09-12T16:04:22.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:21 smithi159 ceph-mon[102522]: pgmap v335: 129 pgs: 129 active+clean; 2.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 184 MiB/s wr, 181 op/s 2023-09-12T16:04:22.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:21 smithi203 ceph-mon[103431]: pgmap v335: 129 pgs: 129 active+clean; 2.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 184 MiB/s wr, 181 op/s 2023-09-12T16:04:22.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.428+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51a80719d0 con 0x7f51a8101080 2023-09-12T16:04:22.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.429+0000 7f519e7fc700 1 -- 172.21.15.130:0/1495641182 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f51a0098da0 con 0x7f51a8101080 2023-09-12T16:04:22.430 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:04:22.430 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:04:18.080387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:04:22.432 INFO:teuthology.orchestra.run.smithi130.stderr:15900000 of 50000000 tuples (31%) done (elapsed 24.80 s, remaining 53.18 s) 2023-09-12T16:04:22.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51940a0690 msgr2=0x7f51940a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51940a0690 0x7f51940a2b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009990 tx=0x7f51a4005fb0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a8101080 msgr2=0x7f51a8070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a8101080 0x7f51a8070c40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c370 tx=0x7f51a000c680 comp rx=0 tx=0).stop 2023-09-12T16:04:22.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 shutdown_connections 2023-09-12T16:04:22.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f51940a0690 0x7f51940a2b50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a8101080 0x7f51a8070c40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51a80ff4f0 0x7f51a8068ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 --2- 172.21.15.130:0/1495641182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51a81006e0 0x7f51a80690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.432+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 >> 172.21.15.130:0/1495641182 conn(0x7f51a80faa60 msgr2=0x7f51a8103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.433+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 shutdown_connections 2023-09-12T16:04:22.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.433+0000 7f51af806700 1 -- 172.21.15.130:0/1495641182 wait complete. 2023-09-12T16:04:22.436 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 27 2023-09-12T16:04:22.447 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-12T16:04:22.511 INFO:teuthology.orchestra.run.smithi130.stderr:16000000 of 50000000 tuples (32%) done (elapsed 24.88 s, remaining 52.86 s) 2023-09-12T16:04:22.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.588+0000 7fd884ef4700 1 -- 172.21.15.130:0/1206369837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd880100af0 msgr2=0x7fd880100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.588+0000 7fd884ef4700 1 --2- 172.21.15.130:0/1206369837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd880100af0 0x7fd880100f10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd870009990 tx=0x7fd870009ca0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.588+0000 7fd884ef4700 1 -- 172.21.15.130:0/1206369837 shutdown_connections 2023-09-12T16:04:22.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.588+0000 7fd884ef4700 1 --2- 172.21.15.130:0/1206369837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd880100100 0x7fd8801079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.588+0000 7fd884ef4700 1 --2- 172.21.15.130:0/1206369837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd880100af0 0x7fd880100f10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.588+0000 7fd884ef4700 1 --2- 172.21.15.130:0/1206369837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8800ff740 0x7fd8800ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.588+0000 7fd884ef4700 1 -- 172.21.15.130:0/1206369837 >> 172.21.15.130:0/1206369837 conn(0x7fd8800fa1d0 msgr2=0x7fd8800fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.588+0000 7fd884ef4700 1 -- 172.21.15.130:0/1206369837 shutdown_connections 2023-09-12T16:04:22.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.588+0000 7fd884ef4700 1 -- 172.21.15.130:0/1206369837 wait complete. 2023-09-12T16:04:22.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd884ef4700 1 Processor -- start 2023-09-12T16:04:22.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd884ef4700 1 -- start start 2023-09-12T16:04:22.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd884ef4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8800ff740 0x7fd8801952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd884ef4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880100100 0x7fd8801957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd884ef4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd880100af0 0x7fd88019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd884ef4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88019ce30 con 0x7fd8800ff740 2023-09-12T16:04:22.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd884ef4700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88019cf70 con 0x7fd880100100 2023-09-12T16:04:22.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd884ef4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88019d100 con 0x7fd880100af0 2023-09-12T16:04:22.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8800ff740 0x7fd8801952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87dd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880100100 0x7fd8801957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8800ff740 0x7fd8801952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55484/0 (socket says 172.21.15.130:55484) 2023-09-12T16:04:22.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87e59c700 1 -- 172.21.15.130:0/3997635460 learned_addr learned my addr 172.21.15.130:0/3997635460 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87ed9d700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd880100af0 0x7fd88019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87e59c700 1 -- 172.21.15.130:0/3997635460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd880100af0 msgr2=0x7fd88019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87e59c700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd880100af0 0x7fd88019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87e59c700 1 -- 172.21.15.130:0/3997635460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880100100 msgr2=0x7fd8801957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87e59c700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880100100 0x7fd8801957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87e59c700 1 -- 172.21.15.130:0/3997635460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8700095f0 con 0x7fd8800ff740 2023-09-12T16:04:22.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880100100 0x7fd8801957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:22.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87ed9d700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd880100af0 0x7fd88019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:04:22.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.589+0000 7fd87e59c700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8800ff740 0x7fd8801952b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd8700098e0 tx=0x7fd870004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.590+0000 7fd86f7fe700 1 -- 172.21.15.130:0/3997635460 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87001d070 con 0x7fd8800ff740 2023-09-12T16:04:22.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.590+0000 7fd884ef4700 1 -- 172.21.15.130:0/3997635460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88019d380 con 0x7fd8800ff740 2023-09-12T16:04:22.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.590+0000 7fd86f7fe700 1 -- 172.21.15.130:0/3997635460 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8700054f0 con 0x7fd8800ff740 2023-09-12T16:04:22.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.590+0000 7fd86f7fe700 1 -- 172.21.15.130:0/3997635460 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870004ca0 con 0x7fd8800ff740 2023-09-12T16:04:22.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.590+0000 7fd884ef4700 1 -- 172.21.15.130:0/3997635460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88019d7a0 con 0x7fd8800ff740 2023-09-12T16:04:22.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.592+0000 7fd86f7fe700 1 -- 172.21.15.130:0/3997635460 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fd87000fc70 con 0x7fd8800ff740 2023-09-12T16:04:22.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.592+0000 7fd86f7fe700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd8680a06c0 0x7fd8680a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.593+0000 7fd86f7fe700 1 -- 172.21.15.130:0/3997635460 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd8700cd310 con 0x7fd8800ff740 2023-09-12T16:04:22.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.593+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd8680a06c0 0x7fd8680a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.593+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd8680a06c0 0x7fd8680a2b80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd874005950 tx=0x7fd87400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.593+0000 7fd8910bab80 1 Processor -- start 2023-09-12T16:04:22.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd8910bab80 1 -- start start 2023-09-12T16:04:22.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 0x558384a21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a1fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a20490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd8910bab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384976730 con 0x558384a20030 2023-09-12T16:04:22.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd8910bab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384974050 con 0x558384a20e40 2023-09-12T16:04:22.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd8910bab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55838496f940 con 0x558384a217a0 2023-09-12T16:04:22.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87dd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a1fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87ed9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a20490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87dd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a1fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35106/0 (socket says 172.21.15.130:35106) 2023-09-12T16:04:22.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87dd9b700 1 -- 172.21.15.130:0/1840680845 learned_addr learned my addr 172.21.15.130:0/1840680845 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87e59c700 1 --2- 172.21.15.130:0/1840680845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 0x558384a21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87dd9b700 1 -- 172.21.15.130:0/1840680845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 msgr2=0x558384a21260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/1840680845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 0x558384a21260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87dd9b700 1 -- 172.21.15.130:0/1840680845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 msgr2=0x558384a20490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/1840680845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a20490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87dd9b700 1 -- 172.21.15.130:0/1840680845 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55838479bde0 con 0x558384a217a0 2023-09-12T16:04:22.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.594+0000 7fd87ed9d700 1 --2- 172.21.15.130:0/1840680845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a20490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:22.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.595+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/1840680845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a1fa50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd874011790 tx=0x7fd874013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e0a48cd5251151 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.595+0000 7fd86cff9700 1 -- 172.21.15.130:0/1840680845 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87401e620 con 0x558384a217a0 2023-09-12T16:04:22.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.595+0000 7fd86cff9700 1 -- 172.21.15.130:0/1840680845 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd874015070 con 0x558384a217a0 2023-09-12T16:04:22.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.595+0000 7fd86cff9700 1 -- 172.21.15.130:0/1840680845 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87401ca30 con 0x558384a217a0 2023-09-12T16:04:22.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.595+0000 7fd8910bab80 1 -- 172.21.15.130:0/1840680845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 msgr2=0x558384a1fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.595+0000 7fd8910bab80 1 --2- 172.21.15.130:0/1840680845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a1fa50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd874011790 tx=0x7fd874013000 comp rx=0 tx=0).stop 2023-09-12T16:04:22.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 -- 172.21.15.130:0/1840680845 shutdown_connections 2023-09-12T16:04:22.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 --2- 172.21.15.130:0/1840680845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a20490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 --2- 172.21.15.130:0/1840680845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 0x558384a21260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 --2- 172.21.15.130:0/1840680845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a1fa50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 -- 172.21.15.130:0/1840680845 >> 172.21.15.130:0/1840680845 conn(0x558384992090 msgr2=0x558384a185c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 -- 172.21.15.130:0/1840680845 shutdown_connections 2023-09-12T16:04:22.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 -- 172.21.15.130:0/1840680845 wait complete. 2023-09-12T16:04:22.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 -- start start 2023-09-12T16:04:22.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 Processor -- start 2023-09-12T16:04:22.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a29850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 0x558384a29d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a30dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55838490db70 con 0x558384a20030 2023-09-12T16:04:22.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55838490d870 con 0x558384a20e40 2023-09-12T16:04:22.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583847f1920 con 0x558384a217a0 2023-09-12T16:04:22.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd8910bab80 10 client.0 _ll_register_callbacks cb 0x558384857a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:22.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a29850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.596+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a29850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55498/0 (socket says 172.21.15.130:55498) 2023-09-12T16:04:22.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd87e59c700 1 -- 172.21.15.130:0/930376622 learned_addr learned my addr 172.21.15.130:0/930376622 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 0x558384a29d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd87ed9d700 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a30dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd87e59c700 1 -- 172.21.15.130:0/930376622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 msgr2=0x558384a30dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd87e59c700 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a30dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd87e59c700 1 -- 172.21.15.130:0/930376622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 msgr2=0x558384a29d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd87e59c700 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 0x558384a29d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd87e59c700 1 -- 172.21.15.130:0/930376622 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd878009050 con 0x558384a20030 2023-09-12T16:04:22.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd87e59c700 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a29850 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd8700095c0 tx=0x7fd870009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55da49b704d3528f server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd865ffb700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:04:22.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd865ffb700 1 -- 172.21.15.130:0/930376622 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8700a2510 con 0x558384a20030 2023-09-12T16:04:22.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd865ffb700 1 -- 172.21.15.130:0/930376622 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd87009aec0 con 0x558384a20030 2023-09-12T16:04:22.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd8910bab80 1 -- 172.21.15.130:0/930376622 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583848c74f0 con 0x558384a20030 2023-09-12T16:04:22.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd865ffb700 1 -- 172.21.15.130:0/930376622 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8700b7030 con 0x558384a20030 2023-09-12T16:04:22.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd8910bab80 10 client.15795 fetch_fsmap learned FSMap version 27 2023-09-12T16:04:22.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd8910bab80 1 -- 172.21.15.130:0/930376622 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fd874011440 con 0x558384a20030 2023-09-12T16:04:22.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.597+0000 7fd865ffb700 1 -- 172.21.15.130:0/930376622 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583848c74f0 con 0x558384a20030 2023-09-12T16:04:22.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd865ffb700 1 -- 172.21.15.130:0/930376622 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd87009e120 con 0x558384a20030 2023-09-12T16:04:22.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd8910bab80 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2023-09-12T16:04:22.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd8910bab80 10 client.15795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-09-12T16:04:22.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd8910bab80 20 client.15795 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:22.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd8910bab80 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x558384a35210 0x558384a37630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd8910bab80 4 client.15795 mds_command: new command op to 24574 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:04:22.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd8910bab80 1 -- 172.21.15.130:0/930376622 --> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583849848a0 con 0x558384a35210 2023-09-12T16:04:22.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x558384a35210 0x558384a37630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x558384a35210 0x558384a37630 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cb1ce79a80f30e3 server_cookie=3e82619ec9a5587 in_seq=0 out_seq=0 2023-09-12T16:04:22.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.598+0000 7fd865ffb700 10 client.15795 ms_handle_connect on v2:172.21.15.203:6836/3950241499 2023-09-12T16:04:22.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd865ffb700 1 -- 172.21.15.130:0/930376622 <== mds.2 v2:172.21.15.203:6836/3950241499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5583849848a0 con 0x558384a35210 2023-09-12T16:04:22.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd865ffb700 10 client.15795 handle_command_reply: tid=0 2023-09-12T16:04:22.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 client.15795 shutdown 2023-09-12T16:04:22.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 -- 172.21.15.130:0/930376622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 msgr2=0x558384a29850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a29850 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd8700095c0 tx=0x7fd870009ca0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 -- 172.21.15.130:0/930376622 shutdown_connections 2023-09-12T16:04:22.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x558384a35210 0x558384a37630 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a217a0 0x558384a30dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd865ffb700 0 client.15795 ms_handle_reset on v2:172.21.15.203:6836/3950241499 2023-09-12T16:04:22.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20030 0x558384a29850 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 --2- 172.21.15.130:0/930376622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a20e40 0x558384a29d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 -- 172.21.15.130:0/930376622 >> 172.21.15.130:0/930376622 conn(0x558384992090 msgr2=0x558384a1e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 -- 172.21.15.130:0/930376622 shutdown_connections 2023-09-12T16:04:22.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 1 -- 172.21.15.130:0/930376622 wait complete. 2023-09-12T16:04:22.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.599+0000 7fd8910bab80 20 client.15795 trim_cache size 0 max 16384 2023-09-12T16:04:22.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.616+0000 7fd8910bab80 1 Processor -- start 2023-09-12T16:04:22.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.616+0000 7fd8910bab80 1 -- start start 2023-09-12T16:04:22.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20e40 0x558384a284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a217a0 0x558384a20410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a17320 0x558384a316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd8910bab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384a29420 con 0x558384a20e40 2023-09-12T16:04:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd8910bab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384a19510 con 0x558384a217a0 2023-09-12T16:04:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd8910bab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384a1a210 con 0x558384a17320 2023-09-12T16:04:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20e40 0x558384a284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20e40 0x558384a284c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55506/0 (socket says 172.21.15.130:55506) 2023-09-12T16:04:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87e59c700 1 -- 172.21.15.130:0/1445220886 learned_addr learned my addr 172.21.15.130:0/1445220886 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87ed9d700 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a17320 0x558384a316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a217a0 0x558384a20410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87dd9b700 1 -- 172.21.15.130:0/1445220886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a17320 msgr2=0x558384a316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a17320 0x558384a316c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87dd9b700 1 -- 172.21.15.130:0/1445220886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20e40 msgr2=0x558384a284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20e40 0x558384a284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87dd9b700 1 -- 172.21.15.130:0/1445220886 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558384784c60 con 0x558384a217a0 2023-09-12T16:04:22.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87e59c700 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20e40 0x558384a284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:22.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.617+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a217a0 0x558384a20410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd874011d80 tx=0x7fd874010ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e6e7c11956dd68b server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd86d7fa700 1 -- 172.21.15.130:0/1445220886 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87402d070 con 0x558384a217a0 2023-09-12T16:04:22.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd86d7fa700 1 -- 172.21.15.130:0/1445220886 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd87402e040 con 0x558384a217a0 2023-09-12T16:04:22.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd86d7fa700 1 -- 172.21.15.130:0/1445220886 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8740053c0 con 0x558384a217a0 2023-09-12T16:04:22.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd8910bab80 1 -- 172.21.15.130:0/1445220886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a217a0 msgr2=0x558384a20410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd8910bab80 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a217a0 0x558384a20410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd874011d80 tx=0x7fd874010ea0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd8910bab80 1 -- 172.21.15.130:0/1445220886 shutdown_connections 2023-09-12T16:04:22.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd8910bab80 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a17320 0x558384a316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd8910bab80 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a20e40 0x558384a284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd8910bab80 1 --2- 172.21.15.130:0/1445220886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a217a0 0x558384a20410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd8910bab80 1 -- 172.21.15.130:0/1445220886 >> 172.21.15.130:0/1445220886 conn(0x558384992090 msgr2=0x558384a3bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd8910bab80 1 -- 172.21.15.130:0/1445220886 shutdown_connections 2023-09-12T16:04:22.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.618+0000 7fd8910bab80 1 -- 172.21.15.130:0/1445220886 wait complete. 2023-09-12T16:04:22.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd8910bab80 1 -- start start 2023-09-12T16:04:22.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd8910bab80 1 Processor -- start 2023-09-12T16:04:22.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a17320 0x558384a83250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a20e40 0x558384a83790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd8910bab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a217a0 0x558384a8a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd8910bab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384a8a9f0 con 0x558384a217a0 2023-09-12T16:04:22.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd8910bab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384a8ab30 con 0x558384a17320 2023-09-12T16:04:22.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd8910bab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384a8acc0 con 0x558384a20e40 2023-09-12T16:04:22.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd8910bab80 10 client.0 _ll_register_callbacks cb 0x7fd87009a3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:22.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a17320 0x558384a83250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a17320 0x558384a83250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:38770/0 (socket says 172.21.15.130:38770) 2023-09-12T16:04:22.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87e59c700 1 -- 172.21.15.130:0/2762277805 learned_addr learned my addr 172.21.15.130:0/2762277805 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87ed9d700 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a217a0 0x558384a8a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a20e40 0x558384a83790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87e59c700 1 -- 172.21.15.130:0/2762277805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a20e40 msgr2=0x558384a83790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87e59c700 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a20e40 0x558384a83790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87e59c700 1 -- 172.21.15.130:0/2762277805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a217a0 msgr2=0x558384a8a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87e59c700 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a217a0 0x558384a8a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87e59c700 1 -- 172.21.15.130:0/2762277805 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd874011440 con 0x558384a17320 2023-09-12T16:04:22.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.619+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a20e40 0x558384a83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:22.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd87e59c700 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a17320 0x558384a83250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd870005f30 tx=0x7fd87009a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e697080770c7e755 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd865ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:04:22.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd865ffb700 1 -- 172.21.15.130:0/2762277805 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8700b7070 con 0x558384a17320 2023-09-12T16:04:22.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd865ffb700 1 -- 172.21.15.130:0/2762277805 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd8700b8070 con 0x558384a17320 2023-09-12T16:04:22.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd8910bab80 1 -- 172.21.15.130:0/2762277805 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd874015070 con 0x558384a17320 2023-09-12T16:04:22.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd865ffb700 1 -- 172.21.15.130:0/2762277805 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8700ca980 con 0x558384a17320 2023-09-12T16:04:22.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd8910bab80 10 client.25265 fetch_fsmap learned FSMap version 27 2023-09-12T16:04:22.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd8910bab80 1 -- 172.21.15.130:0/2762277805 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55838479bde0 con 0x558384a17320 2023-09-12T16:04:22.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd865ffb700 1 -- 172.21.15.130:0/2762277805 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd874015070 con 0x558384a17320 2023-09-12T16:04:22.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd865ffb700 1 -- 172.21.15.130:0/2762277805 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd870066280 con 0x558384a17320 2023-09-12T16:04:22.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd8910bab80 10 client.25265 fetch_fsmap finished waiting for FSMap version 27 2023-09-12T16:04:22.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd8910bab80 10 client.25265 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-09-12T16:04:22.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd8910bab80 20 client.25265 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:22.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd8910bab80 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x558384a8c2f0 0x558384a8e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd8910bab80 4 client.25265 mds_command: new command op to 24574 tid=0[{"prefix": "damage ls"}] 2023-09-12T16:04:22.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.620+0000 7fd8910bab80 1 -- 172.21.15.130:0/2762277805 --> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558384968920 con 0x558384a8c2f0 2023-09-12T16:04:22.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x558384a8c2f0 0x558384a8e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd865ffb700 10 client.25265 ms_handle_connect on v2:172.21.15.203:6836/3950241499 2023-09-12T16:04:22.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd87dd9b700 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x558384a8c2f0 0x558384a8e710 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d5cb7bdeed4bd49 server_cookie=8eb7bca146a7131 in_seq=0 out_seq=0 2023-09-12T16:04:22.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd865ffb700 1 -- 172.21.15.130:0/2762277805 <== mds.2 v2:172.21.15.203:6836/3950241499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558384968920 con 0x558384a8c2f0 2023-09-12T16:04:22.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd865ffb700 10 client.25265 handle_command_reply: tid=0 2023-09-12T16:04:22.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd8910bab80 1 client.25265 shutdown 2023-09-12T16:04:22.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd8910bab80 1 -- 172.21.15.130:0/2762277805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a17320 msgr2=0x558384a83250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd8910bab80 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a17320 0x558384a83250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd870005f30 tx=0x7fd87009a590 comp rx=0 tx=0).stop 2023-09-12T16:04:22.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd8910bab80 1 -- 172.21.15.130:0/2762277805 shutdown_connections 2023-09-12T16:04:22.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.621+0000 7fd8910bab80 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.203:6836/3950241499,v1:172.21.15.203:6837/3950241499] conn(0x558384a8c2f0 0x558384a8e710 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd8910bab80 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558384a217a0 0x558384a8a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd8910bab80 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558384a17320 0x558384a83250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd865ffb700 0 client.25265 ms_handle_reset on v2:172.21.15.203:6836/3950241499 2023-09-12T16:04:22.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd8910bab80 1 --2- 172.21.15.130:0/2762277805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558384a20e40 0x558384a83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd8910bab80 1 -- 172.21.15.130:0/2762277805 >> 172.21.15.130:0/2762277805 conn(0x558384992090 msgr2=0x558384a20410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd8910bab80 1 -- 172.21.15.130:0/2762277805 shutdown_connections 2023-09-12T16:04:22.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd8910bab80 1 -- 172.21.15.130:0/2762277805 wait complete. 2023-09-12T16:04:22.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd8910bab80 20 client.25265 trim_cache size 0 max 16384 2023-09-12T16:04:22.642 INFO:teuthology.orchestra.run.smithi130.stdout:[] 2023-09-12T16:04:22.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 -- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd8680a06c0 msgr2=0x7fd8680a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd8680a06c0 0x7fd8680a2b80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd874005950 tx=0x7fd87400b410 comp rx=0 tx=0).stop 2023-09-12T16:04:22.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 -- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8800ff740 msgr2=0x7fd8801952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8800ff740 0x7fd8801952b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd8700098e0 tx=0x7fd870004730 comp rx=0 tx=0).stop 2023-09-12T16:04:22.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 -- 172.21.15.130:0/3997635460 shutdown_connections 2023-09-12T16:04:22.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd8680a06c0 0x7fd8680a2b80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd880100af0 0x7fd88019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8800ff740 0x7fd8801952b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 --2- 172.21.15.130:0/3997635460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880100100 0x7fd8801957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 -- 172.21.15.130:0/3997635460 >> 172.21.15.130:0/3997635460 conn(0x7fd8800fa1d0 msgr2=0x7fd880103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 -- 172.21.15.130:0/3997635460 shutdown_connections 2023-09-12T16:04:22.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.622+0000 7fd884ef4700 1 -- 172.21.15.130:0/3997635460 wait complete. 2023-09-12T16:04:22.646 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-12T16:04:22.655 INFO:teuthology.orchestra.run.smithi130.stderr:16100000 of 50000000 tuples (32%) done (elapsed 25.02 s, remaining 52.68 s) 2023-09-12T16:04:22.723 INFO:teuthology.orchestra.run.smithi130.stderr:16200000 of 50000000 tuples (32%) done (elapsed 25.09 s, remaining 52.35 s) 2023-09-12T16:04:22.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.779+0000 7f59a453f700 1 -- 172.21.15.130:0/2466734020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c0ff4f0 msgr2=0x7f599c0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.779+0000 7f59a453f700 1 --2- 172.21.15.130:0/2466734020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c0ff4f0 0x7f599c0ff910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f59980099c0 tx=0x7f5998009cd0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.779+0000 7f59a453f700 1 -- 172.21.15.130:0/2466734020 shutdown_connections 2023-09-12T16:04:22.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.779+0000 7f59a453f700 1 --2- 172.21.15.130:0/2466734020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f599c101080 0x7f599c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.779+0000 7f59a453f700 1 --2- 172.21.15.130:0/2466734020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c0ff4f0 0x7f599c0ff910 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.779+0000 7f59a453f700 1 --2- 172.21.15.130:0/2466734020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f599c1006e0 0x7f599c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.779+0000 7f59a453f700 1 -- 172.21.15.130:0/2466734020 >> 172.21.15.130:0/2466734020 conn(0x7f599c0faa60 msgr2=0x7f599c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.779+0000 7f59a453f700 1 -- 172.21.15.130:0/2466734020 shutdown_connections 2023-09-12T16:04:22.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.779+0000 7f59a453f700 1 -- 172.21.15.130:0/2466734020 wait complete. 2023-09-12T16:04:22.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a453f700 1 Processor -- start 2023-09-12T16:04:22.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a453f700 1 -- start start 2023-09-12T16:04:22.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a453f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f599c0ff4f0 0x7f599c195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a453f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c1006e0 0x7f599c195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a453f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f599c101080 0x7f599c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a453f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c19cd80 con 0x7f599c1006e0 2023-09-12T16:04:22.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a453f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c19cec0 con 0x7f599c101080 2023-09-12T16:04:22.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a453f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c19d050 con 0x7f599c0ff4f0 2023-09-12T16:04:22.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a1ada700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c1006e0 0x7f599c195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a1ada700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c1006e0 0x7f599c195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55526/0 (socket says 172.21.15.130:55526) 2023-09-12T16:04:22.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a1ada700 1 -- 172.21.15.130:0/3473489753 learned_addr learned my addr 172.21.15.130:0/3473489753 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a22db700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f599c0ff4f0 0x7f599c195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.780+0000 7f59a2adc700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f599c101080 0x7f599c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59a1ada700 1 -- 172.21.15.130:0/3473489753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f599c0ff4f0 msgr2=0x7f599c195200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59a1ada700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f599c0ff4f0 0x7f599c195200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59a1ada700 1 -- 172.21.15.130:0/3473489753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f599c101080 msgr2=0x7f599c19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59a1ada700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f599c101080 0x7f599c19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59a1ada700 1 -- 172.21.15.130:0/3473489753 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59980095f0 con 0x7f599c1006e0 2023-09-12T16:04:22.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59a2adc700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f599c101080 0x7f599c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:04:22.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59a1ada700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c1006e0 0x7f599c195740 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f598c00e9a0 tx=0x7f598c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59937fe700 1 -- 172.21.15.130:0/3473489753 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c009790 con 0x7f599c1006e0 2023-09-12T16:04:22.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59937fe700 1 -- 172.21.15.130:0/3473489753 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f598c00cbd0 con 0x7f599c1006e0 2023-09-12T16:04:22.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59937fe700 1 -- 172.21.15.130:0/3473489753 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c018680 con 0x7f599c1006e0 2023-09-12T16:04:22.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59a453f700 1 -- 172.21.15.130:0/3473489753 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c19d330 con 0x7f599c1006e0 2023-09-12T16:04:22.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.781+0000 7f59a453f700 1 -- 172.21.15.130:0/3473489753 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c19d7f0 con 0x7f599c1006e0 2023-09-12T16:04:22.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.783+0000 7f59937fe700 1 -- 172.21.15.130:0/3473489753 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f598c00cd40 con 0x7f599c1006e0 2023-09-12T16:04:22.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.784+0000 7f59937fe700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59880a0640 0x7f59880a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.784+0000 7f59a22db700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59880a0640 0x7f59880a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.784+0000 7f59937fe700 1 -- 172.21.15.130:0/3473489753 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f598c014070 con 0x7f599c1006e0 2023-09-12T16:04:22.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.784+0000 7f59a22db700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59880a0640 0x7f59880a2b00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5998009990 tx=0x7f5998005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.785+0000 7f59b0705b80 1 Processor -- start 2023-09-12T16:04:22.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.785+0000 7f59b0705b80 1 -- start start 2023-09-12T16:04:22.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.785+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec1a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.785+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 0x556f8ec18800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.785+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec18e00 0x556f8ec192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.785+0000 7f59b0705b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8eb71730 con 0x556f8ec19c00 2023-09-12T16:04:22.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.785+0000 7f59b0705b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8eb6f050 con 0x556f8ec1a5a0 2023-09-12T16:04:22.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.785+0000 7f59b0705b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8eb6a940 con 0x556f8ec18e00 2023-09-12T16:04:22.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.785+0000 7f59a22db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec1a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a22db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec1a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55532/0 (socket says 172.21.15.130:55532) 2023-09-12T16:04:22.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a22db700 1 -- 172.21.15.130:0/2449421946 learned_addr learned my addr 172.21.15.130:0/2449421946 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a2adc700 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec18e00 0x556f8ec192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a1ada700 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 0x556f8ec18800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a2adc700 1 -- 172.21.15.130:0/2449421946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 msgr2=0x556f8ec18800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a2adc700 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 0x556f8ec18800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a2adc700 1 -- 172.21.15.130:0/2449421946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 msgr2=0x556f8ec1a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a2adc700 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec1a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a2adc700 1 -- 172.21.15.130:0/2449421946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f8e996de0 con 0x556f8ec18e00 2023-09-12T16:04:22.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a22db700 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec1a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:22.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f59a2adc700 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec18e00 0x556f8ec192b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f59940093a0 tx=0x7f5994009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=403db36024a1423e server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f5990ff9700 1 -- 172.21.15.130:0/2449421946 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599400e070 con 0x556f8ec18e00 2023-09-12T16:04:22.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f5990ff9700 1 -- 172.21.15.130:0/2449421946 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f599400f040 con 0x556f8ec18e00 2023-09-12T16:04:22.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.786+0000 7f5990ff9700 1 -- 172.21.15.130:0/2449421946 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994015e60 con 0x556f8ec18e00 2023-09-12T16:04:22.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.787+0000 7f59b0705b80 1 -- 172.21.15.130:0/2449421946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec18e00 msgr2=0x556f8ec192b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.787+0000 7f59b0705b80 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec18e00 0x556f8ec192b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f59940093a0 tx=0x7f5994009920 comp rx=0 tx=0).stop 2023-09-12T16:04:22.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.787+0000 7f59b0705b80 1 -- 172.21.15.130:0/2449421946 shutdown_connections 2023-09-12T16:04:22.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.787+0000 7f59b0705b80 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec18e00 0x556f8ec192b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.787+0000 7f59b0705b80 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec1a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.787+0000 7f59b0705b80 1 --2- 172.21.15.130:0/2449421946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 0x556f8ec18800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.787+0000 7f59b0705b80 1 -- 172.21.15.130:0/2449421946 >> 172.21.15.130:0/2449421946 conn(0x556f8eb8d090 msgr2=0x556f8ec13720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.787+0000 7f59b0705b80 1 -- 172.21.15.130:0/2449421946 shutdown_connections 2023-09-12T16:04:22.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.787+0000 7f59b0705b80 1 -- 172.21.15.130:0/2449421946 wait complete. 2023-09-12T16:04:22.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59b0705b80 1 -- start start 2023-09-12T16:04:22.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59b0705b80 1 Processor -- start 2023-09-12T16:04:22.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec22550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec19c00 0x556f8ec22a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec29ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59b0705b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8eb08b70 con 0x556f8ec18e00 2023-09-12T16:04:22.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59b0705b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8eb08870 con 0x556f8ec19c00 2023-09-12T16:04:22.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59b0705b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8e9ec920 con 0x556f8ec1a5a0 2023-09-12T16:04:22.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59b0705b80 10 client.0 _ll_register_callbacks cb 0x556f8ea52a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:22.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59a22db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec22550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59a22db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec22550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55538/0 (socket says 172.21.15.130:55538) 2023-09-12T16:04:22.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59a1ada700 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec19c00 0x556f8ec22a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59a22db700 1 -- 172.21.15.130:0/831627507 learned_addr learned my addr 172.21.15.130:0/831627507 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.788+0000 7f59a2adc700 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec29ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59a1ada700 1 -- 172.21.15.130:0/831627507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 msgr2=0x556f8ec29ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59a1ada700 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec29ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59a1ada700 1 -- 172.21.15.130:0/831627507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 msgr2=0x556f8ec22550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59a1ada700 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec22550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59a1ada700 1 -- 172.21.15.130:0/831627507 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c108280 con 0x556f8ec19c00 2023-09-12T16:04:22.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59a2adc700 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec29ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:22.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59a22db700 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec22550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:22.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59a1ada700 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec19c00 0x556f8ec22a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f598c099350 tx=0x7f598c099b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74eabe596b44928b server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f5985ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:04:22.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f5985ffb700 1 -- 172.21.15.130:0/831627507 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c082ea0 con 0x556f8ec19c00 2023-09-12T16:04:22.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f5985ffb700 1 -- 172.21.15.130:0/831627507 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f598c099d50 con 0x556f8ec19c00 2023-09-12T16:04:22.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59b0705b80 1 -- 172.21.15.130:0/831627507 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f8eac24f0 con 0x556f8ec19c00 2023-09-12T16:04:22.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f5985ffb700 1 -- 172.21.15.130:0/831627507 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f598c099650 con 0x556f8ec19c00 2023-09-12T16:04:22.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59b0705b80 10 client.25277 fetch_fsmap learned FSMap version 27 2023-09-12T16:04:22.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f59b0705b80 1 -- 172.21.15.130:0/831627507 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f5994009050 con 0x556f8ec19c00 2023-09-12T16:04:22.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.789+0000 7f5985ffb700 1 -- 172.21.15.130:0/831627507 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f8eac24f0 con 0x556f8ec19c00 2023-09-12T16:04:22.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f5985ffb700 1 -- 172.21.15.130:0/831627507 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f598c0840c0 con 0x556f8ec19c00 2023-09-12T16:04:22.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f59b0705b80 10 client.25277 fetch_fsmap finished waiting for FSMap version 27 2023-09-12T16:04:22.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f59b0705b80 10 client.25277 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-12T16:04:22.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f59b0705b80 20 client.25277 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:22.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f59b0705b80 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556f8ec2e300 0x556f8ec30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f59b0705b80 4 client.25277 mds_command: new command op to 24592 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:04:22.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f59b0705b80 1 -- 172.21.15.130:0/831627507 --> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f8eb7f8a0 con 0x556f8ec2e300 2023-09-12T16:04:22.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f59a22db700 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556f8ec2e300 0x556f8ec30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f59a22db700 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556f8ec2e300 0x556f8ec30720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2186c793a94ef9e server_cookie=5218c9bcea676271 in_seq=0 out_seq=0 2023-09-12T16:04:22.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.790+0000 7f5985ffb700 10 client.25277 ms_handle_connect on v2:172.21.15.203:6838/2671073294 2023-09-12T16:04:22.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f5985ffb700 1 -- 172.21.15.130:0/831627507 <== mds.1 v2:172.21.15.203:6838/2671073294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f8eb7f8a0 con 0x556f8ec2e300 2023-09-12T16:04:22.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f5985ffb700 10 client.25277 handle_command_reply: tid=0 2023-09-12T16:04:22.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 client.25277 shutdown 2023-09-12T16:04:22.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 -- 172.21.15.130:0/831627507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec19c00 msgr2=0x556f8ec22a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec19c00 0x556f8ec22a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f598c099350 tx=0x7f598c099b90 comp rx=0 tx=0).stop 2023-09-12T16:04:22.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 -- 172.21.15.130:0/831627507 shutdown_connections 2023-09-12T16:04:22.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec29ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec22550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556f8ec2e300 0x556f8ec30720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 --2- 172.21.15.130:0/831627507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec19c00 0x556f8ec22a90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f5985ffb700 0 client.25277 ms_handle_reset on v2:172.21.15.203:6838/2671073294 2023-09-12T16:04:22.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 -- 172.21.15.130:0/831627507 >> 172.21.15.130:0/831627507 conn(0x556f8eb8d090 msgr2=0x556f8ec12dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 -- 172.21.15.130:0/831627507 shutdown_connections 2023-09-12T16:04:22.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 1 -- 172.21.15.130:0/831627507 wait complete. 2023-09-12T16:04:22.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.791+0000 7f59b0705b80 20 client.25277 trim_cache size 0 max 16384 2023-09-12T16:04:22.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59b0705b80 1 Processor -- start 2023-09-12T16:04:22.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59b0705b80 1 -- start start 2023-09-12T16:04:22.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec18e00 0x556f8ec18710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec29a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec13690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59b0705b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8ec2b800 con 0x556f8ec19c00 2023-09-12T16:04:22.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59b0705b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8ebdd880 con 0x556f8ec18e00 2023-09-12T16:04:22.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59b0705b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8ec20190 con 0x556f8ec1a5a0 2023-09-12T16:04:22.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a1ada700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec29a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a1ada700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec29a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55548/0 (socket says 172.21.15.130:55548) 2023-09-12T16:04:22.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a1ada700 1 -- 172.21.15.130:0/1850651175 learned_addr learned my addr 172.21.15.130:0/1850651175 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a2adc700 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec13690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a22db700 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec18e00 0x556f8ec18710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a22db700 1 -- 172.21.15.130:0/1850651175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 msgr2=0x556f8ec13690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a22db700 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec13690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a22db700 1 -- 172.21.15.130:0/1850651175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 msgr2=0x556f8ec29a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a22db700 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec29a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a22db700 1 -- 172.21.15.130:0/1850651175 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f8e97fc60 con 0x556f8ec18e00 2023-09-12T16:04:22.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a2adc700 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:22.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.810+0000 7f59a1ada700 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec29a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:22.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59a22db700 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec18e00 0x556f8ec18710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59980099c0 tx=0x7f5998021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead4f0fa58e19aed server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59917fa700 1 -- 172.21.15.130:0/1850651175 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998029040 con 0x556f8ec18e00 2023-09-12T16:04:22.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59917fa700 1 -- 172.21.15.130:0/1850651175 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f599802a040 con 0x556f8ec18e00 2023-09-12T16:04:22.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59917fa700 1 -- 172.21.15.130:0/1850651175 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599800f6c0 con 0x556f8ec18e00 2023-09-12T16:04:22.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59b0705b80 1 -- 172.21.15.130:0/1850651175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec18e00 msgr2=0x556f8ec18710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59b0705b80 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec18e00 0x556f8ec18710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59980099c0 tx=0x7f5998021000 comp rx=0 tx=0).stop 2023-09-12T16:04:22.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59b0705b80 1 -- 172.21.15.130:0/1850651175 shutdown_connections 2023-09-12T16:04:22.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59b0705b80 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec1a5a0 0x556f8ec13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59b0705b80 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec18e00 0x556f8ec18710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59b0705b80 1 --2- 172.21.15.130:0/1850651175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec19c00 0x556f8ec29a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.811+0000 7f59b0705b80 1 -- 172.21.15.130:0/1850651175 >> 172.21.15.130:0/1850651175 conn(0x556f8eb8d090 msgr2=0x556f8ec35080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 -- 172.21.15.130:0/1850651175 shutdown_connections 2023-09-12T16:04:22.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 -- 172.21.15.130:0/1850651175 wait complete. 2023-09-12T16:04:22.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 -- start start 2023-09-12T16:04:22.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 Processor -- start 2023-09-12T16:04:22.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec25be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec19c00 0x556f8ec26120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 0x556f8ec29140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8ec29680 con 0x556f8ec18e00 2023-09-12T16:04:22.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8ec297f0 con 0x556f8ec1a5a0 2023-09-12T16:04:22.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f8ec299b0 con 0x556f8ec19c00 2023-09-12T16:04:22.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59b0705b80 10 client.0 _ll_register_callbacks cb 0x7f598c099190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:22.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59a22db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec25be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59a1ada700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec19c00 0x556f8ec26120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59a22db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec25be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55554/0 (socket says 172.21.15.130:55554) 2023-09-12T16:04:22.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59a22db700 1 -- 172.21.15.130:0/3914561808 learned_addr learned my addr 172.21.15.130:0/3914561808 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:22.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59a2adc700 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 0x556f8ec29140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59a1ada700 1 -- 172.21.15.130:0/3914561808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 msgr2=0x556f8ec29140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59a1ada700 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 0x556f8ec29140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59a1ada700 1 -- 172.21.15.130:0/3914561808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 msgr2=0x556f8ec25be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:22.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.812+0000 7f59a1ada700 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec25be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59a1ada700 1 -- 172.21.15.130:0/3914561808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5994009050 con 0x556f8ec19c00 2023-09-12T16:04:22.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59a1ada700 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec19c00 0x556f8ec26120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f598c0995e0 tx=0x7f598c0985b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b228eca870f4385 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:22.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f5985ffb700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:04:22.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f5985ffb700 1 -- 172.21.15.130:0/3914561808 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c082ea0 con 0x556f8ec19c00 2023-09-12T16:04:22.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f5985ffb700 1 -- 172.21.15.130:0/3914561808 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f598c099650 con 0x556f8ec19c00 2023-09-12T16:04:22.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59b0705b80 1 -- 172.21.15.130:0/3914561808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f599400f040 con 0x556f8ec19c00 2023-09-12T16:04:22.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f5985ffb700 1 -- 172.21.15.130:0/3914561808 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f598c09d480 con 0x556f8ec19c00 2023-09-12T16:04:22.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59b0705b80 10 client.25480 fetch_fsmap learned FSMap version 27 2023-09-12T16:04:22.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59b0705b80 1 -- 172.21.15.130:0/3914561808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556f8e996de0 con 0x556f8ec19c00 2023-09-12T16:04:22.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f5985ffb700 1 -- 172.21.15.130:0/3914561808 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f599400f040 con 0x556f8ec19c00 2023-09-12T16:04:22.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f5985ffb700 1 -- 172.21.15.130:0/3914561808 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f598c09e3c0 con 0x556f8ec19c00 2023-09-12T16:04:22.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59b0705b80 10 client.25480 fetch_fsmap finished waiting for FSMap version 27 2023-09-12T16:04:22.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59b0705b80 10 client.25480 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-12T16:04:22.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59b0705b80 20 client.25480 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:22.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59b0705b80 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556f8ec83140 0x556f8ec85560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:22.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59b0705b80 4 client.25480 mds_command: new command op to 24592 tid=0[{"prefix": "damage ls"}] 2023-09-12T16:04:22.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.813+0000 7f59b0705b80 1 -- 172.21.15.130:0/3914561808 --> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556f8eb63920 con 0x556f8ec83140 2023-09-12T16:04:22.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59a22db700 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556f8ec83140 0x556f8ec85560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:22.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f5985ffb700 10 client.25480 ms_handle_connect on v2:172.21.15.203:6838/2671073294 2023-09-12T16:04:22.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59a22db700 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556f8ec83140 0x556f8ec85560 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65017011eec54b0e server_cookie=1420bc8311eddd2a in_seq=0 out_seq=0 2023-09-12T16:04:22.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f5985ffb700 1 -- 172.21.15.130:0/3914561808 <== mds.1 v2:172.21.15.203:6838/2671073294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556f8eb63920 con 0x556f8ec83140 2023-09-12T16:04:22.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f5985ffb700 10 client.25480 handle_command_reply: tid=0 2023-09-12T16:04:22.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 client.25480 shutdown 2023-09-12T16:04:22.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 -- 172.21.15.130:0/3914561808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec19c00 msgr2=0x556f8ec26120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec19c00 0x556f8ec26120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f598c0995e0 tx=0x7f598c0985b0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 -- 172.21.15.130:0/3914561808 shutdown_connections 2023-09-12T16:04:22.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.203:6838/2671073294,v1:172.21.15.203:6839/2671073294] conn(0x556f8ec83140 0x556f8ec85560 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f8ec1a5a0 0x556f8ec29140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f8ec18e00 0x556f8ec25be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 --2- 172.21.15.130:0/3914561808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f8ec19c00 0x556f8ec26120 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f5985ffb700 0 client.25480 ms_handle_reset on v2:172.21.15.203:6838/2671073294 2023-09-12T16:04:22.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 -- 172.21.15.130:0/3914561808 >> 172.21.15.130:0/3914561808 conn(0x556f8eb8d090 msgr2=0x556f8ec203b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 -- 172.21.15.130:0/3914561808 shutdown_connections 2023-09-12T16:04:22.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 1 -- 172.21.15.130:0/3914561808 wait complete. 2023-09-12T16:04:22.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.814+0000 7f59b0705b80 20 client.25480 trim_cache size 0 max 16384 2023-09-12T16:04:22.833 INFO:teuthology.orchestra.run.smithi130.stdout:[] 2023-09-12T16:04:22.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 -- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59880a0640 msgr2=0x7f59880a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59880a0640 0x7f59880a2b00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5998009990 tx=0x7f5998005fb0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 -- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c1006e0 msgr2=0x7f599c195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:22.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c1006e0 0x7f599c195740 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f598c00e9a0 tx=0x7f598c00ecb0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 -- 172.21.15.130:0/3473489753 shutdown_connections 2023-09-12T16:04:22.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f59880a0640 0x7f59880a2b00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f599c101080 0x7f599c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f599c0ff4f0 0x7f599c195200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 --2- 172.21.15.130:0/3473489753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f599c1006e0 0x7f599c195740 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:22.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 -- 172.21.15.130:0/3473489753 >> 172.21.15.130:0/3473489753 conn(0x7f599c0faa60 msgr2=0x7f599c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:22.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 -- 172.21.15.130:0/3473489753 shutdown_connections 2023-09-12T16:04:22.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:22.815+0000 7f59a453f700 1 -- 172.21.15.130:0/3473489753 wait complete. 2023-09-12T16:04:22.837 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-12T16:04:22.867 INFO:teuthology.orchestra.run.smithi130.stderr:16300000 of 50000000 tuples (32%) done (elapsed 25.23 s, remaining 52.17 s) 2023-09-12T16:04:22.986 INFO:teuthology.orchestra.run.smithi130.stderr:16400000 of 50000000 tuples (32%) done (elapsed 25.35 s, remaining 51.94 s) 2023-09-12T16:04:23.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.005+0000 7f4b8f611700 1 -- 172.21.15.130:0/2860714091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b88100720 msgr2=0x7f4b88100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:23.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.005+0000 7f4b8f611700 1 --2- 172.21.15.130:0/2860714091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b88100720 0x7f4b88100b80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4b780099c0 tx=0x7f4b78009cd0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.005+0000 7f4b8f611700 1 -- 172.21.15.130:0/2860714091 shutdown_connections 2023-09-12T16:04:23.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.005+0000 7f4b8f611700 1 --2- 172.21.15.130:0/2860714091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b881010c0 0x7f4b88107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.005+0000 7f4b8f611700 1 --2- 172.21.15.130:0/2860714091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b880ff530 0x7f4b880ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.005+0000 7f4b8f611700 1 --2- 172.21.15.130:0/2860714091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b88100720 0x7f4b88100b80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.005+0000 7f4b8f611700 1 -- 172.21.15.130:0/2860714091 >> 172.21.15.130:0/2860714091 conn(0x7f4b880faa80 msgr2=0x7f4b880fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:23.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.005+0000 7f4b8f611700 1 -- 172.21.15.130:0/2860714091 shutdown_connections 2023-09-12T16:04:23.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8f611700 1 -- 172.21.15.130:0/2860714091 wait complete. 2023-09-12T16:04:23.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8f611700 1 Processor -- start 2023-09-12T16:04:23.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8f611700 1 -- start start 2023-09-12T16:04:23.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8f611700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b880ff530 0x7f4b880692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8f611700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b88100720 0x7f4b8819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8f611700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b881010c0 0x7f4b8819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8f611700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8819ddd0 con 0x7f4b881010c0 2023-09-12T16:04:23.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8f611700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8819df40 con 0x7f4b880ff530 2023-09-12T16:04:23.011 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8f611700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8819e100 con 0x7f4b88100720 2023-09-12T16:04:23.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.006+0000 7f4b8dbae700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b881010c0 0x7f4b8819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8dbae700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b881010c0 0x7f4b8819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55570/0 (socket says 172.21.15.130:55570) 2023-09-12T16:04:23.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8dbae700 1 -- 172.21.15.130:0/3993142964 learned_addr learned my addr 172.21.15.130:0/3993142964 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:23.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8d3ad700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b880ff530 0x7f4b880692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b88100720 0x7f4b8819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8dbae700 1 -- 172.21.15.130:0/3993142964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b88100720 msgr2=0x7f4b8819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:23.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b88100720 0x7f4b8819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8dbae700 1 -- 172.21.15.130:0/3993142964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b880ff530 msgr2=0x7f4b880692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:23.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b880ff530 0x7f4b880692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8dbae700 1 -- 172.21.15.130:0/3993142964 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b780095f0 con 0x7f4b881010c0 2023-09-12T16:04:23.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b880ff530 0x7f4b880692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:23.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b881010c0 0x7f4b8819d630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000ef30 tx=0x7f4b8000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:23.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b7e7fc700 1 -- 172.21.15.130:0/3993142964 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80018070 con 0x7f4b881010c0 2023-09-12T16:04:23.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b7e7fc700 1 -- 172.21.15.130:0/3993142964 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8000f040 con 0x7f4b881010c0 2023-09-12T16:04:23.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8f611700 1 -- 172.21.15.130:0/3993142964 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8819e380 con 0x7f4b881010c0 2023-09-12T16:04:23.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b7e7fc700 1 -- 172.21.15.130:0/3993142964 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000cdb0 con 0x7f4b881010c0 2023-09-12T16:04:23.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.007+0000 7f4b8f611700 1 -- 172.21.15.130:0/3993142964 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8819e840 con 0x7f4b881010c0 2023-09-12T16:04:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.009+0000 7f4b7e7fc700 1 -- 172.21.15.130:0/3993142964 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f4b80003a00 con 0x7f4b881010c0 2023-09-12T16:04:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.010+0000 7f4b7e7fc700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4b740a0640 0x7f4b740a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.010+0000 7f4b7e7fc700 1 -- 172.21.15.130:0/3993142964 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4b800cbb70 con 0x7f4b881010c0 2023-09-12T16:04:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.010+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4b740a0640 0x7f4b740a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4b740a0640 0x7f4b740a2b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b88100580 tx=0x7f4b84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b9b7d7b80 1 Processor -- start 2023-09-12T16:04:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b9b7d7b80 1 -- start start 2023-09-12T16:04:23.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349702c00 0x55c349703020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349701800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 0x55c3497022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34965a730 con 0x55c3497035a0 2023-09-12T16:04:23.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c349658050 con 0x55c349702c00 2023-09-12T16:04:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c349653940 con 0x55c349701e00 2023-09-12T16:04:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b8cbac700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349701800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b8cbac700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349701800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55576/0 (socket says 172.21.15.130:55576) 2023-09-12T16:04:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b8cbac700 1 -- 172.21.15.130:0/3747774089 learned_addr learned my addr 172.21.15.130:0/3747774089 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:23.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349702c00 0x55c349703020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.011+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 0x55c3497022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b8cbac700 1 -- 172.21.15.130:0/3747774089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 msgr2=0x55c3497022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:23.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 0x55c3497022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.025 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b8cbac700 1 -- 172.21.15.130:0/3747774089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349702c00 msgr2=0x55c349703020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:23.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349702c00 0x55c349703020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b8cbac700 1 -- 172.21.15.130:0/3747774089 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c34947fde0 con 0x55c3497035a0 2023-09-12T16:04:23.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349702c00 0x55c349703020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:23.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349701800 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4b780058e0 tx=0x7f4b78005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31ddee1efb28630 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:23.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b737fe700 1 -- 172.21.15.130:0/3747774089 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7801d040 con 0x55c3497035a0 2023-09-12T16:04:23.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b737fe700 1 -- 172.21.15.130:0/3747774089 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b7800b790 con 0x55c3497035a0 2023-09-12T16:04:23.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.012+0000 7f4b737fe700 1 -- 172.21.15.130:0/3747774089 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78003ac0 con 0x55c3497035a0 2023-09-12T16:04:23.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3747774089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 msgr2=0x55c349701800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:23.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349701800 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4b780058e0 tx=0x7f4b78005e80 comp rx=0 tx=0).stop 2023-09-12T16:04:23.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3747774089 shutdown_connections 2023-09-12T16:04:23.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 0x55c3497022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349702c00 0x55c349703020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3747774089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349701800 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3747774089 >> 172.21.15.130:0/3747774089 conn(0x55c349676090 msgr2=0x55c3496fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:23.028 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3747774089 shutdown_connections 2023-09-12T16:04:23.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3747774089 wait complete. 2023-09-12T16:04:23.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 -- start start 2023-09-12T16:04:23.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.013+0000 7f4b9b7d7b80 1 Processor -- start 2023-09-12T16:04:23.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349701e00 0x55c34970b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 0x55c34970bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349712b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3495f1b70 con 0x55c3497035a0 2023-09-12T16:04:23.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3495f1870 con 0x55c349701e00 2023-09-12T16:04:23.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3494d5920 con 0x55c349702c00 2023-09-12T16:04:23.030 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b9b7d7b80 10 client.0 _ll_register_callbacks cb 0x55c34953ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:23.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8dbae700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349712b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8dbae700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349712b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55584/0 (socket says 172.21.15.130:55584) 2023-09-12T16:04:23.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8dbae700 1 -- 172.21.15.130:0/3648493772 learned_addr learned my addr 172.21.15.130:0/3648493772 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:23.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 0x55c34970bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.031 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349701e00 0x55c34970b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8dbae700 1 -- 172.21.15.130:0/3648493772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 msgr2=0x55c34970bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:23.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 0x55c34970bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8dbae700 1 -- 172.21.15.130:0/3648493772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349701e00 msgr2=0x55c34970b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:23.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349701e00 0x55c34970b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.032 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8dbae700 1 -- 172.21.15.130:0/3648493772 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b84017070 con 0x55c3497035a0 2023-09-12T16:04:23.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349712b20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4b800c9670 tx=0x7f4b800989e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb19da0790425c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:23.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:04:23.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b70ff9700 1 -- 172.21.15.130:0/3648493772 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b800b0070 con 0x55c3497035a0 2023-09-12T16:04:23.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b70ff9700 1 -- 172.21.15.130:0/3648493772 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4b8001b320 con 0x55c3497035a0 2023-09-12T16:04:23.033 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.014+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3648493772 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3495ab4f0 con 0x55c3497035a0 2023-09-12T16:04:23.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b70ff9700 1 -- 172.21.15.130:0/3648493772 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b8009c070 con 0x55c3497035a0 2023-09-12T16:04:23.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b9b7d7b80 10 client.15831 fetch_fsmap learned FSMap version 27 2023-09-12T16:04:23.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3648493772 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f4b881082c0 con 0x55c3497035a0 2023-09-12T16:04:23.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b70ff9700 1 -- 172.21.15.130:0/3648493772 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3495ab4f0 con 0x55c3497035a0 2023-09-12T16:04:23.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b70ff9700 1 -- 172.21.15.130:0/3648493772 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4b800b1260 con 0x55c3497035a0 2023-09-12T16:04:23.034 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b9b7d7b80 10 client.15831 fetch_fsmap finished waiting for FSMap version 27 2023-09-12T16:04:23.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b9b7d7b80 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:23.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b9b7d7b80 20 client.15831 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:23.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c349717330 0x55c349719750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b9b7d7b80 4 client.15831 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:04:23.035 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3648493772 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3496688a0 con 0x55c349717330 2023-09-12T16:04:23.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.015+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c349717330 0x55c349719750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b70ff9700 10 client.15831 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:23.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c349717330 0x55c349719750 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab60dee41c11d6e3 server_cookie=c8458139753b250a in_seq=0 out_seq=0 2023-09-12T16:04:23.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b70ff9700 1 -- 172.21.15.130:0/3648493772 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c3496688a0 con 0x55c349717330 2023-09-12T16:04:23.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b70ff9700 10 client.15831 handle_command_reply: tid=0 2023-09-12T16:04:23.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 client.15831 shutdown 2023-09-12T16:04:23.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3648493772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 msgr2=0x55c349712b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:23.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349712b20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4b800c9670 tx=0x7f4b800989e0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3648493772 shutdown_connections 2023-09-12T16:04:23.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c349717330 0x55c349719750 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c3497035a0 0x55c349712b20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b70ff9700 0 client.15831 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:23.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c349701e00 0x55c34970b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3648493772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 0x55c34970bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3648493772 >> 172.21.15.130:0/3648493772 conn(0x55c349676090 msgr2=0x55c3496fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:23.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3648493772 shutdown_connections 2023-09-12T16:04:23.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.016+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3648493772 wait complete. 2023-09-12T16:04:23.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.017+0000 7f4b9b7d7b80 20 client.15831 trim_cache size 0 max 16384 2023-09-12T16:04:23.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b9b7d7b80 1 Processor -- start 2023-09-12T16:04:23.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b9b7d7b80 1 -- start start 2023-09-12T16:04:23.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349701e00 0x55c349701710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 0x55c349712ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c3497133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34970aa50 con 0x55c349701e00 2023-09-12T16:04:23.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34970b110 con 0x55c3497035a0 2023-09-12T16:04:23.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c349716e50 con 0x55c349702c00 2023-09-12T16:04:23.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8d3ad700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349701e00 0x55c349701710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8d3ad700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349701e00 0x55c349701710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55588/0 (socket says 172.21.15.130:55588) 2023-09-12T16:04:23.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8d3ad700 1 -- 172.21.15.130:0/3832296810 learned_addr learned my addr 172.21.15.130:0/3832296810 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:23.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 0x55c349712ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c3497133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8cbac700 1 -- 172.21.15.130:0/3832296810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 msgr2=0x55c3497133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:23.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c3497133b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8cbac700 1 -- 172.21.15.130:0/3832296810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349701e00 msgr2=0x55c349701710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:23.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349701e00 0x55c349701710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8cbac700 1 -- 172.21.15.130:0/3832296810 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c34971e380 con 0x55c349702c00 2023-09-12T16:04:23.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.030+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c3497133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:23.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.031+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 0x55c349712ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005850 tx=0x7f4b7800b5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4863f14091590eff server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:23.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.031+0000 7f4b73fff700 1 -- 172.21.15.130:0/3832296810 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7801d070 con 0x55c349702c00 2023-09-12T16:04:23.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.031+0000 7f4b73fff700 1 -- 172.21.15.130:0/3832296810 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b78005580 con 0x55c349702c00 2023-09-12T16:04:23.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.031+0000 7f4b73fff700 1 -- 172.21.15.130:0/3832296810 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78003e10 con 0x55c349702c00 2023-09-12T16:04:23.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3832296810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 msgr2=0x55c349712ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:23.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 0x55c349712ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005850 tx=0x7f4b7800b5c0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3832296810 shutdown_connections 2023-09-12T16:04:23.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c3497133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349701e00 0x55c349701710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/3832296810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349702c00 0x55c349712ae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3832296810 >> 172.21.15.130:0/3832296810 conn(0x55c349676090 msgr2=0x55c3497021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:23.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3832296810 shutdown_connections 2023-09-12T16:04:23.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/3832296810 wait complete. 2023-09-12T16:04:23.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 -- start start 2023-09-12T16:04:23.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.032+0000 7f4b9b7d7b80 1 Processor -- start 2023-09-12T16:04:23.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 0x55c349710940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349702c00 0x55c349710e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b9b7d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c349711e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3497123d0 con 0x55c349702c00 2023-09-12T16:04:23.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c349712540 con 0x55c3497035a0 2023-09-12T16:04:23.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b9b7d7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c349712700 con 0x55c349701e00 2023-09-12T16:04:23.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b9b7d7b80 10 client.0 _ll_register_callbacks cb 0x7f4b80098e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:23.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8cbac700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349702c00 0x55c349710e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8cbac700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349702c00 0x55c349710e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55596/0 (socket says 172.21.15.130:55596) 2023-09-12T16:04:23.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8cbac700 1 -- 172.21.15.130:0/764154989 learned_addr learned my addr 172.21.15.130:0/764154989 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:23.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c349711e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 0x55c349710940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8cbac700 1 -- 172.21.15.130:0/764154989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 msgr2=0x55c349710940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:23.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 0x55c349710940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8cbac700 1 -- 172.21.15.130:0/764154989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 msgr2=0x55c349711e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:23.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c349711e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8cbac700 1 -- 172.21.15.130:0/764154989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b881082c0 con 0x55c349702c00 2023-09-12T16:04:23.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8dbae700 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c349711e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:23.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b8cbac700 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349702c00 0x55c349710e80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005b40 tx=0x7f4b78009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6efda1dfdf625b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:23.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.033+0000 7f4b70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:04:23.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b70ff9700 1 -- 172.21.15.130:0/764154989 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7801d040 con 0x55c349702c00 2023-09-12T16:04:23.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b70ff9700 1 -- 172.21.15.130:0/764154989 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4b78005050 con 0x55c349702c00 2023-09-12T16:04:23.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/764154989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b7800b790 con 0x55c349702c00 2023-09-12T16:04:23.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b70ff9700 1 -- 172.21.15.130:0/764154989 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b78005210 con 0x55c349702c00 2023-09-12T16:04:23.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b9b7d7b80 10 client.15837 fetch_fsmap learned FSMap version 27 2023-09-12T16:04:23.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/764154989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c34947fde0 con 0x55c349702c00 2023-09-12T16:04:23.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b70ff9700 1 -- 172.21.15.130:0/764154989 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b7800b790 con 0x55c349702c00 2023-09-12T16:04:23.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b70ff9700 1 -- 172.21.15.130:0/764154989 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4b78022510 con 0x55c349702c00 2023-09-12T16:04:23.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b9b7d7b80 10 client.15837 fetch_fsmap finished waiting for FSMap version 27 2023-09-12T16:04:23.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b9b7d7b80 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:23.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b9b7d7b80 20 client.15837 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:23.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c34976bde0 0x55c34976e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:23.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b9b7d7b80 4 client.15837 mds_command: new command op to 24593 tid=0[{"prefix": "damage ls"}] 2023-09-12T16:04:23.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/764154989 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c34964c920 con 0x55c34976bde0 2023-09-12T16:04:23.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.034+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c34976bde0 0x55c34976e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:23.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b8d3ad700 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c34976bde0 0x55c34976e200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578770079b7b990c server_cookie=60c18eb703fc9164 in_seq=0 out_seq=0 2023-09-12T16:04:23.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b70ff9700 10 client.15837 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:23.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b70ff9700 1 -- 172.21.15.130:0/764154989 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c34964c920 con 0x55c34976bde0 2023-09-12T16:04:23.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b70ff9700 10 client.15837 handle_command_reply: tid=0 2023-09-12T16:04:23.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 client.15837 shutdown 2023-09-12T16:04:23.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/764154989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349702c00 msgr2=0x55c349710e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:23.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349702c00 0x55c349710e80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005b40 tx=0x7f4b78009fd0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/764154989 shutdown_connections 2023-09-12T16:04:23.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55c34976bde0 0x55c34976e200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3497035a0 0x55c349711e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b70ff9700 0 client.15837 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:23.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c349701e00 0x55c349710940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 --2- 172.21.15.130:0/764154989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c349702c00 0x55c349710e80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/764154989 >> 172.21.15.130:0/764154989 conn(0x55c349676090 msgr2=0x55c3497133b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:23.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/764154989 shutdown_connections 2023-09-12T16:04:23.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 1 -- 172.21.15.130:0/764154989 wait complete. 2023-09-12T16:04:23.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.035+0000 7f4b9b7d7b80 20 client.15837 trim_cache size 0 max 16384 2023-09-12T16:04:23.062 INFO:teuthology.orchestra.run.smithi130.stdout:[] 2023-09-12T16:04:23.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 -- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4b740a0640 msgr2=0x7f4b740a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:23.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4b740a0640 0x7f4b740a2b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b88100580 tx=0x7f4b84009260 comp rx=0 tx=0).stop 2023-09-12T16:04:23.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 -- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b881010c0 msgr2=0x7f4b8819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:23.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b881010c0 0x7f4b8819d630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000ef30 tx=0x7f4b8000c3a0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 -- 172.21.15.130:0/3993142964 shutdown_connections 2023-09-12T16:04:23.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4b740a0640 0x7f4b740a2b00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b881010c0 0x7f4b8819d630 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b880ff530 0x7f4b880692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 --2- 172.21.15.130:0/3993142964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b88100720 0x7f4b8819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:23.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 -- 172.21.15.130:0/3993142964 >> 172.21.15.130:0/3993142964 conn(0x7f4b880faa80 msgr2=0x7f4b880fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:23.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 -- 172.21.15.130:0/3993142964 shutdown_connections 2023-09-12T16:04:23.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:23.036+0000 7f4b8f611700 1 -- 172.21.15.130:0/3993142964 wait complete. 2023-09-12T16:04:23.095 INFO:teuthology.orchestra.run.smithi130.stderr:16500000 of 50000000 tuples (33%) done (elapsed 25.46 s, remaining 51.69 s) 2023-09-12T16:04:23.182 INFO:teuthology.orchestra.run.smithi130.stderr:16600000 of 50000000 tuples (33%) done (elapsed 25.55 s, remaining 51.40 s) 2023-09-12T16:04:23.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:22 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1495641182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:04:23.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:22 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1495641182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:04:23.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:22 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1495641182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:04:23.301 INFO:teuthology.orchestra.run.smithi130.stderr:16700000 of 50000000 tuples (33%) done (elapsed 25.67 s, remaining 51.18 s) 2023-09-12T16:04:23.402 INFO:teuthology.orchestra.run.smithi130.stderr:16800000 of 50000000 tuples (33%) done (elapsed 25.77 s, remaining 50.92 s) 2023-09-12T16:04:23.481 INFO:teuthology.orchestra.run.smithi130.stderr:16900000 of 50000000 tuples (33%) done (elapsed 25.85 s, remaining 50.62 s) 2023-09-12T16:04:23.579 INFO:teuthology.orchestra.run.smithi130.stderr:17000000 of 50000000 tuples (34%) done (elapsed 25.94 s, remaining 50.36 s) 2023-09-12T16:04:23.712 INFO:teuthology.orchestra.run.smithi130.stderr:17100000 of 50000000 tuples (34%) done (elapsed 26.07 s, remaining 50.17 s) 2023-09-12T16:04:23.953 INFO:teuthology.orchestra.run.smithi130.stderr:17200000 of 50000000 tuples (34%) done (elapsed 26.32 s, remaining 50.19 s) 2023-09-12T16:04:24.067 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-12T16:04:24.068 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-12T16:04:24.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 -- 172.21.15.130:0/2858623896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83401039e0 msgr2=0x7f8340105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 --2- 172.21.15.130:0/2858623896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83401039e0 0x7f8340105dd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83340098f0 tx=0x7f8334009c00 comp rx=0 tx=0).stop 2023-09-12T16:04:24.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 -- 172.21.15.130:0/2858623896 shutdown_connections 2023-09-12T16:04:24.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 --2- 172.21.15.130:0/2858623896 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8340106310 0x7f8340108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.223 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 --2- 172.21.15.130:0/2858623896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83401010b0 0x7f83401034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 --2- 172.21.15.130:0/2858623896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83401039e0 0x7f8340105dd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 -- 172.21.15.130:0/2858623896 >> 172.21.15.130:0/2858623896 conn(0x7f83400faa60 msgr2=0x7f83400fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 -- 172.21.15.130:0/2858623896 shutdown_connections 2023-09-12T16:04:24.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 -- 172.21.15.130:0/2858623896 wait complete. 2023-09-12T16:04:24.224 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.220+0000 7f8345f82700 1 Processor -- start 2023-09-12T16:04:24.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f8345f82700 1 -- start start 2023-09-12T16:04:24.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f8345f82700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83401010b0 0x7f8340196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f8345f82700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83401039e0 0x7f8340196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.225 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f8345f82700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8340106310 0x7f834019d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f8345f82700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834019deb0 con 0x7f83401039e0 2023-09-12T16:04:24.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f8345f82700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834019dff0 con 0x7f83401010b0 2023-09-12T16:04:24.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f8345f82700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834019e180 con 0x7f8340106310 2023-09-12T16:04:24.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83401039e0 0x7f8340196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83401010b0 0x7f8340196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83401010b0 0x7f8340196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:38866/0 (socket says 172.21.15.130:38866) 2023-09-12T16:04:24.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83401039e0 0x7f8340196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55610/0 (socket says 172.21.15.130:55610) 2023-09-12T16:04:24.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8340106310 0x7f834019d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833f7fe700 1 -- 172.21.15.130:0/892167645 learned_addr learned my addr 172.21.15.130:0/892167645 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833f7fe700 1 -- 172.21.15.130:0/892167645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8340106310 msgr2=0x7f834019d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833f7fe700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8340106310 0x7f834019d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833f7fe700 1 -- 172.21.15.130:0/892167645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83401039e0 msgr2=0x7f8340196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833f7fe700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83401039e0 0x7f8340196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.229 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833f7fe700 1 -- 172.21.15.130:0/892167645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8338009d00 con 0x7f83401010b0 2023-09-12T16:04:24.229 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.221+0000 7f833ffff700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8340106310 0x7f834019d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:24.229 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.222+0000 7f833f7fe700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83401010b0 0x7f8340196330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f833000e9a0 tx=0x7f833000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.230 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.222+0000 7f833cff9700 1 -- 172.21.15.130:0/892167645 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330009790 con 0x7f83401010b0 2023-09-12T16:04:24.230 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.222+0000 7f8345f82700 1 -- 172.21.15.130:0/892167645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8334009520 con 0x7f83401010b0 2023-09-12T16:04:24.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.222+0000 7f8345f82700 1 -- 172.21.15.130:0/892167645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834019e830 con 0x7f83401010b0 2023-09-12T16:04:24.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.222+0000 7f833cff9700 1 -- 172.21.15.130:0/892167645 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8330010430 con 0x7f83401010b0 2023-09-12T16:04:24.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.222+0000 7f833cff9700 1 -- 172.21.15.130:0/892167645 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330018680 con 0x7f83401010b0 2023-09-12T16:04:24.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:23 smithi159 ceph-mon[102522]: pgmap v336: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 165 MiB/s wr, 214 op/s 2023-09-12T16:04:24.233 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:23 smithi130 ceph-mon[98670]: pgmap v336: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 165 MiB/s wr, 214 op/s 2023-09-12T16:04:24.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.224+0000 7f833cff9700 1 -- 172.21.15.130:0/892167645 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f833000ca10 con 0x7f83401010b0 2023-09-12T16:04:24.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.225+0000 7f833cff9700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f83280a0640 0x7f83280a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.225+0000 7f833cff9700 1 -- 172.21.15.130:0/892167645 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8330014070 con 0x7f83401010b0 2023-09-12T16:04:24.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.225+0000 7f833effd700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f83280a0640 0x7f83280a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.225+0000 7f8352148b80 1 Processor -- start 2023-09-12T16:04:24.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.225+0000 7f8352148b80 1 -- start start 2023-09-12T16:04:24.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.225+0000 7f833effd700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f83280a0640 0x7f83280a2b00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f833400fa40 tx=0x7f8334019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f8352148b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c434c80 0x561f7c4350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f8352148b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c433880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f8352148b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 0x561f7c434330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f8352148b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c38c7b0 con 0x561f7c435620 2023-09-12T16:04:24.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f8352148b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c38a0d0 con 0x561f7c433e80 2023-09-12T16:04:24.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f8352148b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c3859c0 con 0x561f7c434c80 2023-09-12T16:04:24.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c433880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c433880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55618/0 (socket says 172.21.15.130:55618) 2023-09-12T16:04:24.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c434c80 0x561f7c4350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833effd700 1 -- 172.21.15.130:0/2785345486 learned_addr learned my addr 172.21.15.130:0/2785345486 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833ffff700 1 --2- 172.21.15.130:0/2785345486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 0x561f7c434330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833effd700 1 -- 172.21.15.130:0/2785345486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c434c80 msgr2=0x561f7c4350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833effd700 1 --2- 172.21.15.130:0/2785345486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c434c80 0x561f7c4350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833effd700 1 -- 172.21.15.130:0/2785345486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 msgr2=0x561f7c434330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833effd700 1 --2- 172.21.15.130:0/2785345486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 0x561f7c434330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833effd700 1 -- 172.21.15.130:0/2785345486 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7c1b1e60 con 0x561f7c435620 2023-09-12T16:04:24.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833f7fe700 1 --2- 172.21.15.130:0/2785345486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c434c80 0x561f7c4350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:24.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f833effd700 1 --2- 172.21.15.130:0/2785345486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c433880 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f83340098f0 tx=0x7f833401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1574ee9b89e6e31c server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.226+0000 7f8325ffb700 1 -- 172.21.15.130:0/2785345486 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400b760 con 0x561f7c435620 2023-09-12T16:04:24.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8325ffb700 1 -- 172.21.15.130:0/2785345486 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f833401d040 con 0x561f7c435620 2023-09-12T16:04:24.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8325ffb700 1 -- 172.21.15.130:0/2785345486 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833402f700 con 0x561f7c435620 2023-09-12T16:04:24.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8352148b80 1 -- 172.21.15.130:0/2785345486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 msgr2=0x561f7c433880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8352148b80 1 --2- 172.21.15.130:0/2785345486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c433880 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f83340098f0 tx=0x7f833401e000 comp rx=0 tx=0).stop 2023-09-12T16:04:24.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8352148b80 1 -- 172.21.15.130:0/2785345486 shutdown_connections 2023-09-12T16:04:24.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8352148b80 1 --2- 172.21.15.130:0/2785345486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 0x561f7c434330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8352148b80 1 --2- 172.21.15.130:0/2785345486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c434c80 0x561f7c4350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8352148b80 1 --2- 172.21.15.130:0/2785345486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c433880 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8352148b80 1 -- 172.21.15.130:0/2785345486 >> 172.21.15.130:0/2785345486 conn(0x561f7c3a8110 msgr2=0x561f7c42e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8352148b80 1 -- 172.21.15.130:0/2785345486 shutdown_connections 2023-09-12T16:04:24.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.227+0000 7f8352148b80 1 -- 172.21.15.130:0/2785345486 wait complete. 2023-09-12T16:04:24.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f8352148b80 1 -- start start 2023-09-12T16:04:24.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f8352148b80 1 Processor -- start 2023-09-12T16:04:24.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f8352148b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 0x561f7c43d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f8352148b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c434c80 0x561f7c43dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f8352148b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c444be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f8352148b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c323bf0 con 0x561f7c435620 2023-09-12T16:04:24.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f8352148b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c3238f0 con 0x561f7c434c80 2023-09-12T16:04:24.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f8352148b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c2079a0 con 0x561f7c433e80 2023-09-12T16:04:24.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f8352148b80 10 client.0 _ll_register_callbacks cb 0x561f7c26db00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:24.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f833ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c444be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f833f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 0x561f7c43d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.228+0000 7f833ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c444be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55624/0 (socket says 172.21.15.130:55624) 2023-09-12T16:04:24.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f833ffff700 1 -- 172.21.15.130:0/2911847254 learned_addr learned my addr 172.21.15.130:0/2911847254 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f833effd700 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c434c80 0x561f7c43dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.253 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f833f7fe700 1 -- 172.21.15.130:0/2911847254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c434c80 msgr2=0x561f7c43dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f833f7fe700 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c434c80 0x561f7c43dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f833f7fe700 1 -- 172.21.15.130:0/2911847254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 msgr2=0x561f7c444be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f833f7fe700 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c444be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f833f7fe700 1 -- 172.21.15.130:0/2911847254 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834019e830 con 0x561f7c433e80 2023-09-12T16:04:24.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f833ffff700 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c444be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:24.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f833f7fe700 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 0x561f7c43d660 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f833000c2d0 tx=0x7f83300990f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4535860a65112a7a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f831effd700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:04:24.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f831effd700 1 -- 172.21.15.130:0/2911847254 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83300b0070 con 0x561f7c433e80 2023-09-12T16:04:24.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f831effd700 1 -- 172.21.15.130:0/2911847254 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f833009d0d0 con 0x561f7c433e80 2023-09-12T16:04:24.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f8352148b80 1 -- 172.21.15.130:0/2911847254 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f7c2dd570 con 0x561f7c433e80 2023-09-12T16:04:24.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f831effd700 1 -- 172.21.15.130:0/2911847254 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83300994e0 con 0x561f7c433e80 2023-09-12T16:04:24.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.229+0000 7f8352148b80 10 client.25510 fetch_fsmap learned FSMap version 28 2023-09-12T16:04:24.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f8352148b80 1 -- 172.21.15.130:0/2911847254 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f8340109330 con 0x561f7c433e80 2023-09-12T16:04:24.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f831effd700 1 -- 172.21.15.130:0/2911847254 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8330063080 con 0x561f7c433e80 2023-09-12T16:04:24.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f831effd700 1 -- 172.21.15.130:0/2911847254 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8330065140 con 0x561f7c433e80 2023-09-12T16:04:24.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f8352148b80 10 client.25510 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:04:24.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f8352148b80 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:24.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f8352148b80 20 client.25510 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:24.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f8352148b80 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561f7c447e10 0x561f7c448220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f8352148b80 4 client.25510 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:04:24.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f8352148b80 1 -- 172.21.15.130:0/2911847254 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f7c39a920 con 0x561f7c447e10 2023-09-12T16:04:24.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f833effd700 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561f7c447e10 0x561f7c448220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f831effd700 10 client.25510 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:24.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.230+0000 7f833effd700 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561f7c447e10 0x561f7c448220 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e5fc35868c99c7 server_cookie=afad1fa5df3fd960 in_seq=0 out_seq=0 2023-09-12T16:04:24.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f831effd700 1 -- 172.21.15.130:0/2911847254 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f7c39a920 con 0x561f7c447e10 2023-09-12T16:04:24.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f831effd700 10 client.25510 handle_command_reply: tid=0 2023-09-12T16:04:24.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 client.25510 shutdown 2023-09-12T16:04:24.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 -- 172.21.15.130:0/2911847254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 msgr2=0x561f7c43d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 0x561f7c43d660 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f833000c2d0 tx=0x7f83300990f0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 -- 172.21.15.130:0/2911847254 shutdown_connections 2023-09-12T16:04:24.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561f7c447e10 0x561f7c448220 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c435620 0x561f7c444be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 0x561f7c43d660 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 --2- 172.21.15.130:0/2911847254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c434c80 0x561f7c43dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f831effd700 0 client.25510 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:24.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 -- 172.21.15.130:0/2911847254 >> 172.21.15.130:0/2911847254 conn(0x561f7c3a8110 msgr2=0x561f7c42de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 -- 172.21.15.130:0/2911847254 shutdown_connections 2023-09-12T16:04:24.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 1 -- 172.21.15.130:0/2911847254 wait complete. 2023-09-12T16:04:24.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.231+0000 7f8352148b80 20 client.25510 trim_cache size 0 max 16384 2023-09-12T16:04:24.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:23 smithi203 ceph-mon[103431]: pgmap v336: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 165 MiB/s wr, 214 op/s 2023-09-12T16:04:24.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f8352148b80 1 Processor -- start 2023-09-12T16:04:24.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f8352148b80 1 -- start start 2023-09-12T16:04:24.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f8352148b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 0x561f7c43d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f8352148b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c444ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f8352148b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c435620 0x561f7c43d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f8352148b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c43b220 con 0x561f7c434c80 2023-09-12T16:04:24.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f8352148b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c43e420 con 0x561f7c433e80 2023-09-12T16:04:24.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f8352148b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c447b30 con 0x561f7c435620 2023-09-12T16:04:24.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c444ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 0x561f7c43d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c444ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55632/0 (socket says 172.21.15.130:55632) 2023-09-12T16:04:24.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833effd700 1 -- 172.21.15.130:0/2640834115 learned_addr learned my addr 172.21.15.130:0/2640834115 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833ffff700 1 --2- 172.21.15.130:0/2640834115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c435620 0x561f7c43d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833effd700 1 -- 172.21.15.130:0/2640834115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c435620 msgr2=0x561f7c43d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833effd700 1 --2- 172.21.15.130:0/2640834115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c435620 0x561f7c43d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833effd700 1 -- 172.21.15.130:0/2640834115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 msgr2=0x561f7c43d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833effd700 1 --2- 172.21.15.130:0/2640834115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 0x561f7c43d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833effd700 1 -- 172.21.15.130:0/2640834115 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7c459770 con 0x561f7c434c80 2023-09-12T16:04:24.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.251+0000 7f833f7fe700 1 --2- 172.21.15.130:0/2640834115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 0x561f7c43d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:24.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.252+0000 7f833effd700 1 --2- 172.21.15.130:0/2640834115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c444ba0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8334009810 tx=0x7f833400f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e74e835a0a02919 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.252+0000 7f83267fc700 1 -- 172.21.15.130:0/2640834115 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334034040 con 0x561f7c434c80 2023-09-12T16:04:24.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.252+0000 7f83267fc700 1 -- 172.21.15.130:0/2640834115 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8334039dc0 con 0x561f7c434c80 2023-09-12T16:04:24.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.252+0000 7f83267fc700 1 -- 172.21.15.130:0/2640834115 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400b8c0 con 0x561f7c434c80 2023-09-12T16:04:24.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 -- 172.21.15.130:0/2640834115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 msgr2=0x561f7c444ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 --2- 172.21.15.130:0/2640834115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c444ba0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8334009810 tx=0x7f833400f960 comp rx=0 tx=0).stop 2023-09-12T16:04:24.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 -- 172.21.15.130:0/2640834115 shutdown_connections 2023-09-12T16:04:24.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 --2- 172.21.15.130:0/2640834115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c435620 0x561f7c43d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 --2- 172.21.15.130:0/2640834115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c433e80 0x561f7c43d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 --2- 172.21.15.130:0/2640834115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c444ba0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 -- 172.21.15.130:0/2640834115 >> 172.21.15.130:0/2640834115 conn(0x561f7c3a8110 msgr2=0x561f7c437fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 -- 172.21.15.130:0/2640834115 shutdown_connections 2023-09-12T16:04:24.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 -- 172.21.15.130:0/2640834115 wait complete. 2023-09-12T16:04:24.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 -- start start 2023-09-12T16:04:24.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.253+0000 7f8352148b80 1 Processor -- start 2023-09-12T16:04:24.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f8352148b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 0x561f7c440cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f8352148b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c441200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f8352148b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c435620 0x561f7c444220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f8352148b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c444760 con 0x561f7c434c80 2023-09-12T16:04:24.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f8352148b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c4448d0 con 0x561f7c435620 2023-09-12T16:04:24.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f8352148b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7c444a90 con 0x561f7c433e80 2023-09-12T16:04:24.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f8352148b80 10 client.0 _ll_register_callbacks cb 0x7f83300992b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:24.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c441200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c441200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55638/0 (socket says 172.21.15.130:55638) 2023-09-12T16:04:24.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833effd700 1 -- 172.21.15.130:0/2231177909 learned_addr learned my addr 172.21.15.130:0/2231177909 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833f7fe700 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 0x561f7c440cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833ffff700 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c435620 0x561f7c444220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833effd700 1 -- 172.21.15.130:0/2231177909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 msgr2=0x561f7c440cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833effd700 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 0x561f7c440cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833effd700 1 -- 172.21.15.130:0/2231177909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c435620 msgr2=0x561f7c444220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833effd700 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c435620 0x561f7c444220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833effd700 1 -- 172.21.15.130:0/2231177909 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8340109330 con 0x561f7c434c80 2023-09-12T16:04:24.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833ffff700 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c435620 0x561f7c444220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:04:24.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f833effd700 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c441200 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8334005850 tx=0x7f833400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f13bd7655aa3259 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f831effd700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:04:24.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f831effd700 1 -- 172.21.15.130:0/2231177909 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334034070 con 0x561f7c434c80 2023-09-12T16:04:24.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f831effd700 1 -- 172.21.15.130:0/2231177909 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8334039b00 con 0x561f7c434c80 2023-09-12T16:04:24.277 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.254+0000 7f8352148b80 1 -- 172.21.15.130:0/2231177909 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f833401d040 con 0x561f7c434c80 2023-09-12T16:04:24.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f831effd700 1 -- 172.21.15.130:0/2231177909 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83340055b0 con 0x561f7c434c80 2023-09-12T16:04:24.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f8352148b80 10 client.15855 fetch_fsmap learned FSMap version 28 2023-09-12T16:04:24.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f8352148b80 1 -- 172.21.15.130:0/2231177909 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561f7c1b1e60 con 0x561f7c434c80 2023-09-12T16:04:24.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f831effd700 1 -- 172.21.15.130:0/2231177909 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f833401d040 con 0x561f7c434c80 2023-09-12T16:04:24.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f831effd700 1 -- 172.21.15.130:0/2231177909 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8334040070 con 0x561f7c434c80 2023-09-12T16:04:24.278 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f8352148b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:04:24.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f8352148b80 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:24.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f8352148b80 20 client.15855 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:24.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f8352148b80 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561f7c4a7600 0x561f7c4a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f8352148b80 4 client.15855 mds_command: new command op to 24593 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-12T16:04:24.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f8352148b80 1 -- 172.21.15.130:0/2231177909 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561f7c37e9a0 con 0x561f7c4a7600 2023-09-12T16:04:24.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f833f7fe700 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561f7c4a7600 0x561f7c4a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f833f7fe700 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561f7c4a7600 0x561f7c4a9a20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e505bc57a30cf4d9 server_cookie=2249fd974628391e in_seq=0 out_seq=0 2023-09-12T16:04:24.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.255+0000 7f831effd700 10 client.15855 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:24.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f831effd700 1 -- 172.21.15.130:0/2231177909 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561f7c37e9a0 con 0x561f7c4a7600 2023-09-12T16:04:24.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f831effd700 10 client.15855 handle_command_reply: tid=0 2023-09-12T16:04:24.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 client.15855 shutdown 2023-09-12T16:04:24.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 -- 172.21.15.130:0/2231177909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 msgr2=0x561f7c441200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c441200 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8334005850 tx=0x7f833400f770 comp rx=0 tx=0).stop 2023-09-12T16:04:24.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 -- 172.21.15.130:0/2231177909 shutdown_connections 2023-09-12T16:04:24.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561f7c4a7600 0x561f7c4a9a20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f7c435620 0x561f7c444220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f7c433e80 0x561f7c440cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f831effd700 0 client.15855 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:24.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 --2- 172.21.15.130:0/2231177909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561f7c434c80 0x561f7c441200 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 -- 172.21.15.130:0/2231177909 >> 172.21.15.130:0/2231177909 conn(0x561f7c3a8110 msgr2=0x561f7c43d410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 -- 172.21.15.130:0/2231177909 shutdown_connections 2023-09-12T16:04:24.283 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 1 -- 172.21.15.130:0/2231177909 wait complete. 2023-09-12T16:04:24.283 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.257+0000 7f8352148b80 20 client.15855 trim_cache size 0 max 16384 2023-09-12T16:04:24.283 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:04:24.283 INFO:teuthology.orchestra.run.smithi130.stdout: "return_code": 0, 2023-09-12T16:04:24.284 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:04:24.284 INFO:teuthology.orchestra.run.smithi130.stdout: "mode": "asynchronous" 2023-09-12T16:04:24.284 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:04:24.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 -- 172.21.15.130:0/892167645 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f83280a0640 msgr2=0x7f83280a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f83280a0640 0x7f83280a2b00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f833400fa40 tx=0x7f8334019040 comp rx=0 tx=0).stop 2023-09-12T16:04:24.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 -- 172.21.15.130:0/892167645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83401010b0 msgr2=0x7f8340196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83401010b0 0x7f8340196330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f833000e9a0 tx=0x7f833000ecb0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 -- 172.21.15.130:0/892167645 shutdown_connections 2023-09-12T16:04:24.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f83280a0640 0x7f83280a2b00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8340106310 0x7f834019d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83401010b0 0x7f8340196330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 --2- 172.21.15.130:0/892167645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83401039e0 0x7f8340196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 -- 172.21.15.130:0/892167645 >> 172.21.15.130:0/892167645 conn(0x7f83400faa60 msgr2=0x7f8340101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 -- 172.21.15.130:0/892167645 shutdown_connections 2023-09-12T16:04:24.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.258+0000 7f8345f82700 1 -- 172.21.15.130:0/892167645 wait complete. 2023-09-12T16:04:24.288 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:04:24.432 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.430+0000 7fe32208d700 1 -- 172.21.15.130:0/3109957269 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 msgr2=0x7fe31c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.430+0000 7fe32208d700 1 --2- 172.21.15.130:0/3109957269 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 0x7fe31c100b40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe3100099c0 tx=0x7fe310009cd0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.430+0000 7fe32208d700 1 -- 172.21.15.130:0/3109957269 shutdown_connections 2023-09-12T16:04:24.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.430+0000 7fe32208d700 1 --2- 172.21.15.130:0/3109957269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c101080 0x7fe31c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.430+0000 7fe32208d700 1 --2- 172.21.15.130:0/3109957269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31c0ff4f0 0x7fe31c0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.430+0000 7fe32208d700 1 --2- 172.21.15.130:0/3109957269 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 0x7fe31c100b40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.430+0000 7fe32208d700 1 -- 172.21.15.130:0/3109957269 >> 172.21.15.130:0/3109957269 conn(0x7fe31c0faa60 msgr2=0x7fe31c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.430+0000 7fe32208d700 1 -- 172.21.15.130:0/3109957269 shutdown_connections 2023-09-12T16:04:24.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.430+0000 7fe32208d700 1 -- 172.21.15.130:0/3109957269 wait complete. 2023-09-12T16:04:24.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe32208d700 1 Processor -- start 2023-09-12T16:04:24.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe32208d700 1 -- start start 2023-09-12T16:04:24.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe32208d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c0ff4f0 0x7fe31c0682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe32208d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 0x7fe31c19b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe32208d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31c101080 0x7fe31c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe32208d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c19de60 con 0x7fe31c1006e0 2023-09-12T16:04:24.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe32208d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c19dfd0 con 0x7fe31c0ff4f0 2023-09-12T16:04:24.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe32208d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c19e190 con 0x7fe31c101080 2023-09-12T16:04:24.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 0x7fe31c19b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31c101080 0x7fe31c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 0x7fe31c19b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55650/0 (socket says 172.21.15.130:55650) 2023-09-12T16:04:24.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c0ff4f0 0x7fe31c0682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31affd700 1 -- 172.21.15.130:0/1502116277 learned_addr learned my addr 172.21.15.130:0/1502116277 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31affd700 1 -- 172.21.15.130:0/1502116277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31c101080 msgr2=0x7fe31c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31affd700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31c101080 0x7fe31c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31affd700 1 -- 172.21.15.130:0/1502116277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c0ff4f0 msgr2=0x7fe31c0682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31affd700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c0ff4f0 0x7fe31c0682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31affd700 1 -- 172.21.15.130:0/1502116277 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100095f0 con 0x7fe31c1006e0 2023-09-12T16:04:24.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31bfff700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31c101080 0x7fe31c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:24.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.431+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c0ff4f0 0x7fe31c0682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:24.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.432+0000 7fe31affd700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 0x7fe31c19b040 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe310009fd0 tx=0x7fe310009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.432+0000 7fe318ff9700 1 -- 172.21.15.130:0/1502116277 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31001d070 con 0x7fe31c1006e0 2023-09-12T16:04:24.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.432+0000 7fe318ff9700 1 -- 172.21.15.130:0/1502116277 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe310005030 con 0x7fe31c1006e0 2023-09-12T16:04:24.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.432+0000 7fe32208d700 1 -- 172.21.15.130:0/1502116277 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c19e410 con 0x7fe31c1006e0 2023-09-12T16:04:24.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.432+0000 7fe318ff9700 1 -- 172.21.15.130:0/1502116277 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310004a90 con 0x7fe31c1006e0 2023-09-12T16:04:24.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.432+0000 7fe32208d700 1 -- 172.21.15.130:0/1502116277 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c19e830 con 0x7fe31c1006e0 2023-09-12T16:04:24.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.434+0000 7fe318ff9700 1 -- 172.21.15.130:0/1502116277 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fe3100051a0 con 0x7fe31c1006e0 2023-09-12T16:04:24.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.435+0000 7fe318ff9700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe3040a0640 0x7fe3040a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.435+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe3040a0640 0x7fe3040a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.435+0000 7fe318ff9700 1 -- 172.21.15.130:0/1502116277 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe3100cccb0 con 0x7fe31c1006e0 2023-09-12T16:04:24.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.435+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe3040a0640 0x7fe3040a2b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe31c100540 tx=0x7fe30c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe32e253b80 1 Processor -- start 2023-09-12T16:04:24.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe32e253b80 1 -- start start 2023-09-12T16:04:24.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b714403020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b7144035a0 0x55b714401800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b7144022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe32e253b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71435a730 con 0x55b7144035a0 2023-09-12T16:04:24.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe32e253b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b714358050 con 0x55b714402c00 2023-09-12T16:04:24.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe32e253b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b714353940 con 0x55b714401e00 2023-09-12T16:04:24.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b714403020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe31affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b7144035a0 0x55b714401800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe31bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b7144022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe31bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b7144022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35272/0 (socket says 172.21.15.130:35272) 2023-09-12T16:04:24.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b714403020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:38914/0 (socket says 172.21.15.130:38914) 2023-09-12T16:04:24.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.436+0000 7fe31bfff700 1 -- 172.21.15.130:0/921142385 learned_addr learned my addr 172.21.15.130:0/921142385 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe31affd700 1 -- 172.21.15.130:0/921142385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 msgr2=0x55b7144022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe31affd700 1 --2- 172.21.15.130:0/921142385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b7144022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe31affd700 1 -- 172.21.15.130:0/921142385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 msgr2=0x55b714403020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe31affd700 1 --2- 172.21.15.130:0/921142385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b714403020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe31affd700 1 -- 172.21.15.130:0/921142385 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b71417fde0 con 0x55b7144035a0 2023-09-12T16:04:24.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/921142385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b714403020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:24.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe31affd700 1 --2- 172.21.15.130:0/921142385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b7144035a0 0x55b714401800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe31000bf40 tx=0x7fe31009a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37768c1231f2b10a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe301ffb700 1 -- 172.21.15.130:0/921142385 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100a21a0 con 0x55b7144035a0 2023-09-12T16:04:24.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe301ffb700 1 -- 172.21.15.130:0/921142385 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3100a0070 con 0x55b7144035a0 2023-09-12T16:04:24.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.437+0000 7fe301ffb700 1 -- 172.21.15.130:0/921142385 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31009a420 con 0x55b7144035a0 2023-09-12T16:04:24.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 -- 172.21.15.130:0/921142385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b7144035a0 msgr2=0x55b714401800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 --2- 172.21.15.130:0/921142385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b7144035a0 0x55b714401800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe31000bf40 tx=0x7fe31009a8c0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 -- 172.21.15.130:0/921142385 shutdown_connections 2023-09-12T16:04:24.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 --2- 172.21.15.130:0/921142385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b7144022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 --2- 172.21.15.130:0/921142385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b714403020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 --2- 172.21.15.130:0/921142385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b7144035a0 0x55b714401800 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 -- 172.21.15.130:0/921142385 >> 172.21.15.130:0/921142385 conn(0x55b714376090 msgr2=0x55b7143fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 -- 172.21.15.130:0/921142385 shutdown_connections 2023-09-12T16:04:24.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 -- 172.21.15.130:0/921142385 wait complete. 2023-09-12T16:04:24.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.438+0000 7fe32e253b80 1 -- start start 2023-09-12T16:04:24.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe32e253b80 1 Processor -- start 2023-09-12T16:04:24.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 0x55b71440b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 0x55b714412850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe32e253b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7142f1b70 con 0x55b714401e00 2023-09-12T16:04:24.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe32e253b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7142f1870 con 0x55b714402c00 2023-09-12T16:04:24.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe32e253b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7141d5920 con 0x55b7144035a0 2023-09-12T16:04:24.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe32e253b80 10 client.0 _ll_register_callbacks cb 0x55b71423ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:24.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe31affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe31affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:38922/0 (socket says 172.21.15.130:38922) 2023-09-12T16:04:24.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe31affd700 1 -- 172.21.15.130:0/154177332 learned_addr learned my addr 172.21.15.130:0/154177332 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.439+0000 7fe31bfff700 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 0x55b714412850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe31affd700 1 -- 172.21.15.130:0/154177332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 msgr2=0x55b714412850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe31affd700 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 0x55b714412850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe31affd700 1 -- 172.21.15.130:0/154177332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 msgr2=0x55b71440b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe31affd700 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 0x55b71440b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe31affd700 1 -- 172.21.15.130:0/154177332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c017070 con 0x55b714402c00 2023-09-12T16:04:24.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe31affd700 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440b810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3100966c0 tx=0x7fe310004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e7ff139896d2c2d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe2faffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:04:24.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe2faffd700 1 -- 172.21.15.130:0/154177332 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310098300 con 0x55b714402c00 2023-09-12T16:04:24.457 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe32e253b80 1 -- 172.21.15.130:0/154177332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7142ab4f0 con 0x55b714402c00 2023-09-12T16:04:24.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe2faffd700 1 -- 172.21.15.130:0/154177332 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe310098560 con 0x55b714402c00 2023-09-12T16:04:24.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe2faffd700 1 -- 172.21.15.130:0/154177332 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3100a21a0 con 0x55b714402c00 2023-09-12T16:04:24.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe32e253b80 10 client.25331 fetch_fsmap learned FSMap version 28 2023-09-12T16:04:24.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe32e253b80 1 -- 172.21.15.130:0/154177332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fe31c19e830 con 0x55b714402c00 2023-09-12T16:04:24.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.440+0000 7fe2faffd700 1 -- 172.21.15.130:0/154177332 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe310064070 con 0x55b714402c00 2023-09-12T16:04:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe2faffd700 1 -- 172.21.15.130:0/154177332 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe3100961c0 con 0x55b714402c00 2023-09-12T16:04:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe32e253b80 10 client.25331 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:04:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe32e253b80 10 client.25331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe32e253b80 20 client.25331 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe32e253b80 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b714416c40 0x55b714419080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe32e253b80 4 client.25331 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:04:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe32e253b80 1 -- 172.21.15.130:0/154177332 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7143688a0 con 0x55b714416c40 2023-09-12T16:04:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b714416c40 0x55b714419080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe2faffd700 10 client.25331 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.441+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b714416c40 0x55b714419080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=997bfb8b4f9181ce server_cookie=28aac81f25c662f9 in_seq=0 out_seq=0 2023-09-12T16:04:24.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe2faffd700 1 -- 172.21.15.130:0/154177332 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b7143688a0 con 0x55b714416c40 2023-09-12T16:04:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe2faffd700 10 client.25331 handle_command_reply: tid=0 2023-09-12T16:04:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 client.25331 shutdown 2023-09-12T16:04:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 -- 172.21.15.130:0/154177332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 msgr2=0x55b71440b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440b810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3100966c0 tx=0x7fe310004f90 comp rx=0 tx=0).stop 2023-09-12T16:04:24.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 -- 172.21.15.130:0/154177332 shutdown_connections 2023-09-12T16:04:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b714416c40 0x55b714419080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 0x55b714412850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 0x55b71440b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 --2- 172.21.15.130:0/154177332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440b810 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe2faffd700 0 client.25331 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:24.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 -- 172.21.15.130:0/154177332 >> 172.21.15.130:0/154177332 conn(0x55b714376090 msgr2=0x55b7143fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 -- 172.21.15.130:0/154177332 shutdown_connections 2023-09-12T16:04:24.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 1 -- 172.21.15.130:0/154177332 wait complete. 2023-09-12T16:04:24.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.442+0000 7fe32e253b80 20 client.25331 trim_cache size 0 max 16384 2023-09-12T16:04:24.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe32e253b80 1 Processor -- start 2023-09-12T16:04:24.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe32e253b80 1 -- start start 2023-09-12T16:04:24.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b714401710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714402c00 0x55b714412810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7144035a0 0x55b714413140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe32e253b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7143fd490 con 0x55b714402c00 2023-09-12T16:04:24.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe32e253b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b714401c80 con 0x55b7144035a0 2023-09-12T16:04:24.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe32e253b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71440aea0 con 0x55b714401e00 2023-09-12T16:04:24.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe31affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714402c00 0x55b714412810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.463+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b714401710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714402c00 0x55b714412810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55672/0 (socket says 172.21.15.130:55672) 2023-09-12T16:04:24.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31affd700 1 -- 172.21.15.130:0/3861522247 learned_addr learned my addr 172.21.15.130:0/3861522247 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31bfff700 1 --2- 172.21.15.130:0/3861522247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7144035a0 0x55b714413140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31b7fe700 1 -- 172.21.15.130:0/3861522247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7144035a0 msgr2=0x55b714413140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/3861522247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7144035a0 0x55b714413140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31b7fe700 1 -- 172.21.15.130:0/3861522247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714402c00 msgr2=0x55b714412810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/3861522247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714402c00 0x55b714412810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31b7fe700 1 -- 172.21.15.130:0/3861522247 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b71441dcf0 con 0x55b714401e00 2023-09-12T16:04:24.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31affd700 1 --2- 172.21.15.130:0/3861522247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714402c00 0x55b714412810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:24.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/3861522247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b714401710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe30c011bb0 tx=0x7fe30c013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2809a4e5022b8590 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe3027fc700 1 -- 172.21.15.130:0/3861522247 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c022040 con 0x55b714401e00 2023-09-12T16:04:24.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe3027fc700 1 -- 172.21.15.130:0/3861522247 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe30c0284f0 con 0x55b714401e00 2023-09-12T16:04:24.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.464+0000 7fe3027fc700 1 -- 172.21.15.130:0/3861522247 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c01d740 con 0x55b714401e00 2023-09-12T16:04:24.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 -- 172.21.15.130:0/3861522247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 msgr2=0x55b714401710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 --2- 172.21.15.130:0/3861522247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b714401710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe30c011bb0 tx=0x7fe30c013040 comp rx=0 tx=0).stop 2023-09-12T16:04:24.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 -- 172.21.15.130:0/3861522247 shutdown_connections 2023-09-12T16:04:24.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 --2- 172.21.15.130:0/3861522247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7144035a0 0x55b714413140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 --2- 172.21.15.130:0/3861522247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b714401e00 0x55b714401710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 --2- 172.21.15.130:0/3861522247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714402c00 0x55b714412810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 -- 172.21.15.130:0/3861522247 >> 172.21.15.130:0/3861522247 conn(0x55b714376090 msgr2=0x55b7144021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 -- 172.21.15.130:0/3861522247 shutdown_connections 2023-09-12T16:04:24.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 -- 172.21.15.130:0/3861522247 wait complete. 2023-09-12T16:04:24.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.465+0000 7fe32e253b80 1 -- start start 2023-09-12T16:04:24.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe32e253b80 1 Processor -- start 2023-09-12T16:04:24.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 0x55b71440ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe32e253b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 0x55b714411f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe32e253b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7144124a0 con 0x55b714401e00 2023-09-12T16:04:24.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe32e253b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b714412610 con 0x55b714402c00 2023-09-12T16:04:24.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe32e253b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7144127d0 con 0x55b7144035a0 2023-09-12T16:04:24.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe32e253b80 10 client.0 _ll_register_callbacks cb 0x7fe30c00e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:24.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 0x55b71440ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 0x55b71440ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55676/0 (socket says 172.21.15.130:55676) 2023-09-12T16:04:24.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31b7fe700 1 -- 172.21.15.130:0/2831428043 learned_addr learned my addr 172.21.15.130:0/2831428043 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:24.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31bfff700 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 0x55b714411f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31b7fe700 1 -- 172.21.15.130:0/2831428043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 msgr2=0x55b714411f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 0x55b714411f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31b7fe700 1 -- 172.21.15.130:0/2831428043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 msgr2=0x55b71440ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:24.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.466+0000 7fe31b7fe700 1 -- 172.21.15.130:0/2831428043 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c19e830 con 0x55b714401e00 2023-09-12T16:04:24.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe31b7fe700 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 0x55b71440ea00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe30c005810 tx=0x7fe30c005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43e762a70889d73 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:24.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe2faffd700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:04:24.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe2faffd700 1 -- 172.21.15.130:0/2831428043 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c022040 con 0x55b714401e00 2023-09-12T16:04:24.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe32e253b80 1 -- 172.21.15.130:0/2831428043 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3100a0070 con 0x55b714401e00 2023-09-12T16:04:24.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe2faffd700 1 -- 172.21.15.130:0/2831428043 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe30c00eb90 con 0x55b714401e00 2023-09-12T16:04:24.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe2faffd700 1 -- 172.21.15.130:0/2831428043 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe30c01d480 con 0x55b714401e00 2023-09-12T16:04:24.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe32e253b80 10 client.15870 fetch_fsmap learned FSMap version 28 2023-09-12T16:04:24.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe32e253b80 1 -- 172.21.15.130:0/2831428043 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fe30c028970 con 0x55b714401e00 2023-09-12T16:04:24.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe2faffd700 1 -- 172.21.15.130:0/2831428043 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3100a0070 con 0x55b714401e00 2023-09-12T16:04:24.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe2faffd700 1 -- 172.21.15.130:0/2831428043 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe30c01c750 con 0x55b714401e00 2023-09-12T16:04:24.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe32e253b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:04:24.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe32e253b80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:24.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe32e253b80 20 client.15870 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:24.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe32e253b80 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b71446bef0 0x55b71446e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:24.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe32e253b80 4 client.15870 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:04:24.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.467+0000 7fe32e253b80 1 -- 172.21.15.130:0/2831428043 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b71434c920 con 0x55b71446bef0 2023-09-12T16:04:24.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.468+0000 7fe31affd700 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b71446bef0 0x55b71446e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:24.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.468+0000 7fe31affd700 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b71446bef0 0x55b71446e310 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103bab7c3a005825 server_cookie=24d3944bea0c2965 in_seq=0 out_seq=0 2023-09-12T16:04:24.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.468+0000 7fe2faffd700 10 client.15870 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:24.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.468+0000 7fe2faffd700 1 -- 172.21.15.130:0/2831428043 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b71434c920 con 0x55b71446bef0 2023-09-12T16:04:24.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.468+0000 7fe2faffd700 10 client.15870 handle_command_reply: tid=0 2023-09-12T16:04:24.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.468+0000 7fe32e253b80 1 client.15870 shutdown 2023-09-12T16:04:24.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 -- 172.21.15.130:0/2831428043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 msgr2=0x55b71440ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 0x55b71440ea00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe30c005810 tx=0x7fe30c005860 comp rx=0 tx=0).stop 2023-09-12T16:04:24.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 -- 172.21.15.130:0/2831428043 shutdown_connections 2023-09-12T16:04:24.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b71446bef0 0x55b71446e310 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7144035a0 0x55b714411f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b714401e00 0x55b71440ea00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe2faffd700 0 client.15870 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:24.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 --2- 172.21.15.130:0/2831428043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b714402c00 0x55b71440ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 -- 172.21.15.130:0/2831428043 >> 172.21.15.130:0/2831428043 conn(0x55b714376090 msgr2=0x55b714413140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 -- 172.21.15.130:0/2831428043 shutdown_connections 2023-09-12T16:04:24.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 1 -- 172.21.15.130:0/2831428043 wait complete. 2023-09-12T16:04:24.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32e253b80 20 client.15870 trim_cache size 0 max 16384 2023-09-12T16:04:24.486 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:04:24.486 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:04:24.486 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:04:24.486 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:04:24.486 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:04:24.487 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:04:24.487 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:04:24.487 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:04:24.487 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:04:24.487 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:04:24.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 -- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe3040a0640 msgr2=0x7fe3040a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe3040a0640 0x7fe3040a2b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe31c100540 tx=0x7fe30c009260 comp rx=0 tx=0).stop 2023-09-12T16:04:24.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 -- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 msgr2=0x7fe31c19b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:24.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 0x7fe31c19b040 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe310009fd0 tx=0x7fe310009580 comp rx=0 tx=0).stop 2023-09-12T16:04:24.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 -- 172.21.15.130:0/1502116277 shutdown_connections 2023-09-12T16:04:24.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe3040a0640 0x7fe3040a2b00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31c101080 0x7fe31c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c0ff4f0 0x7fe31c0682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 --2- 172.21.15.130:0/1502116277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe31c1006e0 0x7fe31c19b040 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:24.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 -- 172.21.15.130:0/1502116277 >> 172.21.15.130:0/1502116277 conn(0x7fe31c0faa60 msgr2=0x7fe31c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:24.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 -- 172.21.15.130:0/1502116277 shutdown_connections 2023-09-12T16:04:24.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:24.469+0000 7fe32208d700 1 -- 172.21.15.130:0/1502116277 wait complete. 2023-09-12T16:04:24.491 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:04:24.606 INFO:teuthology.orchestra.run.smithi130.stderr:17300000 of 50000000 tuples (34%) done (elapsed 26.97 s, remaining 50.98 s) 2023-09-12T16:04:24.680 INFO:teuthology.orchestra.run.smithi130.stderr:17400000 of 50000000 tuples (34%) done (elapsed 27.05 s, remaining 50.67 s) 2023-09-12T16:04:24.820 INFO:teuthology.orchestra.run.smithi130.stderr:17500000 of 50000000 tuples (35%) done (elapsed 27.19 s, remaining 50.49 s) 2023-09-12T16:04:24.891 INFO:teuthology.orchestra.run.smithi130.stderr:17600000 of 50000000 tuples (35%) done (elapsed 27.26 s, remaining 50.18 s) 2023-09-12T16:04:25.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:24 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:25.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:24 smithi130 ceph-mon[98670]: scrub queued for path: / 2023-09-12T16:04:25.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:24 smithi130 ceph-mon[98670]: scrub summary: idle+waiting paths [/] 2023-09-12T16:04:25.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:24 smithi130 ceph-mon[98670]: scrub summary: active paths [/] 2023-09-12T16:04:25.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:24 smithi130 ceph-mon[98670]: scrub summary: active paths [/] 2023-09-12T16:04:25.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:24 smithi130 ceph-mon[98670]: scrub summary: idle+waiting paths [/] 2023-09-12T16:04:25.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:24 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:25.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:24 smithi159 ceph-mon[102522]: scrub queued for path: / 2023-09-12T16:04:25.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:24 smithi159 ceph-mon[102522]: scrub summary: idle+waiting paths [/] 2023-09-12T16:04:25.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:24 smithi159 ceph-mon[102522]: scrub summary: active paths [/] 2023-09-12T16:04:25.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:24 smithi159 ceph-mon[102522]: scrub summary: active paths [/] 2023-09-12T16:04:25.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:24 smithi159 ceph-mon[102522]: scrub summary: idle+waiting paths [/] 2023-09-12T16:04:25.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:24 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:04:25.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:24 smithi203 ceph-mon[103431]: scrub queued for path: / 2023-09-12T16:04:25.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:24 smithi203 ceph-mon[103431]: scrub summary: idle+waiting paths [/] 2023-09-12T16:04:25.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:24 smithi203 ceph-mon[103431]: scrub summary: active paths [/] 2023-09-12T16:04:25.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:24 smithi203 ceph-mon[103431]: scrub summary: active paths [/] 2023-09-12T16:04:25.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:24 smithi203 ceph-mon[103431]: scrub summary: idle+waiting paths [/] 2023-09-12T16:04:25.285 INFO:teuthology.orchestra.run.smithi130.stderr:17700000 of 50000000 tuples (35%) done (elapsed 27.65 s, remaining 50.46 s) 2023-09-12T16:04:25.689 INFO:teuthology.orchestra.run.smithi130.stderr:17800000 of 50000000 tuples (35%) done (elapsed 28.06 s, remaining 50.75 s) 2023-09-12T16:04:25.791 INFO:teuthology.orchestra.run.smithi130.stderr:17900000 of 50000000 tuples (35%) done (elapsed 28.16 s, remaining 50.49 s) 2023-09-12T16:04:25.900 INFO:teuthology.orchestra.run.smithi130.stderr:18000000 of 50000000 tuples (36%) done (elapsed 28.27 s, remaining 50.25 s) 2023-09-12T16:04:26.016 INFO:teuthology.orchestra.run.smithi130.stderr:18100000 of 50000000 tuples (36%) done (elapsed 28.38 s, remaining 50.02 s) 2023-09-12T16:04:26.111 INFO:teuthology.orchestra.run.smithi130.stderr:18200000 of 50000000 tuples (36%) done (elapsed 28.48 s, remaining 49.75 s) 2023-09-12T16:04:26.205 INFO:teuthology.orchestra.run.smithi130.stderr:18300000 of 50000000 tuples (36%) done (elapsed 28.57 s, remaining 49.49 s) 2023-09-12T16:04:26.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:25 smithi130 ceph-mon[98670]: scrub summary: active paths [/] 2023-09-12T16:04:26.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:25 smithi130 ceph-mon[98670]: scrub summary: idle+waiting paths [/] 2023-09-12T16:04:26.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:25 smithi130 ceph-mon[98670]: pgmap v337: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 159 MiB/s wr, 208 op/s 2023-09-12T16:04:26.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:25 smithi159 ceph-mon[102522]: scrub summary: active paths [/] 2023-09-12T16:04:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:25 smithi159 ceph-mon[102522]: scrub summary: idle+waiting paths [/] 2023-09-12T16:04:26.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:25 smithi159 ceph-mon[102522]: pgmap v337: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 159 MiB/s wr, 208 op/s 2023-09-12T16:04:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:25 smithi203 ceph-mon[103431]: scrub summary: active paths [/] 2023-09-12T16:04:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:25 smithi203 ceph-mon[103431]: scrub summary: idle+waiting paths [/] 2023-09-12T16:04:26.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:25 smithi203 ceph-mon[103431]: pgmap v337: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 159 MiB/s wr, 208 op/s 2023-09-12T16:04:26.321 INFO:teuthology.orchestra.run.smithi130.stderr:18400000 of 50000000 tuples (36%) done (elapsed 28.69 s, remaining 49.27 s) 2023-09-12T16:04:26.387 INFO:teuthology.orchestra.run.smithi130.stderr:18500000 of 50000000 tuples (37%) done (elapsed 28.75 s, remaining 48.96 s) 2023-09-12T16:04:26.499 INFO:teuthology.orchestra.run.smithi130.stderr:18600000 of 50000000 tuples (37%) done (elapsed 28.86 s, remaining 48.73 s) 2023-09-12T16:04:26.597 INFO:teuthology.orchestra.run.smithi130.stderr:18700000 of 50000000 tuples (37%) done (elapsed 28.96 s, remaining 48.48 s) 2023-09-12T16:04:26.689 INFO:teuthology.orchestra.run.smithi130.stderr:18800000 of 50000000 tuples (37%) done (elapsed 29.05 s, remaining 48.22 s) 2023-09-12T16:04:26.808 INFO:teuthology.orchestra.run.smithi130.stderr:18900000 of 50000000 tuples (37%) done (elapsed 29.17 s, remaining 48.00 s) 2023-09-12T16:04:26.911 INFO:teuthology.orchestra.run.smithi130.stderr:19000000 of 50000000 tuples (38%) done (elapsed 29.28 s, remaining 47.77 s) 2023-09-12T16:04:27.018 INFO:teuthology.orchestra.run.smithi130.stderr:19100000 of 50000000 tuples (38%) done (elapsed 29.38 s, remaining 47.54 s) 2023-09-12T16:04:27.132 INFO:teuthology.orchestra.run.smithi130.stderr:19200000 of 50000000 tuples (38%) done (elapsed 29.50 s, remaining 47.32 s) 2023-09-12T16:04:27.222 INFO:teuthology.orchestra.run.smithi130.stderr:19300000 of 50000000 tuples (38%) done (elapsed 29.59 s, remaining 47.06 s) 2023-09-12T16:04:27.317 INFO:teuthology.orchestra.run.smithi130.stderr:19400000 of 50000000 tuples (38%) done (elapsed 29.68 s, remaining 46.82 s) 2023-09-12T16:04:27.431 INFO:teuthology.orchestra.run.smithi130.stderr:19500000 of 50000000 tuples (39%) done (elapsed 29.80 s, remaining 46.60 s) 2023-09-12T16:04:27.538 INFO:teuthology.orchestra.run.smithi130.stderr:19600000 of 50000000 tuples (39%) done (elapsed 29.90 s, remaining 46.38 s) 2023-09-12T16:04:27.639 INFO:teuthology.orchestra.run.smithi130.stderr:19700000 of 50000000 tuples (39%) done (elapsed 30.00 s, remaining 46.15 s) 2023-09-12T16:04:27.747 INFO:teuthology.orchestra.run.smithi130.stderr:19800000 of 50000000 tuples (39%) done (elapsed 30.11 s, remaining 45.93 s) 2023-09-12T16:04:27.819 INFO:teuthology.orchestra.run.smithi130.stderr:19900000 of 50000000 tuples (39%) done (elapsed 30.18 s, remaining 45.66 s) 2023-09-12T16:04:27.941 INFO:teuthology.orchestra.run.smithi130.stderr:20000000 of 50000000 tuples (40%) done (elapsed 30.31 s, remaining 45.46 s) 2023-09-12T16:04:27.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:27 smithi130 ceph-mon[98670]: pgmap v338: 129 pgs: 129 active+clean; 2.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 153 MiB/s wr, 200 op/s 2023-09-12T16:04:27.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:27 smithi159 ceph-mon[102522]: pgmap v338: 129 pgs: 129 active+clean; 2.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 153 MiB/s wr, 200 op/s 2023-09-12T16:04:27.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:27 smithi203 ceph-mon[103431]: pgmap v338: 129 pgs: 129 active+clean; 2.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 153 MiB/s wr, 200 op/s 2023-09-12T16:04:28.034 INFO:teuthology.orchestra.run.smithi130.stderr:20100000 of 50000000 tuples (40%) done (elapsed 30.40 s, remaining 45.22 s) 2023-09-12T16:04:28.143 INFO:teuthology.orchestra.run.smithi130.stderr:20200000 of 50000000 tuples (40%) done (elapsed 30.51 s, remaining 45.01 s) 2023-09-12T16:04:28.248 INFO:teuthology.orchestra.run.smithi130.stderr:20300000 of 50000000 tuples (40%) done (elapsed 30.61 s, remaining 44.79 s) 2023-09-12T16:04:28.344 INFO:teuthology.orchestra.run.smithi130.stderr:20400000 of 50000000 tuples (40%) done (elapsed 30.71 s, remaining 44.56 s) 2023-09-12T16:04:28.443 INFO:teuthology.orchestra.run.smithi130.stderr:20500000 of 50000000 tuples (41%) done (elapsed 30.81 s, remaining 44.33 s) 2023-09-12T16:04:28.551 INFO:teuthology.orchestra.run.smithi130.stderr:20600000 of 50000000 tuples (41%) done (elapsed 30.92 s, remaining 44.12 s) 2023-09-12T16:04:28.652 INFO:teuthology.orchestra.run.smithi130.stderr:20700000 of 50000000 tuples (41%) done (elapsed 31.02 s, remaining 43.90 s) 2023-09-12T16:04:28.756 INFO:teuthology.orchestra.run.smithi130.stderr:20800000 of 50000000 tuples (41%) done (elapsed 31.12 s, remaining 43.69 s) 2023-09-12T16:04:28.856 INFO:teuthology.orchestra.run.smithi130.stderr:20900000 of 50000000 tuples (41%) done (elapsed 31.22 s, remaining 43.47 s) 2023-09-12T16:04:28.926 INFO:teuthology.orchestra.run.smithi130.stderr:21000000 of 50000000 tuples (42%) done (elapsed 31.29 s, remaining 43.21 s) 2023-09-12T16:04:29.046 INFO:teuthology.orchestra.run.smithi130.stderr:21100000 of 50000000 tuples (42%) done (elapsed 31.41 s, remaining 43.02 s) 2023-09-12T16:04:29.144 INFO:teuthology.orchestra.run.smithi130.stderr:21200000 of 50000000 tuples (42%) done (elapsed 31.51 s, remaining 42.81 s) 2023-09-12T16:04:29.246 INFO:teuthology.orchestra.run.smithi130.stderr:21300000 of 50000000 tuples (42%) done (elapsed 31.61 s, remaining 42.59 s) 2023-09-12T16:04:29.347 INFO:teuthology.orchestra.run.smithi130.stderr:21400000 of 50000000 tuples (42%) done (elapsed 31.71 s, remaining 42.38 s) 2023-09-12T16:04:29.441 INFO:teuthology.orchestra.run.smithi130.stderr:21500000 of 50000000 tuples (43%) done (elapsed 31.81 s, remaining 42.16 s) 2023-09-12T16:04:30.062 INFO:teuthology.orchestra.run.smithi130.stderr:21600000 of 50000000 tuples (43%) done (elapsed 32.43 s, remaining 42.64 s) 2023-09-12T16:04:30.176 INFO:teuthology.orchestra.run.smithi130.stderr:21700000 of 50000000 tuples (43%) done (elapsed 32.54 s, remaining 42.44 s) 2023-09-12T16:04:30.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:29 smithi130 ceph-mon[98670]: pgmap v339: 129 pgs: 129 active+clean; 3.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 151 MiB/s wr, 217 op/s 2023-09-12T16:04:30.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:29 smithi159 ceph-mon[102522]: pgmap v339: 129 pgs: 129 active+clean; 3.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 151 MiB/s wr, 217 op/s 2023-09-12T16:04:30.244 INFO:teuthology.orchestra.run.smithi130.stderr:21800000 of 50000000 tuples (43%) done (elapsed 32.61 s, remaining 42.18 s) 2023-09-12T16:04:30.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:29 smithi203 ceph-mon[103431]: pgmap v339: 129 pgs: 129 active+clean; 3.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 151 MiB/s wr, 217 op/s 2023-09-12T16:04:30.378 INFO:teuthology.orchestra.run.smithi130.stderr:21900000 of 50000000 tuples (43%) done (elapsed 32.74 s, remaining 42.01 s) 2023-09-12T16:04:30.502 INFO:teuthology.orchestra.run.smithi130.stderr:22000000 of 50000000 tuples (44%) done (elapsed 32.87 s, remaining 41.83 s) 2023-09-12T16:04:30.606 INFO:teuthology.orchestra.run.smithi130.stderr:22100000 of 50000000 tuples (44%) done (elapsed 32.97 s, remaining 41.63 s) 2023-09-12T16:04:30.690 INFO:teuthology.orchestra.run.smithi130.stderr:22200000 of 50000000 tuples (44%) done (elapsed 33.06 s, remaining 41.39 s) 2023-09-12T16:04:30.814 INFO:teuthology.orchestra.run.smithi130.stderr:22300000 of 50000000 tuples (44%) done (elapsed 33.18 s, remaining 41.21 s) 2023-09-12T16:04:30.892 INFO:teuthology.orchestra.run.smithi130.stderr:22400000 of 50000000 tuples (44%) done (elapsed 33.26 s, remaining 40.98 s) 2023-09-12T16:04:31.006 INFO:teuthology.orchestra.run.smithi130.stderr:22500000 of 50000000 tuples (45%) done (elapsed 33.37 s, remaining 40.79 s) 2023-09-12T16:04:31.100 INFO:teuthology.orchestra.run.smithi130.stderr:22600000 of 50000000 tuples (45%) done (elapsed 33.47 s, remaining 40.57 s) 2023-09-12T16:04:31.210 INFO:teuthology.orchestra.run.smithi130.stderr:22700000 of 50000000 tuples (45%) done (elapsed 33.58 s, remaining 40.38 s) 2023-09-12T16:04:31.317 INFO:teuthology.orchestra.run.smithi130.stderr:22800000 of 50000000 tuples (45%) done (elapsed 33.68 s, remaining 40.18 s) 2023-09-12T16:04:31.410 INFO:teuthology.orchestra.run.smithi130.stderr:22900000 of 50000000 tuples (45%) done (elapsed 33.78 s, remaining 39.97 s) 2023-09-12T16:04:31.509 INFO:teuthology.orchestra.run.smithi130.stderr:23000000 of 50000000 tuples (46%) done (elapsed 33.87 s, remaining 39.77 s) 2023-09-12T16:04:31.618 INFO:teuthology.orchestra.run.smithi130.stderr:23100000 of 50000000 tuples (46%) done (elapsed 33.98 s, remaining 39.57 s) 2023-09-12T16:04:31.721 INFO:teuthology.orchestra.run.smithi130.stderr:23200000 of 50000000 tuples (46%) done (elapsed 34.09 s, remaining 39.38 s) 2023-09-12T16:04:31.827 INFO:teuthology.orchestra.run.smithi130.stderr:23300000 of 50000000 tuples (46%) done (elapsed 34.19 s, remaining 39.18 s) 2023-09-12T16:04:31.932 INFO:teuthology.orchestra.run.smithi130.stderr:23400000 of 50000000 tuples (46%) done (elapsed 34.30 s, remaining 38.99 s) 2023-09-12T16:04:31.998 INFO:teuthology.orchestra.run.smithi130.stderr:23500000 of 50000000 tuples (47%) done (elapsed 34.36 s, remaining 38.75 s) 2023-09-12T16:04:32.114 INFO:teuthology.orchestra.run.smithi130.stderr:23600000 of 50000000 tuples (47%) done (elapsed 34.48 s, remaining 38.57 s) 2023-09-12T16:04:32.211 INFO:teuthology.orchestra.run.smithi130.stderr:23700000 of 50000000 tuples (47%) done (elapsed 34.58 s, remaining 38.37 s) 2023-09-12T16:04:32.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:31 smithi130 ceph-mon[98670]: pgmap v340: 129 pgs: 129 active+clean; 3.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 169 MiB/s wr, 194 op/s 2023-09-12T16:04:32.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:31 smithi159 ceph-mon[102522]: pgmap v340: 129 pgs: 129 active+clean; 3.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 169 MiB/s wr, 194 op/s 2023-09-12T16:04:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:31 smithi203 ceph-mon[103431]: pgmap v340: 129 pgs: 129 active+clean; 3.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 169 MiB/s wr, 194 op/s 2023-09-12T16:04:32.317 INFO:teuthology.orchestra.run.smithi130.stderr:23800000 of 50000000 tuples (47%) done (elapsed 34.68 s, remaining 38.18 s) 2023-09-12T16:04:32.421 INFO:teuthology.orchestra.run.smithi130.stderr:23900000 of 50000000 tuples (47%) done (elapsed 34.79 s, remaining 37.99 s) 2023-09-12T16:04:32.526 INFO:teuthology.orchestra.run.smithi130.stderr:24000000 of 50000000 tuples (48%) done (elapsed 34.89 s, remaining 37.80 s) 2023-09-12T16:04:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:24100000 of 50000000 tuples (48%) done (elapsed 34.98 s, remaining 37.59 s) 2023-09-12T16:04:32.735 INFO:teuthology.orchestra.run.smithi130.stderr:24200000 of 50000000 tuples (48%) done (elapsed 35.10 s, remaining 37.42 s) 2023-09-12T16:04:32.831 INFO:teuthology.orchestra.run.smithi130.stderr:24300000 of 50000000 tuples (48%) done (elapsed 35.20 s, remaining 37.22 s) 2023-09-12T16:04:32.943 INFO:teuthology.orchestra.run.smithi130.stderr:24400000 of 50000000 tuples (48%) done (elapsed 35.31 s, remaining 37.05 s) 2023-09-12T16:04:33.049 INFO:teuthology.orchestra.run.smithi130.stderr:24500000 of 50000000 tuples (49%) done (elapsed 35.41 s, remaining 36.86 s) 2023-09-12T16:04:33.116 INFO:teuthology.orchestra.run.smithi130.stderr:24600000 of 50000000 tuples (49%) done (elapsed 35.48 s, remaining 36.64 s) 2023-09-12T16:04:33.225 INFO:teuthology.orchestra.run.smithi130.stderr:24700000 of 50000000 tuples (49%) done (elapsed 35.59 s, remaining 36.46 s) 2023-09-12T16:04:33.327 INFO:teuthology.orchestra.run.smithi130.stderr:24800000 of 50000000 tuples (49%) done (elapsed 35.69 s, remaining 36.27 s) 2023-09-12T16:04:33.614 INFO:teuthology.orchestra.run.smithi130.stderr:24900000 of 50000000 tuples (49%) done (elapsed 35.98 s, remaining 36.27 s) 2023-09-12T16:04:34.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:33 smithi130 ceph-mon[98670]: pgmap v341: 129 pgs: 129 active+clean; 3.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 187 MiB/s wr, 173 op/s 2023-09-12T16:04:34.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:33 smithi159 ceph-mon[102522]: pgmap v341: 129 pgs: 129 active+clean; 3.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 187 MiB/s wr, 173 op/s 2023-09-12T16:04:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:33 smithi203 ceph-mon[103431]: pgmap v341: 129 pgs: 129 active+clean; 3.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 187 MiB/s wr, 173 op/s 2023-09-12T16:04:34.251 INFO:teuthology.orchestra.run.smithi130.stderr:25000000 of 50000000 tuples (50%) done (elapsed 36.62 s, remaining 36.62 s) 2023-09-12T16:04:34.890 INFO:teuthology.orchestra.run.smithi130.stderr:25100000 of 50000000 tuples (50%) done (elapsed 37.26 s, remaining 36.96 s) 2023-09-12T16:04:34.958 INFO:teuthology.orchestra.run.smithi130.stderr:25200000 of 50000000 tuples (50%) done (elapsed 37.32 s, remaining 36.73 s) 2023-09-12T16:04:35.077 INFO:teuthology.orchestra.run.smithi130.stderr:25300000 of 50000000 tuples (50%) done (elapsed 37.44 s, remaining 36.55 s) 2023-09-12T16:04:35.192 INFO:teuthology.orchestra.run.smithi130.stderr:25400000 of 50000000 tuples (50%) done (elapsed 37.56 s, remaining 36.37 s) 2023-09-12T16:04:35.832 INFO:teuthology.orchestra.run.smithi130.stderr:25500000 of 50000000 tuples (51%) done (elapsed 38.20 s, remaining 36.70 s) 2023-09-12T16:04:36.007 INFO:teuthology.orchestra.run.smithi130.stderr:25600000 of 50000000 tuples (51%) done (elapsed 38.37 s, remaining 36.57 s) 2023-09-12T16:04:36.113 INFO:teuthology.orchestra.run.smithi130.stderr:25700000 of 50000000 tuples (51%) done (elapsed 38.48 s, remaining 36.38 s) 2023-09-12T16:04:36.220 INFO:teuthology.orchestra.run.smithi130.stderr:25800000 of 50000000 tuples (51%) done (elapsed 38.59 s, remaining 36.19 s) 2023-09-12T16:04:36.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:35 smithi130 ceph-mon[98670]: pgmap v342: 129 pgs: 129 active+clean; 3.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 173 MiB/s wr, 140 op/s 2023-09-12T16:04:36.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:35 smithi159 ceph-mon[102522]: pgmap v342: 129 pgs: 129 active+clean; 3.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 173 MiB/s wr, 140 op/s 2023-09-12T16:04:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:35 smithi203 ceph-mon[103431]: pgmap v342: 129 pgs: 129 active+clean; 3.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 173 MiB/s wr, 140 op/s 2023-09-12T16:04:36.328 INFO:teuthology.orchestra.run.smithi130.stderr:25900000 of 50000000 tuples (51%) done (elapsed 38.69 s, remaining 36.00 s) 2023-09-12T16:04:36.401 INFO:teuthology.orchestra.run.smithi130.stderr:26000000 of 50000000 tuples (52%) done (elapsed 38.77 s, remaining 35.78 s) 2023-09-12T16:04:36.512 INFO:teuthology.orchestra.run.smithi130.stderr:26100000 of 50000000 tuples (52%) done (elapsed 38.88 s, remaining 35.60 s) 2023-09-12T16:04:36.625 INFO:teuthology.orchestra.run.smithi130.stderr:26200000 of 50000000 tuples (52%) done (elapsed 38.99 s, remaining 35.42 s) 2023-09-12T16:04:36.708 INFO:teuthology.orchestra.run.smithi130.stderr:26300000 of 50000000 tuples (52%) done (elapsed 39.07 s, remaining 35.21 s) 2023-09-12T16:04:36.827 INFO:teuthology.orchestra.run.smithi130.stderr:26400000 of 50000000 tuples (52%) done (elapsed 39.19 s, remaining 35.04 s) 2023-09-12T16:04:36.917 INFO:teuthology.orchestra.run.smithi130.stderr:26500000 of 50000000 tuples (53%) done (elapsed 39.28 s, remaining 34.84 s) 2023-09-12T16:04:37.015 INFO:teuthology.orchestra.run.smithi130.stderr:26600000 of 50000000 tuples (53%) done (elapsed 39.38 s, remaining 34.64 s) 2023-09-12T16:04:37.124 INFO:teuthology.orchestra.run.smithi130.stderr:26700000 of 50000000 tuples (53%) done (elapsed 39.49 s, remaining 34.46 s) 2023-09-12T16:04:37.224 INFO:teuthology.orchestra.run.smithi130.stderr:26800000 of 50000000 tuples (53%) done (elapsed 39.59 s, remaining 34.27 s) 2023-09-12T16:04:37.323 INFO:teuthology.orchestra.run.smithi130.stderr:26900000 of 50000000 tuples (53%) done (elapsed 39.69 s, remaining 34.08 s) 2023-09-12T16:04:37.433 INFO:teuthology.orchestra.run.smithi130.stderr:27000000 of 50000000 tuples (54%) done (elapsed 39.80 s, remaining 33.90 s) 2023-09-12T16:04:37.499 INFO:teuthology.orchestra.run.smithi130.stderr:27100000 of 50000000 tuples (54%) done (elapsed 39.86 s, remaining 33.69 s) 2023-09-12T16:04:37.611 INFO:teuthology.orchestra.run.smithi130.stderr:27200000 of 50000000 tuples (54%) done (elapsed 39.98 s, remaining 33.51 s) 2023-09-12T16:04:37.718 INFO:teuthology.orchestra.run.smithi130.stderr:27300000 of 50000000 tuples (54%) done (elapsed 40.08 s, remaining 33.33 s) 2023-09-12T16:04:37.815 INFO:teuthology.orchestra.run.smithi130.stderr:27400000 of 50000000 tuples (54%) done (elapsed 40.18 s, remaining 33.14 s) 2023-09-12T16:04:37.927 INFO:teuthology.orchestra.run.smithi130.stderr:27500000 of 50000000 tuples (55%) done (elapsed 40.29 s, remaining 32.97 s) 2023-09-12T16:04:37.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:37 smithi130 ceph-mon[98670]: pgmap v343: 129 pgs: 129 active+clean; 3.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 177 MiB/s wr, 155 op/s 2023-09-12T16:04:37.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:37 smithi159 ceph-mon[102522]: pgmap v343: 129 pgs: 129 active+clean; 3.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 177 MiB/s wr, 155 op/s 2023-09-12T16:04:37.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:37 smithi203 ceph-mon[103431]: pgmap v343: 129 pgs: 129 active+clean; 3.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 177 MiB/s wr, 155 op/s 2023-09-12T16:04:38.023 INFO:teuthology.orchestra.run.smithi130.stderr:27600000 of 50000000 tuples (55%) done (elapsed 40.39 s, remaining 32.78 s) 2023-09-12T16:04:38.445 INFO:teuthology.orchestra.run.smithi130.stderr:27700000 of 50000000 tuples (55%) done (elapsed 40.81 s, remaining 32.85 s) 2023-09-12T16:04:38.888 INFO:teuthology.orchestra.run.smithi130.stderr:27800000 of 50000000 tuples (55%) done (elapsed 41.25 s, remaining 32.94 s) 2023-09-12T16:04:38.990 INFO:teuthology.orchestra.run.smithi130.stderr:27900000 of 50000000 tuples (55%) done (elapsed 41.36 s, remaining 32.76 s) 2023-09-12T16:04:39.090 INFO:teuthology.orchestra.run.smithi130.stderr:28000000 of 50000000 tuples (56%) done (elapsed 41.46 s, remaining 32.57 s) 2023-09-12T16:04:39.197 INFO:teuthology.orchestra.run.smithi130.stderr:28100000 of 50000000 tuples (56%) done (elapsed 41.56 s, remaining 32.39 s) 2023-09-12T16:04:39.740 INFO:teuthology.orchestra.run.smithi130.stderr:28200000 of 50000000 tuples (56%) done (elapsed 42.11 s, remaining 32.55 s) 2023-09-12T16:04:39.995 INFO:teuthology.orchestra.run.smithi130.stderr:28300000 of 50000000 tuples (56%) done (elapsed 42.36 s, remaining 32.48 s) 2023-09-12T16:04:40.095 INFO:teuthology.orchestra.run.smithi130.stderr:28400000 of 50000000 tuples (56%) done (elapsed 42.46 s, remaining 32.29 s) 2023-09-12T16:04:40.199 INFO:teuthology.orchestra.run.smithi130.stderr:28500000 of 50000000 tuples (57%) done (elapsed 42.56 s, remaining 32.11 s) 2023-09-12T16:04:40.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:39 smithi130 ceph-mon[98670]: pgmap v344: 129 pgs: 129 active+clean; 4.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 175 MiB/s wr, 174 op/s 2023-09-12T16:04:40.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:39 smithi159 ceph-mon[102522]: pgmap v344: 129 pgs: 129 active+clean; 4.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 175 MiB/s wr, 174 op/s 2023-09-12T16:04:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:39 smithi203 ceph-mon[103431]: pgmap v344: 129 pgs: 129 active+clean; 4.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 175 MiB/s wr, 174 op/s 2023-09-12T16:04:40.311 INFO:teuthology.orchestra.run.smithi130.stderr:28600000 of 50000000 tuples (57%) done (elapsed 42.68 s, remaining 31.93 s) 2023-09-12T16:04:40.413 INFO:teuthology.orchestra.run.smithi130.stderr:28700000 of 50000000 tuples (57%) done (elapsed 42.78 s, remaining 31.75 s) 2023-09-12T16:04:40.496 INFO:teuthology.orchestra.run.smithi130.stderr:28800000 of 50000000 tuples (57%) done (elapsed 42.86 s, remaining 31.55 s) 2023-09-12T16:04:40.617 INFO:teuthology.orchestra.run.smithi130.stderr:28900000 of 50000000 tuples (57%) done (elapsed 42.98 s, remaining 31.38 s) 2023-09-12T16:04:40.715 INFO:teuthology.orchestra.run.smithi130.stderr:29000000 of 50000000 tuples (58%) done (elapsed 43.08 s, remaining 31.20 s) 2023-09-12T16:04:40.801 INFO:teuthology.orchestra.run.smithi130.stderr:29100000 of 50000000 tuples (58%) done (elapsed 43.17 s, remaining 31.00 s) 2023-09-12T16:04:40.923 INFO:teuthology.orchestra.run.smithi130.stderr:29200000 of 50000000 tuples (58%) done (elapsed 43.29 s, remaining 30.84 s) 2023-09-12T16:04:41.453 INFO:teuthology.orchestra.run.smithi130.stderr:29300000 of 50000000 tuples (58%) done (elapsed 43.82 s, remaining 30.96 s) 2023-09-12T16:04:41.682 INFO:teuthology.orchestra.run.smithi130.stderr:29400000 of 50000000 tuples (58%) done (elapsed 44.05 s, remaining 30.86 s) 2023-09-12T16:04:41.788 INFO:teuthology.orchestra.run.smithi130.stderr:29500000 of 50000000 tuples (59%) done (elapsed 44.15 s, remaining 30.68 s) 2023-09-12T16:04:41.898 INFO:teuthology.orchestra.run.smithi130.stderr:29600000 of 50000000 tuples (59%) done (elapsed 44.26 s, remaining 30.51 s) 2023-09-12T16:04:41.997 INFO:teuthology.orchestra.run.smithi130.stderr:29700000 of 50000000 tuples (59%) done (elapsed 44.36 s, remaining 30.32 s) 2023-09-12T16:04:42.106 INFO:teuthology.orchestra.run.smithi130.stderr:29800000 of 50000000 tuples (59%) done (elapsed 44.47 s, remaining 30.15 s) 2023-09-12T16:04:42.189 INFO:teuthology.orchestra.run.smithi130.stderr:29900000 of 50000000 tuples (59%) done (elapsed 44.55 s, remaining 29.95 s) 2023-09-12T16:04:42.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:41 smithi130 ceph-mon[98670]: pgmap v345: 129 pgs: 129 active+clean; 4.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 178 MiB/s wr, 160 op/s 2023-09-12T16:04:42.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:41 smithi159 ceph-mon[102522]: pgmap v345: 129 pgs: 129 active+clean; 4.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 178 MiB/s wr, 160 op/s 2023-09-12T16:04:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:41 smithi203 ceph-mon[103431]: pgmap v345: 129 pgs: 129 active+clean; 4.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 178 MiB/s wr, 160 op/s 2023-09-12T16:04:42.313 INFO:teuthology.orchestra.run.smithi130.stderr:30000000 of 50000000 tuples (60%) done (elapsed 44.68 s, remaining 29.79 s) 2023-09-12T16:04:42.415 INFO:teuthology.orchestra.run.smithi130.stderr:30100000 of 50000000 tuples (60%) done (elapsed 44.78 s, remaining 29.61 s) 2023-09-12T16:04:42.520 INFO:teuthology.orchestra.run.smithi130.stderr:30200000 of 50000000 tuples (60%) done (elapsed 44.89 s, remaining 29.43 s) 2023-09-12T16:04:42.626 INFO:teuthology.orchestra.run.smithi130.stderr:30300000 of 50000000 tuples (60%) done (elapsed 44.99 s, remaining 29.25 s) 2023-09-12T16:04:42.722 INFO:teuthology.orchestra.run.smithi130.stderr:30400000 of 50000000 tuples (60%) done (elapsed 45.09 s, remaining 29.07 s) 2023-09-12T16:04:42.812 INFO:teuthology.orchestra.run.smithi130.stderr:30500000 of 50000000 tuples (61%) done (elapsed 45.18 s, remaining 28.88 s) 2023-09-12T16:04:42.912 INFO:teuthology.orchestra.run.smithi130.stderr:30600000 of 50000000 tuples (61%) done (elapsed 45.28 s, remaining 28.71 s) 2023-09-12T16:04:43.027 INFO:teuthology.orchestra.run.smithi130.stderr:30700000 of 50000000 tuples (61%) done (elapsed 45.39 s, remaining 28.54 s) 2023-09-12T16:04:43.253 INFO:teuthology.orchestra.run.smithi130.stderr:30800000 of 50000000 tuples (61%) done (elapsed 45.62 s, remaining 28.44 s) 2023-09-12T16:04:43.864 INFO:teuthology.orchestra.run.smithi130.stderr:30900000 of 50000000 tuples (61%) done (elapsed 46.23 s, remaining 28.58 s) 2023-09-12T16:04:43.972 INFO:teuthology.orchestra.run.smithi130.stderr:31000000 of 50000000 tuples (62%) done (elapsed 46.34 s, remaining 28.40 s) 2023-09-12T16:04:44.052 INFO:teuthology.orchestra.run.smithi130.stderr:31100000 of 50000000 tuples (62%) done (elapsed 46.42 s, remaining 28.21 s) 2023-09-12T16:04:44.176 INFO:teuthology.orchestra.run.smithi130.stderr:31200000 of 50000000 tuples (62%) done (elapsed 46.54 s, remaining 28.04 s) 2023-09-12T16:04:44.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:43 smithi130 ceph-mon[98670]: pgmap v346: 129 pgs: 129 active+clean; 4.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 173 MiB/s wr, 194 op/s 2023-09-12T16:04:44.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:43 smithi159 ceph-mon[102522]: pgmap v346: 129 pgs: 129 active+clean; 4.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 173 MiB/s wr, 194 op/s 2023-09-12T16:04:44.242 INFO:teuthology.orchestra.run.smithi130.stderr:31300000 of 50000000 tuples (62%) done (elapsed 46.61 s, remaining 27.85 s) 2023-09-12T16:04:44.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:43 smithi203 ceph-mon[103431]: pgmap v346: 129 pgs: 129 active+clean; 4.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 173 MiB/s wr, 194 op/s 2023-09-12T16:04:44.359 INFO:teuthology.orchestra.run.smithi130.stderr:31400000 of 50000000 tuples (62%) done (elapsed 46.72 s, remaining 27.68 s) 2023-09-12T16:04:44.458 INFO:teuthology.orchestra.run.smithi130.stderr:31500000 of 50000000 tuples (63%) done (elapsed 46.82 s, remaining 27.50 s) 2023-09-12T16:04:44.564 INFO:teuthology.orchestra.run.smithi130.stderr:31600000 of 50000000 tuples (63%) done (elapsed 46.93 s, remaining 27.33 s) 2023-09-12T16:04:44.667 INFO:teuthology.orchestra.run.smithi130.stderr:31700000 of 50000000 tuples (63%) done (elapsed 47.03 s, remaining 27.15 s) 2023-09-12T16:04:44.764 INFO:teuthology.orchestra.run.smithi130.stderr:31800000 of 50000000 tuples (63%) done (elapsed 47.13 s, remaining 26.97 s) 2023-09-12T16:04:44.853 INFO:teuthology.orchestra.run.smithi130.stderr:31900000 of 50000000 tuples (63%) done (elapsed 47.22 s, remaining 26.79 s) 2023-09-12T16:04:44.974 INFO:teuthology.orchestra.run.smithi130.stderr:32000000 of 50000000 tuples (64%) done (elapsed 47.34 s, remaining 26.63 s) 2023-09-12T16:04:45.077 INFO:teuthology.orchestra.run.smithi130.stderr:32100000 of 50000000 tuples (64%) done (elapsed 47.44 s, remaining 26.45 s) 2023-09-12T16:04:45.157 INFO:teuthology.orchestra.run.smithi130.stderr:32200000 of 50000000 tuples (64%) done (elapsed 47.52 s, remaining 26.27 s) 2023-09-12T16:04:45.258 INFO:teuthology.orchestra.run.smithi130.stderr:32300000 of 50000000 tuples (64%) done (elapsed 47.62 s, remaining 26.10 s) 2023-09-12T16:04:45.662 INFO:teuthology.orchestra.run.smithi130.stderr:32400000 of 50000000 tuples (64%) done (elapsed 48.03 s, remaining 26.09 s) 2023-09-12T16:04:46.108 INFO:teuthology.orchestra.run.smithi130.stderr:32500000 of 50000000 tuples (65%) done (elapsed 48.47 s, remaining 26.10 s) 2023-09-12T16:04:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:45 smithi130 ceph-mon[98670]: pgmap v347: 129 pgs: 129 active+clean; 4.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 154 MiB/s wr, 192 op/s 2023-09-12T16:04:46.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:45 smithi159 ceph-mon[102522]: pgmap v347: 129 pgs: 129 active+clean; 4.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 154 MiB/s wr, 192 op/s 2023-09-12T16:04:46.233 INFO:teuthology.orchestra.run.smithi130.stderr:32600000 of 50000000 tuples (65%) done (elapsed 48.60 s, remaining 25.94 s) 2023-09-12T16:04:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:45 smithi203 ceph-mon[103431]: pgmap v347: 129 pgs: 129 active+clean; 4.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 154 MiB/s wr, 192 op/s 2023-09-12T16:04:46.335 INFO:teuthology.orchestra.run.smithi130.stderr:32700000 of 50000000 tuples (65%) done (elapsed 48.70 s, remaining 25.77 s) 2023-09-12T16:04:46.439 INFO:teuthology.orchestra.run.smithi130.stderr:32800000 of 50000000 tuples (65%) done (elapsed 48.80 s, remaining 25.59 s) 2023-09-12T16:04:46.537 INFO:teuthology.orchestra.run.smithi130.stderr:32900000 of 50000000 tuples (65%) done (elapsed 48.90 s, remaining 25.42 s) 2023-09-12T16:04:46.624 INFO:teuthology.orchestra.run.smithi130.stderr:33000000 of 50000000 tuples (66%) done (elapsed 48.99 s, remaining 25.24 s) 2023-09-12T16:04:46.740 INFO:teuthology.orchestra.run.smithi130.stderr:33100000 of 50000000 tuples (66%) done (elapsed 49.11 s, remaining 25.07 s) 2023-09-12T16:04:47.129 INFO:teuthology.orchestra.run.smithi130.stderr:33200000 of 50000000 tuples (66%) done (elapsed 49.49 s, remaining 25.05 s) 2023-09-12T16:04:47.417 INFO:teuthology.orchestra.run.smithi130.stderr:33300000 of 50000000 tuples (66%) done (elapsed 49.78 s, remaining 24.97 s) 2023-09-12T16:04:47.516 INFO:teuthology.orchestra.run.smithi130.stderr:33400000 of 50000000 tuples (66%) done (elapsed 49.88 s, remaining 24.79 s) 2023-09-12T16:04:47.606 INFO:teuthology.orchestra.run.smithi130.stderr:33500000 of 50000000 tuples (67%) done (elapsed 49.97 s, remaining 24.61 s) 2023-09-12T16:04:47.728 INFO:teuthology.orchestra.run.smithi130.stderr:33600000 of 50000000 tuples (67%) done (elapsed 50.09 s, remaining 24.45 s) 2023-09-12T16:04:47.837 INFO:teuthology.orchestra.run.smithi130.stderr:33700000 of 50000000 tuples (67%) done (elapsed 50.20 s, remaining 24.28 s) 2023-09-12T16:04:47.937 INFO:teuthology.orchestra.run.smithi130.stderr:33800000 of 50000000 tuples (67%) done (elapsed 50.30 s, remaining 24.11 s) 2023-09-12T16:04:47.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:47 smithi130 ceph-mon[98670]: pgmap v348: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 150 MiB/s wr, 204 op/s 2023-09-12T16:04:47.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:47 smithi159 ceph-mon[102522]: pgmap v348: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 150 MiB/s wr, 204 op/s 2023-09-12T16:04:47.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:47 smithi203 ceph-mon[103431]: pgmap v348: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 150 MiB/s wr, 204 op/s 2023-09-12T16:04:48.053 INFO:teuthology.orchestra.run.smithi130.stderr:33900000 of 50000000 tuples (67%) done (elapsed 50.42 s, remaining 23.95 s) 2023-09-12T16:04:48.163 INFO:teuthology.orchestra.run.smithi130.stderr:34000000 of 50000000 tuples (68%) done (elapsed 50.53 s, remaining 23.78 s) 2023-09-12T16:04:48.254 INFO:teuthology.orchestra.run.smithi130.stderr:34100000 of 50000000 tuples (68%) done (elapsed 50.62 s, remaining 23.60 s) 2023-09-12T16:04:48.369 INFO:teuthology.orchestra.run.smithi130.stderr:34200000 of 50000000 tuples (68%) done (elapsed 50.73 s, remaining 23.44 s) 2023-09-12T16:04:48.470 INFO:teuthology.orchestra.run.smithi130.stderr:34300000 of 50000000 tuples (68%) done (elapsed 50.84 s, remaining 23.27 s) 2023-09-12T16:04:48.552 INFO:teuthology.orchestra.run.smithi130.stderr:34400000 of 50000000 tuples (68%) done (elapsed 50.92 s, remaining 23.09 s) 2023-09-12T16:04:48.645 INFO:teuthology.orchestra.run.smithi130.stderr:34500000 of 50000000 tuples (69%) done (elapsed 51.01 s, remaining 22.92 s) 2023-09-12T16:04:48.766 INFO:teuthology.orchestra.run.smithi130.stderr:34600000 of 50000000 tuples (69%) done (elapsed 51.13 s, remaining 22.76 s) 2023-09-12T16:04:48.846 INFO:teuthology.orchestra.run.smithi130.stderr:34700000 of 50000000 tuples (69%) done (elapsed 51.21 s, remaining 22.58 s) 2023-09-12T16:04:48.944 INFO:teuthology.orchestra.run.smithi130.stderr:34800000 of 50000000 tuples (69%) done (elapsed 51.31 s, remaining 22.41 s) 2023-09-12T16:04:49.050 INFO:teuthology.orchestra.run.smithi130.stderr:34900000 of 50000000 tuples (69%) done (elapsed 51.42 s, remaining 22.25 s) 2023-09-12T16:04:49.240 INFO:teuthology.orchestra.run.smithi130.stderr:35000000 of 50000000 tuples (70%) done (elapsed 51.61 s, remaining 22.12 s) 2023-09-12T16:04:49.875 INFO:teuthology.orchestra.run.smithi130.stderr:35100000 of 50000000 tuples (70%) done (elapsed 52.24 s, remaining 22.18 s) 2023-09-12T16:04:49.944 INFO:teuthology.orchestra.run.smithi130.stderr:35200000 of 50000000 tuples (70%) done (elapsed 52.31 s, remaining 21.99 s) 2023-09-12T16:04:50.063 INFO:teuthology.orchestra.run.smithi130.stderr:35300000 of 50000000 tuples (70%) done (elapsed 52.43 s, remaining 21.83 s) 2023-09-12T16:04:50.145 INFO:teuthology.orchestra.run.smithi130.stderr:35400000 of 50000000 tuples (70%) done (elapsed 52.51 s, remaining 21.66 s) 2023-09-12T16:04:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:49 smithi130 ceph-mon[98670]: pgmap v349: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 137 MiB/s wr, 197 op/s 2023-09-12T16:04:50.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:49 smithi159 ceph-mon[102522]: pgmap v349: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 137 MiB/s wr, 197 op/s 2023-09-12T16:04:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:49 smithi203 ceph-mon[103431]: pgmap v349: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 137 MiB/s wr, 197 op/s 2023-09-12T16:04:50.247 INFO:teuthology.orchestra.run.smithi130.stderr:35500000 of 50000000 tuples (71%) done (elapsed 52.61 s, remaining 21.49 s) 2023-09-12T16:04:50.345 INFO:teuthology.orchestra.run.smithi130.stderr:35600000 of 50000000 tuples (71%) done (elapsed 52.71 s, remaining 21.32 s) 2023-09-12T16:04:50.963 INFO:teuthology.orchestra.run.smithi130.stderr:35700000 of 50000000 tuples (71%) done (elapsed 53.33 s, remaining 21.36 s) 2023-09-12T16:04:51.227 INFO:teuthology.orchestra.run.smithi130.stderr:35800000 of 50000000 tuples (71%) done (elapsed 53.59 s, remaining 21.26 s) 2023-09-12T16:04:51.327 INFO:teuthology.orchestra.run.smithi130.stderr:35900000 of 50000000 tuples (71%) done (elapsed 53.69 s, remaining 21.09 s) 2023-09-12T16:04:51.827 INFO:teuthology.orchestra.run.smithi130.stderr:36000000 of 50000000 tuples (72%) done (elapsed 54.19 s, remaining 21.07 s) 2023-09-12T16:04:52.099 INFO:teuthology.orchestra.run.smithi130.stderr:36100000 of 50000000 tuples (72%) done (elapsed 54.47 s, remaining 20.97 s) 2023-09-12T16:04:52.197 INFO:teuthology.orchestra.run.smithi130.stderr:36200000 of 50000000 tuples (72%) done (elapsed 54.56 s, remaining 20.80 s) 2023-09-12T16:04:52.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:51 smithi130 ceph-mon[98670]: pgmap v350: 129 pgs: 129 active+clean; 4.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 151 MiB/s wr, 177 op/s 2023-09-12T16:04:52.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:51 smithi159 ceph-mon[102522]: pgmap v350: 129 pgs: 129 active+clean; 4.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 151 MiB/s wr, 177 op/s 2023-09-12T16:04:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:51 smithi203 ceph-mon[103431]: pgmap v350: 129 pgs: 129 active+clean; 4.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 151 MiB/s wr, 177 op/s 2023-09-12T16:04:52.293 INFO:teuthology.orchestra.run.smithi130.stderr:36300000 of 50000000 tuples (72%) done (elapsed 54.66 s, remaining 20.63 s) 2023-09-12T16:04:52.401 INFO:teuthology.orchestra.run.smithi130.stderr:36400000 of 50000000 tuples (72%) done (elapsed 54.77 s, remaining 20.46 s) 2023-09-12T16:04:52.500 INFO:teuthology.orchestra.run.smithi130.stderr:36500000 of 50000000 tuples (73%) done (elapsed 54.87 s, remaining 20.29 s) 2023-09-12T16:04:52.578 INFO:teuthology.orchestra.run.smithi130.stderr:36600000 of 50000000 tuples (73%) done (elapsed 54.94 s, remaining 20.12 s) 2023-09-12T16:04:52.716 INFO:teuthology.orchestra.run.smithi130.stderr:36700000 of 50000000 tuples (73%) done (elapsed 55.08 s, remaining 19.96 s) 2023-09-12T16:04:52.781 INFO:teuthology.orchestra.run.smithi130.stderr:36800000 of 50000000 tuples (73%) done (elapsed 55.15 s, remaining 19.78 s) 2023-09-12T16:04:52.883 INFO:teuthology.orchestra.run.smithi130.stderr:36900000 of 50000000 tuples (73%) done (elapsed 55.25 s, remaining 19.61 s) 2023-09-12T16:04:52.983 INFO:teuthology.orchestra.run.smithi130.stderr:37000000 of 50000000 tuples (74%) done (elapsed 55.35 s, remaining 19.45 s) 2023-09-12T16:04:53.649 INFO:teuthology.orchestra.run.smithi130.stderr:37100000 of 50000000 tuples (74%) done (elapsed 56.01 s, remaining 19.48 s) 2023-09-12T16:04:53.749 INFO:teuthology.orchestra.run.smithi130.stderr:37200000 of 50000000 tuples (74%) done (elapsed 56.11 s, remaining 19.31 s) 2023-09-12T16:04:53.854 INFO:teuthology.orchestra.run.smithi130.stderr:37300000 of 50000000 tuples (74%) done (elapsed 56.22 s, remaining 19.14 s) 2023-09-12T16:04:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:53 smithi130 ceph-mon[98670]: pgmap v351: 129 pgs: 129 active+clean; 5.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 154 MiB/s wr, 210 op/s 2023-09-12T16:04:54.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:53 smithi159 ceph-mon[102522]: pgmap v351: 129 pgs: 129 active+clean; 5.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 154 MiB/s wr, 210 op/s 2023-09-12T16:04:54.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:53 smithi203 ceph-mon[103431]: pgmap v351: 129 pgs: 129 active+clean; 5.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 154 MiB/s wr, 210 op/s 2023-09-12T16:04:54.269 INFO:teuthology.orchestra.run.smithi130.stderr:37400000 of 50000000 tuples (74%) done (elapsed 56.64 s, remaining 19.08 s) 2023-09-12T16:04:54.492 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:04:54.624 INFO:teuthology.orchestra.run.smithi130.stderr:37500000 of 50000000 tuples (75%) done (elapsed 56.99 s, remaining 19.00 s) 2023-09-12T16:04:54.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 -- 172.21.15.130:0/3778024590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f941039a0 msgr2=0x7f1f94105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:54.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/3778024590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f941039a0 0x7f1f94105d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1f800098f0 tx=0x7f1f80009c00 comp rx=0 tx=0).stop 2023-09-12T16:04:54.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 -- 172.21.15.130:0/3778024590 shutdown_connections 2023-09-12T16:04:54.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/3778024590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f941062d0 0x7f1f94108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/3778024590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f94101070 0x7f1f94103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/3778024590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f941039a0 0x7f1f94105d90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 -- 172.21.15.130:0/3778024590 >> 172.21.15.130:0/3778024590 conn(0x7f1f940faa60 msgr2=0x7f1f940fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:54.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 -- 172.21.15.130:0/3778024590 shutdown_connections 2023-09-12T16:04:54.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 -- 172.21.15.130:0/3778024590 wait complete. 2023-09-12T16:04:54.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 Processor -- start 2023-09-12T16:04:54.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.630+0000 7f1f9b17c700 1 -- start start 2023-09-12T16:04:54.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f9b17c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94101070 0x7f1f94196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f9b17c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f941039a0 0x7f1f94196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.642 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f9b17c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f941062d0 0x7f1f9419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f9b17c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9419de80 con 0x7f1f941062d0 2023-09-12T16:04:54.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f9b17c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9419dfc0 con 0x7f1f94101070 2023-09-12T16:04:54.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f9b17c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9419e150 con 0x7f1f941039a0 2023-09-12T16:04:54.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f98f18700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94101070 0x7f1f94196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f99719700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f941062d0 0x7f1f9419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f98f18700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94101070 0x7f1f94196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:38948/0 (socket says 172.21.15.130:38948) 2023-09-12T16:04:54.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f99719700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f941062d0 0x7f1f9419d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55694/0 (socket says 172.21.15.130:55694) 2023-09-12T16:04:54.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f93fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f941039a0 0x7f1f94196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.644 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f98f18700 1 -- 172.21.15.130:0/1117645933 learned_addr learned my addr 172.21.15.130:0/1117645933 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:54.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f93fff700 1 -- 172.21.15.130:0/1117645933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94101070 msgr2=0x7f1f94196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:54.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f93fff700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94101070 0x7f1f94196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f93fff700 1 -- 172.21.15.130:0/1117645933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f941062d0 msgr2=0x7f1f9419d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:54.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f93fff700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f941062d0 0x7f1f9419d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.645 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f93fff700 1 -- 172.21.15.130:0/1117645933 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880095f0 con 0x7f1f941039a0 2023-09-12T16:04:54.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f98f18700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94101070 0x7f1f94196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:54.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f93fff700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f941039a0 0x7f1f94196840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000c00 tx=0x7f1f8000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:54.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f91ffb700 1 -- 172.21.15.130:0/1117645933 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8001c070 con 0x7f1f941039a0 2023-09-12T16:04:54.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f91ffb700 1 -- 172.21.15.130:0/1117645933 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f8000f7d0 con 0x7f1f941039a0 2023-09-12T16:04:54.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f9b17c700 1 -- 172.21.15.130:0/1117645933 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f80009520 con 0x7f1f941039a0 2023-09-12T16:04:54.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.631+0000 7f1f91ffb700 1 -- 172.21.15.130:0/1117645933 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80003990 con 0x7f1f941039a0 2023-09-12T16:04:54.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.632+0000 7f1f9b17c700 1 -- 172.21.15.130:0/1117645933 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9419e660 con 0x7f1f941039a0 2023-09-12T16:04:54.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.634+0000 7f1f91ffb700 1 -- 172.21.15.130:0/1117645933 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f1f80020030 con 0x7f1f941039a0 2023-09-12T16:04:54.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1f91ffb700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f840a0640 0x7f1f840a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1f91ffb700 1 -- 172.21.15.130:0/1117645933 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1f800cce90 con 0x7f1f941039a0 2023-09-12T16:04:54.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1f98f18700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f840a0640 0x7f1f840a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1fa7342b80 1 Processor -- start 2023-09-12T16:04:54.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1fa7342b80 1 -- start start 2023-09-12T16:04:54.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8baff6730 con 0x55a8bb0a39d0 2023-09-12T16:04:54.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8baff4050 con 0x55a8bb0a3070 2023-09-12T16:04:54.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bafef940 con 0x55a8bb0a2230 2023-09-12T16:04:54.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1f98f18700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.635+0000 7f1f93fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f98f18700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:38956/0 (socket says 172.21.15.130:38956) 2023-09-12T16:04:54.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f98f18700 1 -- 172.21.15.130:0/867180731 learned_addr learned my addr 172.21.15.130:0/867180731 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:54.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f99719700 1 --2- 172.21.15.130:0/867180731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f98f18700 1 -- 172.21.15.130:0/867180731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 msgr2=0x55a8bb0a26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:54.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f98f18700 1 --2- 172.21.15.130:0/867180731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0a26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f98f18700 1 -- 172.21.15.130:0/867180731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 msgr2=0x55a8bb0a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:54.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f98f18700 1 --2- 172.21.15.130:0/867180731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f98f18700 1 -- 172.21.15.130:0/867180731 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bae1bde0 con 0x55a8bb0a3070 2023-09-12T16:04:54.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f98f18700 1 --2- 172.21.15.130:0/867180731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a3490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800fe80 tx=0x7f1f88013740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d79391ced45af73 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:54.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f7effd700 1 -- 172.21.15.130:0/867180731 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880049c0 con 0x55a8bb0a3070 2023-09-12T16:04:54.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f98f18700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f840a0640 0x7f1f840a2b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f88022000 tx=0x7f1f88016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:54.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f7effd700 1 -- 172.21.15.130:0/867180731 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f88013e60 con 0x55a8bb0a3070 2023-09-12T16:04:54.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.636+0000 7f1f7effd700 1 -- 172.21.15.130:0/867180731 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88020850 con 0x55a8bb0a3070 2023-09-12T16:04:54.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 -- 172.21.15.130:0/867180731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 msgr2=0x55a8bb0a3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:54.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/867180731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a3490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800fe80 tx=0x7f1f88013740 comp rx=0 tx=0).stop 2023-09-12T16:04:54.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 -- 172.21.15.130:0/867180731 shutdown_connections 2023-09-12T16:04:54.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/867180731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0a26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/867180731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a3490 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/867180731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 -- 172.21.15.130:0/867180731 >> 172.21.15.130:0/867180731 conn(0x55a8bb012090 msgr2=0x55a8bb098720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:54.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 -- 172.21.15.130:0/867180731 shutdown_connections 2023-09-12T16:04:54.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 -- 172.21.15.130:0/867180731 wait complete. 2023-09-12T16:04:54.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 -- start start 2023-09-12T16:04:54.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.637+0000 7f1fa7342b80 1 Processor -- start 2023-09-12T16:04:54.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a2230 0x55a8bb0aba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a3070 0x55a8bb0abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8baf8db70 con 0x55a8bb0a2230 2023-09-12T16:04:54.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8baf8d870 con 0x55a8bb0a39d0 2023-09-12T16:04:54.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bae71920 con 0x55a8bb0a3070 2023-09-12T16:04:54.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1fa7342b80 10 client.0 _ll_register_callbacks cb 0x55a8baed7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:54.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f98f18700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a2230 0x55a8bb0aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f99719700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f98f18700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a2230 0x55a8bb0aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55704/0 (socket says 172.21.15.130:55704) 2023-09-12T16:04:54.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f93fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a3070 0x55a8bb0abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f98f18700 1 -- 172.21.15.130:0/1773199272 learned_addr learned my addr 172.21.15.130:0/1773199272 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:54.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f98f18700 1 -- 172.21.15.130:0/1773199272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a3070 msgr2=0x55a8bb0abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:54.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f98f18700 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a3070 0x55a8bb0abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f98f18700 1 -- 172.21.15.130:0/1773199272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 msgr2=0x55a8bb0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:54.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f98f18700 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f98f18700 1 -- 172.21.15.130:0/1773199272 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8800fab0 con 0x55a8bb0a2230 2023-09-12T16:04:54.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f99719700 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:54.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f98f18700 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a2230 0x55a8bb0aba60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1f88010990 tx=0x7f1f880109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d09b0ceba481ccf server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:54.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.638+0000 7f1f73fff700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:04:54.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1f73fff700 1 -- 172.21.15.130:0/1773199272 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88004220 con 0x55a8bb0a2230 2023-09-12T16:04:54.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1f73fff700 1 -- 172.21.15.130:0/1773199272 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1f88004380 con 0x55a8bb0a2230 2023-09-12T16:04:54.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1fa7342b80 1 -- 172.21.15.130:0/1773199272 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8baf474f0 con 0x55a8bb0a2230 2023-09-12T16:04:54.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1f73fff700 1 -- 172.21.15.130:0/1773199272 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f88004540 con 0x55a8bb0a2230 2023-09-12T16:04:54.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1fa7342b80 10 client.15882 fetch_fsmap learned FSMap version 28 2023-09-12T16:04:54.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1fa7342b80 1 -- 172.21.15.130:0/1773199272 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a8bae04c60 con 0x55a8bb0a2230 2023-09-12T16:04:54.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1f73fff700 1 -- 172.21.15.130:0/1773199272 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8baf474f0 con 0x55a8bb0a2230 2023-09-12T16:04:54.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1f73fff700 1 -- 172.21.15.130:0/1773199272 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1f88004950 con 0x55a8bb0a2230 2023-09-12T16:04:54.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1fa7342b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:04:54.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1fa7342b80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:54.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1fa7342b80 20 client.15882 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:54.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a8bb0b77d0 0x55a8bb0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1fa7342b80 4 client.15882 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:04:54.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1fa7342b80 1 -- 172.21.15.130:0/1773199272 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8bb0048a0 con 0x55a8bb0b77d0 2023-09-12T16:04:54.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.639+0000 7f1f93fff700 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a8bb0b77d0 0x55a8bb0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1f93fff700 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a8bb0b77d0 0x55a8bb0b9bf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a0b7564c47fe47 server_cookie=ad7f2245b54c37be in_seq=0 out_seq=0 2023-09-12T16:04:54.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1f73fff700 10 client.15882 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:54.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1f73fff700 1 -- 172.21.15.130:0/1773199272 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a8bb0048a0 con 0x55a8bb0b77d0 2023-09-12T16:04:54.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1f73fff700 10 client.15882 handle_command_reply: tid=0 2023-09-12T16:04:54.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1fa7342b80 1 client.15882 shutdown 2023-09-12T16:04:54.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1fa7342b80 1 -- 172.21.15.130:0/1773199272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a2230 msgr2=0x55a8bb0aba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:54.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a2230 0x55a8bb0aba60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1f88010990 tx=0x7f1f880109c0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1fa7342b80 1 -- 172.21.15.130:0/1773199272 shutdown_connections 2023-09-12T16:04:54.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a8bb0b77d0 0x55a8bb0b9bf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a2230 0x55a8bb0aba60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1f73fff700 0 client.15882 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:54.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/1773199272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a3070 0x55a8bb0abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.640+0000 7f1fa7342b80 1 -- 172.21.15.130:0/1773199272 >> 172.21.15.130:0/1773199272 conn(0x55a8bb012090 msgr2=0x55a8bb097dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:54.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.641+0000 7f1fa7342b80 1 -- 172.21.15.130:0/1773199272 shutdown_connections 2023-09-12T16:04:54.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.641+0000 7f1fa7342b80 1 -- 172.21.15.130:0/1773199272 wait complete. 2023-09-12T16:04:54.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.641+0000 7f1fa7342b80 20 client.15882 trim_cache size 0 max 16384 2023-09-12T16:04:54.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1fa7342b80 1 Processor -- start 2023-09-12T16:04:54.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1fa7342b80 1 -- start start 2023-09-12T16:04:54.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.675 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bb0aa320 con 0x55a8bb0a3070 2023-09-12T16:04:54.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bb0aaf10 con 0x55a8bb0a39d0 2023-09-12T16:04:54.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bb099670 con 0x55a8bb0a2230 2023-09-12T16:04:54.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1f99719700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1f99719700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:38974/0 (socket says 172.21.15.130:38974) 2023-09-12T16:04:54.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1f93fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1f99719700 1 -- 172.21.15.130:0/886243258 learned_addr learned my addr 172.21.15.130:0/886243258 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.662+0000 7f1f98f18700 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f99719700 1 -- 172.21.15.130:0/886243258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 msgr2=0x55a8bb0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f99719700 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f99719700 1 -- 172.21.15.130:0/886243258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a3070 msgr2=0x55a8bb0a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:54.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f99719700 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f99719700 1 -- 172.21.15.130:0/886243258 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bae42250 con 0x55a8bb0a39d0 2023-09-12T16:04:54.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f98f18700 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:54.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f93fff700 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:54.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f99719700 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a6930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c011b80 tx=0x7f1f8c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a26fa7bc173a163 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:54.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f7f7fe700 1 -- 172.21.15.130:0/886243258 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00e040 con 0x55a8bb0a39d0 2023-09-12T16:04:54.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f7f7fe700 1 -- 172.21.15.130:0/886243258 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f8c00f040 con 0x55a8bb0a39d0 2023-09-12T16:04:54.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.663+0000 7f1f7f7fe700 1 -- 172.21.15.130:0/886243258 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0042b0 con 0x55a8bb0a39d0 2023-09-12T16:04:54.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 -- 172.21.15.130:0/886243258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 msgr2=0x55a8bb0a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:54.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a6930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c011b80 tx=0x7f1f8c009920 comp rx=0 tx=0).stop 2023-09-12T16:04:54.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 -- 172.21.15.130:0/886243258 shutdown_connections 2023-09-12T16:04:54.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0a6930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/886243258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a3070 0x55a8bb0a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 -- 172.21.15.130:0/886243258 >> 172.21.15.130:0/886243258 conn(0x55a8bb012090 msgr2=0x55a8bb0aa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:54.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 -- 172.21.15.130:0/886243258 shutdown_connections 2023-09-12T16:04:54.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 -- 172.21.15.130:0/886243258 wait complete. 2023-09-12T16:04:54.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 -- start start 2023-09-12T16:04:54.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.664+0000 7f1fa7342b80 1 Processor -- start 2023-09-12T16:04:54.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1fa7342b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bb0b02b0 con 0x55a8bb0a39d0 2023-09-12T16:04:54.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bb0b3270 con 0x55a8bb0a3070 2023-09-12T16:04:54.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1fa7342b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bb0b3400 con 0x55a8bb0a2230 2023-09-12T16:04:54.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1fa7342b80 10 client.0 _ll_register_callbacks cb 0x7f1f80005720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:04:54.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f93fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f99719700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f98f18700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f93fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:38978/0 (socket says 172.21.15.130:38978) 2023-09-12T16:04:54.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f93fff700 1 -- 172.21.15.130:0/4183754997 learned_addr learned my addr 172.21.15.130:0/4183754997 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:04:54.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f99719700 1 -- 172.21.15.130:0/4183754997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 msgr2=0x55a8bb0b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:54.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f99719700 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f99719700 1 -- 172.21.15.130:0/4183754997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 msgr2=0x55a8bb0af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:04:54.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f99719700 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f99719700 1 -- 172.21.15.130:0/4183754997 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8bae04c60 con 0x55a8bb0a39d0 2023-09-12T16:04:54.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f93fff700 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:54.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f98f18700 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:04:54.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f99719700 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0afbd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00b010 tx=0x7f1f8c009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef797d35643459be server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:04:54.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f73fff700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:04:54.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1f73fff700 1 -- 172.21.15.130:0/4183754997 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00e070 con 0x55a8bb0a39d0 2023-09-12T16:04:54.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.665+0000 7f1fa7342b80 1 -- 172.21.15.130:0/4183754997 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f88013e60 con 0x55a8bb0a39d0 2023-09-12T16:04:54.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1f73fff700 1 -- 172.21.15.130:0/4183754997 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1f8c007500 con 0x55a8bb0a39d0 2023-09-12T16:04:54.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1f73fff700 1 -- 172.21.15.130:0/4183754997 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f8c004e60 con 0x55a8bb0a39d0 2023-09-12T16:04:54.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1fa7342b80 10 client.15888 fetch_fsmap learned FSMap version 28 2023-09-12T16:04:54.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1fa7342b80 1 -- 172.21.15.130:0/4183754997 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f1f9419e660 con 0x55a8bb0a39d0 2023-09-12T16:04:54.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1f73fff700 1 -- 172.21.15.130:0/4183754997 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f88013e60 con 0x55a8bb0a39d0 2023-09-12T16:04:54.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1f73fff700 1 -- 172.21.15.130:0/4183754997 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1f8c005480 con 0x55a8bb0a39d0 2023-09-12T16:04:54.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1fa7342b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:04:54.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1fa7342b80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:04:54.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1fa7342b80 20 client.15888 populate_metadata read hostname 'smithi130' 2023-09-12T16:04:54.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a8bb0b77d0 0x55a8bb10eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:04:54.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1fa7342b80 4 client.15888 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:04:54.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1fa7342b80 1 -- 172.21.15.130:0/4183754997 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a8bafe8920 con 0x55a8bb0b77d0 2023-09-12T16:04:54.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.666+0000 7f1f98f18700 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a8bb0b77d0 0x55a8bb10eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:04:54.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1f73fff700 10 client.15888 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:54.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1f98f18700 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a8bb0b77d0 0x55a8bb10eea0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c63dedaefe8a9d6c server_cookie=ae92616c5e2d6f1f in_seq=0 out_seq=0 2023-09-12T16:04:54.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1f73fff700 1 -- 172.21.15.130:0/4183754997 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a8bafe8920 con 0x55a8bb0b77d0 2023-09-12T16:04:54.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1f73fff700 10 client.15888 handle_command_reply: tid=0 2023-09-12T16:04:54.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 client.15888 shutdown 2023-09-12T16:04:54.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 -- 172.21.15.130:0/4183754997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 msgr2=0x55a8bb0afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:54.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0afbd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00b010 tx=0x7f1f8c009e80 comp rx=0 tx=0).stop 2023-09-12T16:04:54.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 -- 172.21.15.130:0/4183754997 shutdown_connections 2023-09-12T16:04:54.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a8bb0b77d0 0x55a8bb10eea0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a8bb0a39d0 0x55a8bb0afbd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8bb0a2230 0x55a8bb0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1f73fff700 0 client.15888 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:04:54.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 --2- 172.21.15.130:0/4183754997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8bb0a3070 0x55a8bb0af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 -- 172.21.15.130:0/4183754997 >> 172.21.15.130:0/4183754997 conn(0x55a8bb012090 msgr2=0x55a8bb0a63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:54.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 -- 172.21.15.130:0/4183754997 shutdown_connections 2023-09-12T16:04:54.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 1 -- 172.21.15.130:0/4183754997 wait complete. 2023-09-12T16:04:54.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.667+0000 7f1fa7342b80 20 client.15888 trim_cache size 0 max 16384 2023-09-12T16:04:54.697 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:04:54.697 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:04:54.697 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:04:54.697 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:04:54.698 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:04:54.698 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:04:54.698 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:04:54.699 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:04:54.699 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:04:54.699 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:04:54.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 -- 172.21.15.130:0/1117645933 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f840a0640 msgr2=0x7f1f840a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:54.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f840a0640 0x7f1f840a2b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f88022000 tx=0x7f1f88016000 comp rx=0 tx=0).stop 2023-09-12T16:04:54.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 -- 172.21.15.130:0/1117645933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f941039a0 msgr2=0x7f1f94196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:04:54.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f941039a0 0x7f1f94196840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000c00 tx=0x7f1f8000b790 comp rx=0 tx=0).stop 2023-09-12T16:04:54.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 -- 172.21.15.130:0/1117645933 shutdown_connections 2023-09-12T16:04:54.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1f840a0640 0x7f1f840a2b00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f941062d0 0x7f1f9419d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94101070 0x7f1f94196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 --2- 172.21.15.130:0/1117645933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f941039a0 0x7f1f94196840 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:04:54.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 -- 172.21.15.130:0/1117645933 >> 172.21.15.130:0/1117645933 conn(0x7f1f940faa60 msgr2=0x7f1f94101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:04:54.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 -- 172.21.15.130:0/1117645933 shutdown_connections 2023-09-12T16:04:54.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:04:54.668+0000 7f1f9b17c700 1 -- 172.21.15.130:0/1117645933 wait complete. 2023-09-12T16:04:54.704 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:04:54.730 INFO:teuthology.orchestra.run.smithi130.stderr:37600000 of 50000000 tuples (75%) done (elapsed 57.10 s, remaining 18.83 s) 2023-09-12T16:04:54.844 INFO:teuthology.orchestra.run.smithi130.stderr:37700000 of 50000000 tuples (75%) done (elapsed 57.21 s, remaining 18.66 s) 2023-09-12T16:04:54.948 INFO:teuthology.orchestra.run.smithi130.stderr:37800000 of 50000000 tuples (75%) done (elapsed 57.31 s, remaining 18.50 s) 2023-09-12T16:04:55.049 INFO:teuthology.orchestra.run.smithi130.stderr:37900000 of 50000000 tuples (75%) done (elapsed 57.41 s, remaining 18.33 s) 2023-09-12T16:04:55.466 INFO:teuthology.orchestra.run.smithi130.stderr:38000000 of 50000000 tuples (76%) done (elapsed 57.83 s, remaining 18.26 s) 2023-09-12T16:04:55.914 INFO:teuthology.orchestra.run.smithi130.stderr:38100000 of 50000000 tuples (76%) done (elapsed 58.28 s, remaining 18.20 s) 2023-09-12T16:04:56.013 INFO:teuthology.orchestra.run.smithi130.stderr:38200000 of 50000000 tuples (76%) done (elapsed 58.38 s, remaining 18.03 s) 2023-09-12T16:04:56.122 INFO:teuthology.orchestra.run.smithi130.stderr:38300000 of 50000000 tuples (76%) done (elapsed 58.49 s, remaining 17.87 s) 2023-09-12T16:04:56.187 INFO:teuthology.orchestra.run.smithi130.stderr:38400000 of 50000000 tuples (76%) done (elapsed 58.55 s, remaining 17.69 s) 2023-09-12T16:04:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:55 smithi130 ceph-mon[98670]: pgmap v352: 129 pgs: 129 active+clean; 5.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 141 MiB/s wr, 187 op/s 2023-09-12T16:04:56.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:55 smithi159 ceph-mon[102522]: pgmap v352: 129 pgs: 129 active+clean; 5.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 141 MiB/s wr, 187 op/s 2023-09-12T16:04:56.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:55 smithi203 ceph-mon[103431]: pgmap v352: 129 pgs: 129 active+clean; 5.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 141 MiB/s wr, 187 op/s 2023-09-12T16:04:56.319 INFO:teuthology.orchestra.run.smithi130.stderr:38500000 of 50000000 tuples (77%) done (elapsed 58.68 s, remaining 17.53 s) 2023-09-12T16:04:56.556 INFO:teuthology.orchestra.run.smithi130.stderr:38600000 of 50000000 tuples (77%) done (elapsed 58.92 s, remaining 17.40 s) 2023-09-12T16:04:57.191 INFO:teuthology.orchestra.run.smithi130.stderr:38700000 of 50000000 tuples (77%) done (elapsed 59.56 s, remaining 17.39 s) 2023-09-12T16:04:57.258 INFO:teuthology.orchestra.run.smithi130.stderr:38800000 of 50000000 tuples (77%) done (elapsed 59.62 s, remaining 17.21 s) 2023-09-12T16:04:57.408 INFO:teuthology.orchestra.run.smithi130.stderr:38900000 of 50000000 tuples (77%) done (elapsed 59.77 s, remaining 17.06 s) 2023-09-12T16:04:57.929 INFO:teuthology.orchestra.run.smithi130.stderr:39000000 of 50000000 tuples (78%) done (elapsed 60.29 s, remaining 17.01 s) 2023-09-12T16:04:57.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:57 smithi130 ceph-mon[98670]: pgmap v353: 129 pgs: 129 active+clean; 5.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 145 MiB/s wr, 207 op/s 2023-09-12T16:04:57.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:57 smithi159 ceph-mon[102522]: pgmap v353: 129 pgs: 129 active+clean; 5.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 145 MiB/s wr, 207 op/s 2023-09-12T16:04:57.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:57 smithi203 ceph-mon[103431]: pgmap v353: 129 pgs: 129 active+clean; 5.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 145 MiB/s wr, 207 op/s 2023-09-12T16:04:58.013 INFO:teuthology.orchestra.run.smithi130.stderr:39100000 of 50000000 tuples (78%) done (elapsed 60.38 s, remaining 16.83 s) 2023-09-12T16:04:58.129 INFO:teuthology.orchestra.run.smithi130.stderr:39200000 of 50000000 tuples (78%) done (elapsed 60.49 s, remaining 16.67 s) 2023-09-12T16:04:58.225 INFO:teuthology.orchestra.run.smithi130.stderr:39300000 of 50000000 tuples (78%) done (elapsed 60.59 s, remaining 16.50 s) 2023-09-12T16:04:58.613 INFO:teuthology.orchestra.run.smithi130.stderr:39400000 of 50000000 tuples (78%) done (elapsed 60.98 s, remaining 16.41 s) 2023-09-12T16:04:59.068 INFO:teuthology.orchestra.run.smithi130.stderr:39500000 of 50000000 tuples (79%) done (elapsed 61.43 s, remaining 16.33 s) 2023-09-12T16:04:59.169 INFO:teuthology.orchestra.run.smithi130.stderr:39600000 of 50000000 tuples (79%) done (elapsed 61.53 s, remaining 16.16 s) 2023-09-12T16:04:59.273 INFO:teuthology.orchestra.run.smithi130.stderr:39700000 of 50000000 tuples (79%) done (elapsed 61.64 s, remaining 15.99 s) 2023-09-12T16:04:59.377 INFO:teuthology.orchestra.run.smithi130.stderr:39800000 of 50000000 tuples (79%) done (elapsed 61.74 s, remaining 15.82 s) 2023-09-12T16:04:59.448 INFO:teuthology.orchestra.run.smithi130.stderr:39900000 of 50000000 tuples (79%) done (elapsed 61.81 s, remaining 15.65 s) 2023-09-12T16:04:59.557 INFO:teuthology.orchestra.run.smithi130.stderr:40000000 of 50000000 tuples (80%) done (elapsed 61.92 s, remaining 15.48 s) 2023-09-12T16:04:59.658 INFO:teuthology.orchestra.run.smithi130.stderr:40100000 of 50000000 tuples (80%) done (elapsed 62.02 s, remaining 15.31 s) 2023-09-12T16:04:59.761 INFO:teuthology.orchestra.run.smithi130.stderr:40200000 of 50000000 tuples (80%) done (elapsed 62.13 s, remaining 15.15 s) 2023-09-12T16:04:59.873 INFO:teuthology.orchestra.run.smithi130.stderr:40300000 of 50000000 tuples (80%) done (elapsed 62.24 s, remaining 14.98 s) 2023-09-12T16:04:59.974 INFO:teuthology.orchestra.run.smithi130.stderr:40400000 of 50000000 tuples (80%) done (elapsed 62.34 s, remaining 14.81 s) 2023-09-12T16:05:00.071 INFO:teuthology.orchestra.run.smithi130.stderr:40500000 of 50000000 tuples (81%) done (elapsed 62.44 s, remaining 14.65 s) 2023-09-12T16:05:00.181 INFO:teuthology.orchestra.run.smithi130.stderr:40600000 of 50000000 tuples (81%) done (elapsed 62.55 s, remaining 14.48 s) 2023-09-12T16:05:00.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:04:59 smithi130 ceph-mon[98670]: pgmap v354: 129 pgs: 129 active+clean; 5.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 146 MiB/s wr, 235 op/s 2023-09-12T16:05:00.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:04:59 smithi159 ceph-mon[102522]: pgmap v354: 129 pgs: 129 active+clean; 5.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 146 MiB/s wr, 235 op/s 2023-09-12T16:05:00.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:04:59 smithi203 ceph-mon[103431]: pgmap v354: 129 pgs: 129 active+clean; 5.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 146 MiB/s wr, 235 op/s 2023-09-12T16:05:00.286 INFO:teuthology.orchestra.run.smithi130.stderr:40700000 of 50000000 tuples (81%) done (elapsed 62.65 s, remaining 14.32 s) 2023-09-12T16:05:00.389 INFO:teuthology.orchestra.run.smithi130.stderr:40800000 of 50000000 tuples (81%) done (elapsed 62.76 s, remaining 14.15 s) 2023-09-12T16:05:00.497 INFO:teuthology.orchestra.run.smithi130.stderr:40900000 of 50000000 tuples (81%) done (elapsed 62.86 s, remaining 13.99 s) 2023-09-12T16:05:00.915 INFO:teuthology.orchestra.run.smithi130.stderr:41000000 of 50000000 tuples (82%) done (elapsed 63.28 s, remaining 13.89 s) 2023-09-12T16:05:01.235 INFO:teuthology.orchestra.run.smithi130.stderr:41100000 of 50000000 tuples (82%) done (elapsed 63.60 s, remaining 13.77 s) 2023-09-12T16:05:01.331 INFO:teuthology.orchestra.run.smithi130.stderr:41200000 of 50000000 tuples (82%) done (elapsed 63.70 s, remaining 13.61 s) 2023-09-12T16:05:01.430 INFO:teuthology.orchestra.run.smithi130.stderr:41300000 of 50000000 tuples (82%) done (elapsed 63.80 s, remaining 13.44 s) 2023-09-12T16:05:01.554 INFO:teuthology.orchestra.run.smithi130.stderr:41400000 of 50000000 tuples (82%) done (elapsed 63.92 s, remaining 13.28 s) 2023-09-12T16:05:01.655 INFO:teuthology.orchestra.run.smithi130.stderr:41500000 of 50000000 tuples (83%) done (elapsed 64.02 s, remaining 13.11 s) 2023-09-12T16:05:01.750 INFO:teuthology.orchestra.run.smithi130.stderr:41600000 of 50000000 tuples (83%) done (elapsed 64.12 s, remaining 12.95 s) 2023-09-12T16:05:01.873 INFO:teuthology.orchestra.run.smithi130.stderr:41700000 of 50000000 tuples (83%) done (elapsed 64.24 s, remaining 12.79 s) 2023-09-12T16:05:01.959 INFO:teuthology.orchestra.run.smithi130.stderr:41800000 of 50000000 tuples (83%) done (elapsed 64.32 s, remaining 12.62 s) 2023-09-12T16:05:02.065 INFO:teuthology.orchestra.run.smithi130.stderr:41900000 of 50000000 tuples (83%) done (elapsed 64.43 s, remaining 12.46 s) 2023-09-12T16:05:02.162 INFO:teuthology.orchestra.run.smithi130.stderr:42000000 of 50000000 tuples (84%) done (elapsed 64.53 s, remaining 12.29 s) 2023-09-12T16:05:02.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:01 smithi130 ceph-mon[98670]: pgmap v355: 129 pgs: 129 active+clean; 5.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 143 MiB/s wr, 260 op/s 2023-09-12T16:05:02.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:01 smithi159 ceph-mon[102522]: pgmap v355: 129 pgs: 129 active+clean; 5.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 143 MiB/s wr, 260 op/s 2023-09-12T16:05:02.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:01 smithi203 ceph-mon[103431]: pgmap v355: 129 pgs: 129 active+clean; 5.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 143 MiB/s wr, 260 op/s 2023-09-12T16:05:02.787 INFO:teuthology.orchestra.run.smithi130.stderr:42100000 of 50000000 tuples (84%) done (elapsed 65.15 s, remaining 12.23 s) 2023-09-12T16:05:03.046 INFO:teuthology.orchestra.run.smithi130.stderr:42200000 of 50000000 tuples (84%) done (elapsed 65.41 s, remaining 12.09 s) 2023-09-12T16:05:03.142 INFO:teuthology.orchestra.run.smithi130.stderr:42300000 of 50000000 tuples (84%) done (elapsed 65.51 s, remaining 11.92 s) 2023-09-12T16:05:03.253 INFO:teuthology.orchestra.run.smithi130.stderr:42400000 of 50000000 tuples (84%) done (elapsed 65.62 s, remaining 11.76 s) 2023-09-12T16:05:03.355 INFO:teuthology.orchestra.run.smithi130.stderr:42500000 of 50000000 tuples (85%) done (elapsed 65.72 s, remaining 11.60 s) 2023-09-12T16:05:03.455 INFO:teuthology.orchestra.run.smithi130.stderr:42600000 of 50000000 tuples (85%) done (elapsed 65.82 s, remaining 11.43 s) 2023-09-12T16:05:03.533 INFO:teuthology.orchestra.run.smithi130.stderr:42700000 of 50000000 tuples (85%) done (elapsed 65.90 s, remaining 11.27 s) 2023-09-12T16:05:03.672 INFO:teuthology.orchestra.run.smithi130.stderr:42800000 of 50000000 tuples (85%) done (elapsed 66.04 s, remaining 11.11 s) 2023-09-12T16:05:03.739 INFO:teuthology.orchestra.run.smithi130.stderr:42900000 of 50000000 tuples (85%) done (elapsed 66.10 s, remaining 10.94 s) 2023-09-12T16:05:03.841 INFO:teuthology.orchestra.run.smithi130.stderr:43000000 of 50000000 tuples (86%) done (elapsed 66.21 s, remaining 10.78 s) 2023-09-12T16:05:03.963 INFO:teuthology.orchestra.run.smithi130.stderr:43100000 of 50000000 tuples (86%) done (elapsed 66.33 s, remaining 10.62 s) 2023-09-12T16:05:04.066 INFO:teuthology.orchestra.run.smithi130.stderr:43200000 of 50000000 tuples (86%) done (elapsed 66.43 s, remaining 10.46 s) 2023-09-12T16:05:04.151 INFO:teuthology.orchestra.run.smithi130.stderr:43300000 of 50000000 tuples (86%) done (elapsed 66.52 s, remaining 10.29 s) 2023-09-12T16:05:04.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:03 smithi130 ceph-mon[98670]: pgmap v356: 129 pgs: 129 active+clean; 5.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 138 MiB/s wr, 287 op/s 2023-09-12T16:05:04.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:03 smithi159 ceph-mon[102522]: pgmap v356: 129 pgs: 129 active+clean; 5.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 138 MiB/s wr, 287 op/s 2023-09-12T16:05:04.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:03 smithi203 ceph-mon[103431]: pgmap v356: 129 pgs: 129 active+clean; 5.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 138 MiB/s wr, 287 op/s 2023-09-12T16:05:04.259 INFO:teuthology.orchestra.run.smithi130.stderr:43400000 of 50000000 tuples (86%) done (elapsed 66.62 s, remaining 10.13 s) 2023-09-12T16:05:04.365 INFO:teuthology.orchestra.run.smithi130.stderr:43500000 of 50000000 tuples (87%) done (elapsed 66.73 s, remaining 9.97 s) 2023-09-12T16:05:04.460 INFO:teuthology.orchestra.run.smithi130.stderr:43600000 of 50000000 tuples (87%) done (elapsed 66.83 s, remaining 9.81 s) 2023-09-12T16:05:04.560 INFO:teuthology.orchestra.run.smithi130.stderr:43700000 of 50000000 tuples (87%) done (elapsed 66.93 s, remaining 9.65 s) 2023-09-12T16:05:04.642 INFO:teuthology.orchestra.run.smithi130.stderr:43800000 of 50000000 tuples (87%) done (elapsed 67.01 s, remaining 9.49 s) 2023-09-12T16:05:04.787 INFO:teuthology.orchestra.run.smithi130.stderr:43900000 of 50000000 tuples (87%) done (elapsed 67.15 s, remaining 9.33 s) 2023-09-12T16:05:04.853 INFO:teuthology.orchestra.run.smithi130.stderr:44000000 of 50000000 tuples (88%) done (elapsed 67.22 s, remaining 9.17 s) 2023-09-12T16:05:04.971 INFO:teuthology.orchestra.run.smithi130.stderr:44100000 of 50000000 tuples (88%) done (elapsed 67.34 s, remaining 9.01 s) 2023-09-12T16:05:05.086 INFO:teuthology.orchestra.run.smithi130.stderr:44200000 of 50000000 tuples (88%) done (elapsed 67.45 s, remaining 8.85 s) 2023-09-12T16:05:05.185 INFO:teuthology.orchestra.run.smithi130.stderr:44300000 of 50000000 tuples (88%) done (elapsed 67.55 s, remaining 8.69 s) 2023-09-12T16:05:05.644 INFO:teuthology.orchestra.run.smithi130.stderr:44400000 of 50000000 tuples (88%) done (elapsed 68.01 s, remaining 8.58 s) 2023-09-12T16:05:06.092 INFO:teuthology.orchestra.run.smithi130.stderr:44500000 of 50000000 tuples (89%) done (elapsed 68.46 s, remaining 8.46 s) 2023-09-12T16:05:06.196 INFO:teuthology.orchestra.run.smithi130.stderr:44600000 of 50000000 tuples (89%) done (elapsed 68.56 s, remaining 8.30 s) 2023-09-12T16:05:06.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:05 smithi130 ceph-mon[98670]: pgmap v357: 129 pgs: 129 active+clean; 6.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 132 MiB/s wr, 247 op/s 2023-09-12T16:05:06.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:05 smithi159 ceph-mon[102522]: pgmap v357: 129 pgs: 129 active+clean; 6.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 132 MiB/s wr, 247 op/s 2023-09-12T16:05:06.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:05 smithi203 ceph-mon[103431]: pgmap v357: 129 pgs: 129 active+clean; 6.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 132 MiB/s wr, 247 op/s 2023-09-12T16:05:06.298 INFO:teuthology.orchestra.run.smithi130.stderr:44700000 of 50000000 tuples (89%) done (elapsed 68.66 s, remaining 8.14 s) 2023-09-12T16:05:06.399 INFO:teuthology.orchestra.run.smithi130.stderr:44800000 of 50000000 tuples (89%) done (elapsed 68.77 s, remaining 7.98 s) 2023-09-12T16:05:06.470 INFO:teuthology.orchestra.run.smithi130.stderr:44900000 of 50000000 tuples (89%) done (elapsed 68.84 s, remaining 7.82 s) 2023-09-12T16:05:07.070 INFO:teuthology.orchestra.run.smithi130.stderr:45000000 of 50000000 tuples (90%) done (elapsed 69.44 s, remaining 7.72 s) 2023-09-12T16:05:07.139 INFO:teuthology.orchestra.run.smithi130.stderr:45100000 of 50000000 tuples (90%) done (elapsed 69.50 s, remaining 7.55 s) 2023-09-12T16:05:07.251 INFO:teuthology.orchestra.run.smithi130.stderr:45200000 of 50000000 tuples (90%) done (elapsed 69.62 s, remaining 7.39 s) 2023-09-12T16:05:07.372 INFO:teuthology.orchestra.run.smithi130.stderr:45300000 of 50000000 tuples (90%) done (elapsed 69.74 s, remaining 7.24 s) 2023-09-12T16:05:07.473 INFO:teuthology.orchestra.run.smithi130.stderr:45400000 of 50000000 tuples (90%) done (elapsed 69.84 s, remaining 7.08 s) 2023-09-12T16:05:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:45500000 of 50000000 tuples (91%) done (elapsed 70.32 s, remaining 6.95 s) 2023-09-12T16:05:07.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:07 smithi130 ceph-mon[98670]: pgmap v358: 129 pgs: 129 active+clean; 6.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 132 MiB/s wr, 236 op/s 2023-09-12T16:05:07.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:07 smithi159 ceph-mon[102522]: pgmap v358: 129 pgs: 129 active+clean; 6.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 132 MiB/s wr, 236 op/s 2023-09-12T16:05:07.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:07 smithi203 ceph-mon[103431]: pgmap v358: 129 pgs: 129 active+clean; 6.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 132 MiB/s wr, 236 op/s 2023-09-12T16:05:08.402 INFO:teuthology.orchestra.run.smithi130.stderr:45600000 of 50000000 tuples (91%) done (elapsed 70.77 s, remaining 6.83 s) 2023-09-12T16:05:08.496 INFO:teuthology.orchestra.run.smithi130.stderr:45700000 of 50000000 tuples (91%) done (elapsed 70.86 s, remaining 6.67 s) 2023-09-12T16:05:08.605 INFO:teuthology.orchestra.run.smithi130.stderr:45800000 of 50000000 tuples (91%) done (elapsed 70.97 s, remaining 6.51 s) 2023-09-12T16:05:08.708 INFO:teuthology.orchestra.run.smithi130.stderr:45900000 of 50000000 tuples (91%) done (elapsed 71.07 s, remaining 6.35 s) 2023-09-12T16:05:08.774 INFO:teuthology.orchestra.run.smithi130.stderr:46000000 of 50000000 tuples (92%) done (elapsed 71.14 s, remaining 6.19 s) 2023-09-12T16:05:08.885 INFO:teuthology.orchestra.run.smithi130.stderr:46100000 of 50000000 tuples (92%) done (elapsed 71.25 s, remaining 6.03 s) 2023-09-12T16:05:08.991 INFO:teuthology.orchestra.run.smithi130.stderr:46200000 of 50000000 tuples (92%) done (elapsed 71.36 s, remaining 5.87 s) 2023-09-12T16:05:09.103 INFO:teuthology.orchestra.run.smithi130.stderr:46300000 of 50000000 tuples (92%) done (elapsed 71.47 s, remaining 5.71 s) 2023-09-12T16:05:09.210 INFO:teuthology.orchestra.run.smithi130.stderr:46400000 of 50000000 tuples (92%) done (elapsed 71.58 s, remaining 5.55 s) 2023-09-12T16:05:09.309 INFO:teuthology.orchestra.run.smithi130.stderr:46500000 of 50000000 tuples (93%) done (elapsed 71.67 s, remaining 5.39 s) 2023-09-12T16:05:09.403 INFO:teuthology.orchestra.run.smithi130.stderr:46600000 of 50000000 tuples (93%) done (elapsed 71.77 s, remaining 5.24 s) 2023-09-12T16:05:09.519 INFO:teuthology.orchestra.run.smithi130.stderr:46700000 of 50000000 tuples (93%) done (elapsed 71.89 s, remaining 5.08 s) 2023-09-12T16:05:09.625 INFO:teuthology.orchestra.run.smithi130.stderr:46800000 of 50000000 tuples (93%) done (elapsed 71.99 s, remaining 4.92 s) 2023-09-12T16:05:09.731 INFO:teuthology.orchestra.run.smithi130.stderr:46900000 of 50000000 tuples (93%) done (elapsed 72.10 s, remaining 4.77 s) 2023-09-12T16:05:09.797 INFO:teuthology.orchestra.run.smithi130.stderr:47000000 of 50000000 tuples (94%) done (elapsed 72.16 s, remaining 4.61 s) 2023-09-12T16:05:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:09 smithi130 ceph-mon[98670]: pgmap v359: 129 pgs: 129 active+clean; 6.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 130 MiB/s wr, 223 op/s 2023-09-12T16:05:10.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:09 smithi159 ceph-mon[102522]: pgmap v359: 129 pgs: 129 active+clean; 6.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 130 MiB/s wr, 223 op/s 2023-09-12T16:05:10.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:09 smithi203 ceph-mon[103431]: pgmap v359: 129 pgs: 129 active+clean; 6.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 130 MiB/s wr, 223 op/s 2023-09-12T16:05:10.382 INFO:teuthology.orchestra.run.smithi130.stderr:47100000 of 50000000 tuples (94%) done (elapsed 72.75 s, remaining 4.48 s) 2023-09-12T16:05:10.637 INFO:teuthology.orchestra.run.smithi130.stderr:47200000 of 50000000 tuples (94%) done (elapsed 73.00 s, remaining 4.33 s) 2023-09-12T16:05:10.737 INFO:teuthology.orchestra.run.smithi130.stderr:47300000 of 50000000 tuples (94%) done (elapsed 73.10 s, remaining 4.17 s) 2023-09-12T16:05:10.846 INFO:teuthology.orchestra.run.smithi130.stderr:47400000 of 50000000 tuples (94%) done (elapsed 73.21 s, remaining 4.02 s) 2023-09-12T16:05:10.939 INFO:teuthology.orchestra.run.smithi130.stderr:47500000 of 50000000 tuples (95%) done (elapsed 73.30 s, remaining 3.86 s) 2023-09-12T16:05:11.040 INFO:teuthology.orchestra.run.smithi130.stderr:47600000 of 50000000 tuples (95%) done (elapsed 73.41 s, remaining 3.70 s) 2023-09-12T16:05:11.117 INFO:teuthology.orchestra.run.smithi130.stderr:47700000 of 50000000 tuples (95%) done (elapsed 73.48 s, remaining 3.54 s) 2023-09-12T16:05:11.247 INFO:teuthology.orchestra.run.smithi130.stderr:47800000 of 50000000 tuples (95%) done (elapsed 73.61 s, remaining 3.39 s) 2023-09-12T16:05:11.318 INFO:teuthology.orchestra.run.smithi130.stderr:47900000 of 50000000 tuples (95%) done (elapsed 73.68 s, remaining 3.23 s) 2023-09-12T16:05:11.420 INFO:teuthology.orchestra.run.smithi130.stderr:48000000 of 50000000 tuples (96%) done (elapsed 73.79 s, remaining 3.07 s) 2023-09-12T16:05:11.543 INFO:teuthology.orchestra.run.smithi130.stderr:48100000 of 50000000 tuples (96%) done (elapsed 73.91 s, remaining 2.92 s) 2023-09-12T16:05:11.653 INFO:teuthology.orchestra.run.smithi130.stderr:48200000 of 50000000 tuples (96%) done (elapsed 74.02 s, remaining 2.76 s) 2023-09-12T16:05:11.737 INFO:teuthology.orchestra.run.smithi130.stderr:48300000 of 50000000 tuples (96%) done (elapsed 74.10 s, remaining 2.61 s) 2023-09-12T16:05:11.843 INFO:teuthology.orchestra.run.smithi130.stderr:48400000 of 50000000 tuples (96%) done (elapsed 74.21 s, remaining 2.45 s) 2023-09-12T16:05:11.954 INFO:teuthology.orchestra.run.smithi130.stderr:48500000 of 50000000 tuples (97%) done (elapsed 74.32 s, remaining 2.30 s) 2023-09-12T16:05:12.044 INFO:teuthology.orchestra.run.smithi130.stderr:48600000 of 50000000 tuples (97%) done (elapsed 74.41 s, remaining 2.14 s) 2023-09-12T16:05:12.141 INFO:teuthology.orchestra.run.smithi130.stderr:48700000 of 50000000 tuples (97%) done (elapsed 74.51 s, remaining 1.99 s) 2023-09-12T16:05:12.220 INFO:teuthology.orchestra.run.smithi130.stderr:48800000 of 50000000 tuples (97%) done (elapsed 74.59 s, remaining 1.83 s) 2023-09-12T16:05:12.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:11 smithi130 ceph-mon[98670]: pgmap v360: 129 pgs: 129 active+clean; 6.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 148 MiB/s wr, 216 op/s 2023-09-12T16:05:12.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:11 smithi159 ceph-mon[102522]: pgmap v360: 129 pgs: 129 active+clean; 6.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 148 MiB/s wr, 216 op/s 2023-09-12T16:05:12.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:11 smithi203 ceph-mon[103431]: pgmap v360: 129 pgs: 129 active+clean; 6.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 148 MiB/s wr, 216 op/s 2023-09-12T16:05:12.650 INFO:teuthology.orchestra.run.smithi130.stderr:48900000 of 50000000 tuples (97%) done (elapsed 75.02 s, remaining 1.69 s) 2023-09-12T16:05:12.944 INFO:teuthology.orchestra.run.smithi130.stderr:49000000 of 50000000 tuples (98%) done (elapsed 75.31 s, remaining 1.54 s) 2023-09-12T16:05:13.075 INFO:teuthology.orchestra.run.smithi130.stderr:49100000 of 50000000 tuples (98%) done (elapsed 75.44 s, remaining 1.38 s) 2023-09-12T16:05:13.187 INFO:teuthology.orchestra.run.smithi130.stderr:49200000 of 50000000 tuples (98%) done (elapsed 75.55 s, remaining 1.23 s) 2023-09-12T16:05:13.297 INFO:teuthology.orchestra.run.smithi130.stderr:49300000 of 50000000 tuples (98%) done (elapsed 75.66 s, remaining 1.07 s) 2023-09-12T16:05:13.400 INFO:teuthology.orchestra.run.smithi130.stderr:49400000 of 50000000 tuples (98%) done (elapsed 75.77 s, remaining 0.92 s) 2023-09-12T16:05:13.499 INFO:teuthology.orchestra.run.smithi130.stderr:49500000 of 50000000 tuples (99%) done (elapsed 75.87 s, remaining 0.77 s) 2023-09-12T16:05:14.136 INFO:teuthology.orchestra.run.smithi130.stderr:49600000 of 50000000 tuples (99%) done (elapsed 76.50 s, remaining 0.62 s) 2023-09-12T16:05:14.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:13 smithi130 ceph-mon[98670]: pgmap v361: 129 pgs: 129 active+clean; 6.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 157 MiB/s wr, 224 op/s 2023-09-12T16:05:14.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:13 smithi159 ceph-mon[102522]: pgmap v361: 129 pgs: 129 active+clean; 6.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 157 MiB/s wr, 224 op/s 2023-09-12T16:05:14.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:13 smithi203 ceph-mon[103431]: pgmap v361: 129 pgs: 129 active+clean; 6.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 157 MiB/s wr, 224 op/s 2023-09-12T16:05:14.394 INFO:teuthology.orchestra.run.smithi130.stderr:49700000 of 50000000 tuples (99%) done (elapsed 76.76 s, remaining 0.46 s) 2023-09-12T16:05:14.492 INFO:teuthology.orchestra.run.smithi130.stderr:49800000 of 50000000 tuples (99%) done (elapsed 76.86 s, remaining 0.31 s) 2023-09-12T16:05:14.605 INFO:teuthology.orchestra.run.smithi130.stderr:49900000 of 50000000 tuples (99%) done (elapsed 76.97 s, remaining 0.15 s) 2023-09-12T16:05:14.714 INFO:teuthology.orchestra.run.smithi130.stderr:50000000 of 50000000 tuples (100%) done (elapsed 77.08 s, remaining 0.00 s) 2023-09-12T16:05:14.752 INFO:teuthology.orchestra.run.smithi130.stderr:vacuum... 2023-09-12T16:05:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:05:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:05:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:05:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:14 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:05:15.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:05:15.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:05:15.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:05:15.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:14 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:05:15.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:05:15.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:05:15.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:05:15.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:14 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:05:16.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:15 smithi130 ceph-mon[98670]: pgmap v362: 129 pgs: 129 active+clean; 6.8 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 149 MiB/s wr, 191 op/s 2023-09-12T16:05:16.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:15 smithi159 ceph-mon[102522]: pgmap v362: 129 pgs: 129 active+clean; 6.8 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 149 MiB/s wr, 191 op/s 2023-09-12T16:05:16.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:15 smithi203 ceph-mon[103431]: pgmap v362: 129 pgs: 129 active+clean; 6.8 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 149 MiB/s wr, 191 op/s 2023-09-12T16:05:17.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:17 smithi130 ceph-mon[98670]: pgmap v363: 129 pgs: 129 active+clean; 7.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 144 MiB/s wr, 178 op/s 2023-09-12T16:05:17.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:17 smithi159 ceph-mon[102522]: pgmap v363: 129 pgs: 129 active+clean; 7.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 144 MiB/s wr, 178 op/s 2023-09-12T16:05:17.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:17 smithi203 ceph-mon[103431]: pgmap v363: 129 pgs: 129 active+clean; 7.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 144 MiB/s wr, 178 op/s 2023-09-12T16:05:20.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:20 smithi159 ceph-mon[102522]: pgmap v364: 129 pgs: 129 active+clean; 7.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 146 MiB/s wr, 204 op/s 2023-09-12T16:05:20.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:20 smithi130 ceph-mon[98670]: pgmap v364: 129 pgs: 129 active+clean; 7.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 146 MiB/s wr, 204 op/s 2023-09-12T16:05:20.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:20 smithi203 ceph-mon[103431]: pgmap v364: 129 pgs: 129 active+clean; 7.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 146 MiB/s wr, 204 op/s 2023-09-12T16:05:22.474 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:22 smithi130 ceph-mon[98670]: pgmap v365: 129 pgs: 129 active+clean; 7.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 209 MiB/s wr, 196 op/s 2023-09-12T16:05:22.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:22 smithi159 ceph-mon[102522]: pgmap v365: 129 pgs: 129 active+clean; 7.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 209 MiB/s wr, 196 op/s 2023-09-12T16:05:22.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:22 smithi203 ceph-mon[103431]: pgmap v365: 129 pgs: 129 active+clean; 7.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 209 MiB/s wr, 196 op/s 2023-09-12T16:05:24.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:24 smithi159 ceph-mon[102522]: pgmap v366: 129 pgs: 129 active+clean; 7.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 298 MiB/s wr, 185 op/s 2023-09-12T16:05:24.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:24 smithi130 ceph-mon[98670]: pgmap v366: 129 pgs: 129 active+clean; 7.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 298 MiB/s wr, 185 op/s 2023-09-12T16:05:24.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:24 smithi203 ceph-mon[103431]: pgmap v366: 129 pgs: 129 active+clean; 7.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 298 MiB/s wr, 185 op/s 2023-09-12T16:05:24.705 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:05:24.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.839+0000 7f14086e1700 1 -- 172.21.15.130:0/874645049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1400101070 msgr2=0x7f1400103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:24.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.839+0000 7f14086e1700 1 --2- 172.21.15.130:0/874645049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1400101070 0x7f1400103460 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009990 tx=0x7f13fc009ca0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.840+0000 7f14086e1700 1 -- 172.21.15.130:0/874645049 shutdown_connections 2023-09-12T16:05:24.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.840+0000 7f14086e1700 1 --2- 172.21.15.130:0/874645049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14001062d0 0x7f1400108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.840+0000 7f14086e1700 1 --2- 172.21.15.130:0/874645049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1400101070 0x7f1400103460 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.840+0000 7f14086e1700 1 --2- 172.21.15.130:0/874645049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14001039a0 0x7f1400105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.840+0000 7f14086e1700 1 -- 172.21.15.130:0/874645049 >> 172.21.15.130:0/874645049 conn(0x7f14000faa60 msgr2=0x7f14000fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:24.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.840+0000 7f14086e1700 1 -- 172.21.15.130:0/874645049 shutdown_connections 2023-09-12T16:05:24.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.840+0000 7f14086e1700 1 -- 172.21.15.130:0/874645049 wait complete. 2023-09-12T16:05:24.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.840+0000 7f14086e1700 1 Processor -- start 2023-09-12T16:05:24.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.840+0000 7f14086e1700 1 -- start start 2023-09-12T16:05:24.845 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f14086e1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400101070 0x7f14001962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f14086e1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f14001039a0 0x7f1400196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f14086e1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14001062d0 0x7f140019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f14086e1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140019de40 con 0x7f14001039a0 2023-09-12T16:05:24.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f14086e1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140019df80 con 0x7f14001062d0 2023-09-12T16:05:24.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f14086e1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140019e110 con 0x7f1400101070 2023-09-12T16:05:24.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f140647d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400101070 0x7f14001962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f1405c7c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f14001039a0 0x7f1400196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f140647d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400101070 0x7f14001962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35344/0 (socket says 172.21.15.130:35344) 2023-09-12T16:05:24.847 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f1406c7e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14001062d0 0x7f140019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f140647d700 1 -- 172.21.15.130:0/1453967224 learned_addr learned my addr 172.21.15.130:0/1453967224 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:24.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f1406c7e700 1 -- 172.21.15.130:0/1453967224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400101070 msgr2=0x7f14001962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:24.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f1406c7e700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400101070 0x7f14001962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f1406c7e700 1 -- 172.21.15.130:0/1453967224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f14001039a0 msgr2=0x7f1400196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:24.848 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f1406c7e700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f14001039a0 0x7f1400196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f1406c7e700 1 -- 172.21.15.130:0/1453967224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc0095f0 con 0x7f14001062d0 2023-09-12T16:05:24.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f140647d700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400101070 0x7f14001962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:05:24.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.841+0000 7f1406c7e700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14001062d0 0x7f140019d870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f13f800efc0 tx=0x7f13f800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:24.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.842+0000 7f13f77fe700 1 -- 172.21.15.130:0/1453967224 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8018070 con 0x7f14001062d0 2023-09-12T16:05:24.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.842+0000 7f14086e1700 1 -- 172.21.15.130:0/1453967224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140019e390 con 0x7f14001062d0 2023-09-12T16:05:24.849 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.842+0000 7f13f77fe700 1 -- 172.21.15.130:0/1453967224 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f800f040 con 0x7f14001062d0 2023-09-12T16:05:24.850 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.842+0000 7f13f77fe700 1 -- 172.21.15.130:0/1453967224 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f800cd40 con 0x7f14001062d0 2023-09-12T16:05:24.850 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.842+0000 7f14086e1700 1 -- 172.21.15.130:0/1453967224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140019e8a0 con 0x7f14001062d0 2023-09-12T16:05:24.850 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.844+0000 7f13f77fe700 1 -- 172.21.15.130:0/1453967224 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f13f8003a00 con 0x7f14001062d0 2023-09-12T16:05:24.850 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.844+0000 7f13f77fe700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f13ec0a0640 0x7f13ec0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.850 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.845+0000 7f13f77fe700 1 -- 172.21.15.130:0/1453967224 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f13f80cbb30 con 0x7f14001062d0 2023-09-12T16:05:24.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.845+0000 7f140647d700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f13ec0a0640 0x7f13ec0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.845+0000 7f140647d700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f13ec0a0640 0x7f13ec0a2b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00b2e0 tx=0x7f13fc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:24.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.845+0000 7f14148a7b80 1 Processor -- start 2023-09-12T16:05:24.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f14148a7b80 1 -- start start 2023-09-12T16:05:24.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab07020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab05800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.857 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f14148a7b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4aa5e730 con 0x559f4ab075a0 2023-09-12T16:05:24.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f14148a7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4aa5c050 con 0x559f4ab05e00 2023-09-12T16:05:24.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f14148a7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4aa57940 con 0x559f4ab06c00 2023-09-12T16:05:24.858 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f140647d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab07020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f1405c7c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab05800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f140647d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab07020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35352/0 (socket says 172.21.15.130:35352) 2023-09-12T16:05:24.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f1406c7e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f140647d700 1 -- 172.21.15.130:0/3503961395 learned_addr learned my addr 172.21.15.130:0/3503961395 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:24.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f1405c7c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab05800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55742/0 (socket says 172.21.15.130:55742) 2023-09-12T16:05:24.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f140647d700 1 -- 172.21.15.130:0/3503961395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 msgr2=0x559f4ab062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:24.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f140647d700 1 --2- 172.21.15.130:0/3503961395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab062b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f140647d700 1 -- 172.21.15.130:0/3503961395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 msgr2=0x559f4ab05800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:24.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f140647d700 1 --2- 172.21.15.130:0/3503961395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab05800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f140647d700 1 -- 172.21.15.130:0/3503961395 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f4a883de0 con 0x559f4ab06c00 2023-09-12T16:05:24.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f1405c7c700 1 --2- 172.21.15.130:0/3503961395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab05800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:05:24.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f1406c7e700 1 --2- 172.21.15.130:0/3503961395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:05:24.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.846+0000 7f140647d700 1 --2- 172.21.15.130:0/3503961395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab07020 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005850 tx=0x7f13fc020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef9a655744bace4 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:24.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f13f4ff9700 1 -- 172.21.15.130:0/3503961395 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc02b070 con 0x559f4ab06c00 2023-09-12T16:05:24.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f13f4ff9700 1 -- 172.21.15.130:0/3503961395 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13fc01e040 con 0x559f4ab06c00 2023-09-12T16:05:24.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f13f4ff9700 1 -- 172.21.15.130:0/3503961395 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc02f400 con 0x559f4ab06c00 2023-09-12T16:05:24.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f14148a7b80 1 -- 172.21.15.130:0/3503961395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 msgr2=0x559f4ab07020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:24.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3503961395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab07020 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005850 tx=0x7f13fc020000 comp rx=0 tx=0).stop 2023-09-12T16:05:24.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f14148a7b80 1 -- 172.21.15.130:0/3503961395 shutdown_connections 2023-09-12T16:05:24.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3503961395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3503961395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab07020 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3503961395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab05800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.847+0000 7f14148a7b80 1 -- 172.21.15.130:0/3503961395 >> 172.21.15.130:0/3503961395 conn(0x559f4aa7a090 msgr2=0x559f4ab00720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:24.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 -- 172.21.15.130:0/3503961395 shutdown_connections 2023-09-12T16:05:24.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 -- 172.21.15.130:0/3503961395 wait complete. 2023-09-12T16:05:24.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 -- start start 2023-09-12T16:05:24.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 Processor -- start 2023-09-12T16:05:24.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab0f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab0fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab075a0 0x559f4ab16a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4a9f5b70 con 0x559f4ab05e00 2023-09-12T16:05:24.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4a9f5870 con 0x559f4ab075a0 2023-09-12T16:05:24.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4a8d9920 con 0x559f4ab06c00 2023-09-12T16:05:24.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f14148a7b80 10 client.0 _ll_register_callbacks cb 0x559f4a93fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:05:24.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f140647d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab0f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f1405c7c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab0fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f140647d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab0f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55746/0 (socket says 172.21.15.130:55746) 2023-09-12T16:05:24.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f140647d700 1 -- 172.21.15.130:0/1060541794 learned_addr learned my addr 172.21.15.130:0/1060541794 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:24.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.848+0000 7f1406c7e700 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab075a0 0x559f4ab16a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f140647d700 1 -- 172.21.15.130:0/1060541794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 msgr2=0x559f4ab0fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:24.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f140647d700 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab0fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f140647d700 1 -- 172.21.15.130:0/1060541794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab075a0 msgr2=0x559f4ab16a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:24.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f140647d700 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab075a0 0x559f4ab16a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f140647d700 1 -- 172.21.15.130:0/1060541794 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140019e8a0 con 0x559f4ab05e00 2023-09-12T16:05:24.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f1406c7e700 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab075a0 0x559f4ab16a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:05:24.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f1405c7c700 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab0fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:05:24.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f140647d700 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab0f500 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f13fc031040 tx=0x7f13fc024670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce95b28251d343d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:24.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f13e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:05:24.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f13e9ffb700 1 -- 172.21.15.130:0/1060541794 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc02b040 con 0x559f4ab05e00 2023-09-12T16:05:24.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f13e9ffb700 1 -- 172.21.15.130:0/1060541794 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f13fc02f400 con 0x559f4ab05e00 2023-09-12T16:05:24.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f14148a7b80 1 -- 172.21.15.130:0/1060541794 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f4a9af4f0 con 0x559f4ab05e00 2023-09-12T16:05:24.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f13e9ffb700 1 -- 172.21.15.130:0/1060541794 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13fc02f890 con 0x559f4ab05e00 2023-09-12T16:05:24.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f14148a7b80 10 client.15900 fetch_fsmap learned FSMap version 28 2023-09-12T16:05:24.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f14148a7b80 1 -- 172.21.15.130:0/1060541794 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f13f00098e0 con 0x559f4ab05e00 2023-09-12T16:05:24.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.849+0000 7f13e9ffb700 1 -- 172.21.15.130:0/1060541794 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f4a9af4f0 con 0x559f4ab05e00 2023-09-12T16:05:24.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f13e9ffb700 1 -- 172.21.15.130:0/1060541794 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f13fc00f4b0 con 0x559f4ab05e00 2023-09-12T16:05:24.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f14148a7b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:05:24.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f14148a7b80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:05:24.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f14148a7b80 20 client.15900 populate_metadata read hostname 'smithi130' 2023-09-12T16:05:24.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f14148a7b80 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f4ab1ae50 0x559f4ab1d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f14148a7b80 4 client.15900 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:05:24.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f14148a7b80 1 -- 172.21.15.130:0/1060541794 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f4aa6c8a0 con 0x559f4ab1ae50 2023-09-12T16:05:24.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f1405c7c700 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f4ab1ae50 0x559f4ab1d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f13e9ffb700 10 client.15900 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:05:24.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.850+0000 7f1405c7c700 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f4ab1ae50 0x559f4ab1d270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf21ab281a6d781 server_cookie=6edff872a8e22292 in_seq=0 out_seq=0 2023-09-12T16:05:24.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f13e9ffb700 1 -- 172.21.15.130:0/1060541794 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559f4aa6c8a0 con 0x559f4ab1ae50 2023-09-12T16:05:24.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f13e9ffb700 10 client.15900 handle_command_reply: tid=0 2023-09-12T16:05:24.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 client.15900 shutdown 2023-09-12T16:05:24.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 -- 172.21.15.130:0/1060541794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 msgr2=0x559f4ab0f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:24.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab0f500 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f13fc031040 tx=0x7f13fc024670 comp rx=0 tx=0).stop 2023-09-12T16:05:24.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 -- 172.21.15.130:0/1060541794 shutdown_connections 2023-09-12T16:05:24.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f4ab1ae50 0x559f4ab1d270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab075a0 0x559f4ab16a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab0f500 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f13e9ffb700 0 client.15900 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:05:24.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 --2- 172.21.15.130:0/1060541794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab0fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 -- 172.21.15.130:0/1060541794 >> 172.21.15.130:0/1060541794 conn(0x559f4aa7a090 msgr2=0x559f4aaffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:24.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 -- 172.21.15.130:0/1060541794 shutdown_connections 2023-09-12T16:05:24.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 1 -- 172.21.15.130:0/1060541794 wait complete. 2023-09-12T16:05:24.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.851+0000 7f14148a7b80 20 client.15900 trim_cache size 0 max 16384 2023-09-12T16:05:24.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f14148a7b80 1 Processor -- start 2023-09-12T16:05:24.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f14148a7b80 1 -- start start 2023-09-12T16:05:24.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab05710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab06c00 0x559f4ab17370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab075a0 0x559f4ab09eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f14148a7b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab05c50 con 0x559f4ab05e00 2023-09-12T16:05:24.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f14148a7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4aaca850 con 0x559f4ab06c00 2023-09-12T16:05:24.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f14148a7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab16e30 con 0x559f4ab075a0 2023-09-12T16:05:24.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f1406c7e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab075a0 0x559f4ab09eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f1405c7c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab06c00 0x559f4ab17370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.868+0000 7f140647d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab05710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f1406c7e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab075a0 0x559f4ab09eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35370/0 (socket says 172.21.15.130:35370) 2023-09-12T16:05:24.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f140647d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab05710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55754/0 (socket says 172.21.15.130:55754) 2023-09-12T16:05:24.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f1406c7e700 1 -- 172.21.15.130:0/3628930807 learned_addr learned my addr 172.21.15.130:0/3628930807 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:24.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f1405c7c700 1 -- 172.21.15.130:0/3628930807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab075a0 msgr2=0x559f4ab09eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:24.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f1405c7c700 1 --2- 172.21.15.130:0/3628930807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab075a0 0x559f4ab09eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f1405c7c700 1 -- 172.21.15.130:0/3628930807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 msgr2=0x559f4ab05710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:24.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f1405c7c700 1 --2- 172.21.15.130:0/3628930807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab05710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f1405c7c700 1 -- 172.21.15.130:0/3628930807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1400109330 con 0x559f4ab06c00 2023-09-12T16:05:24.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f1405c7c700 1 --2- 172.21.15.130:0/3628930807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab06c00 0x559f4ab17370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009350 tx=0x7f13f0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2196cbe5e2272 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:24.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f13f57fa700 1 -- 172.21.15.130:0/3628930807 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0017040 con 0x559f4ab06c00 2023-09-12T16:05:24.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f13f57fa700 1 -- 172.21.15.130:0/3628930807 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f0018040 con 0x559f4ab06c00 2023-09-12T16:05:24.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.869+0000 7f13f57fa700 1 -- 172.21.15.130:0/3628930807 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0010960 con 0x559f4ab06c00 2023-09-12T16:05:24.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 -- 172.21.15.130:0/3628930807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab06c00 msgr2=0x559f4ab17370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:24.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3628930807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab06c00 0x559f4ab17370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009350 tx=0x7f13f0009260 comp rx=0 tx=0).stop 2023-09-12T16:05:24.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 -- 172.21.15.130:0/3628930807 shutdown_connections 2023-09-12T16:05:24.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3628930807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab075a0 0x559f4ab09eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3628930807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab05e00 0x559f4ab05710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3628930807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab06c00 0x559f4ab17370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 -- 172.21.15.130:0/3628930807 >> 172.21.15.130:0/3628930807 conn(0x559f4aa7a090 msgr2=0x559f4ab006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:24.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 -- 172.21.15.130:0/3628930807 shutdown_connections 2023-09-12T16:05:24.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 -- 172.21.15.130:0/3628930807 wait complete. 2023-09-12T16:05:24.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 -- start start 2023-09-12T16:05:24.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.870+0000 7f14148a7b80 1 Processor -- start 2023-09-12T16:05:24.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab6b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab6b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f14148a7b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f14148a7b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab72f00 con 0x559f4ab075a0 2023-09-12T16:05:24.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f14148a7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab73040 con 0x559f4ab05e00 2023-09-12T16:05:24.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f14148a7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f4ab731d0 con 0x559f4ab06c00 2023-09-12T16:05:24.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f14148a7b80 10 client.0 _ll_register_callbacks cb 0x7f13f0009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:05:24.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f140647d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab6b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f1405c7c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab6b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f140647d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab6b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39018/0 (socket says 172.21.15.130:39018) 2023-09-12T16:05:24.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f1406c7e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f140647d700 1 -- 172.21.15.130:0/3488148358 learned_addr learned my addr 172.21.15.130:0/3488148358 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:24.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f140647d700 1 -- 172.21.15.130:0/3488148358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 msgr2=0x559f4ab6b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:24.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f140647d700 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab6b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f140647d700 1 -- 172.21.15.130:0/3488148358 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 msgr2=0x559f4ab729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:24.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f140647d700 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f140647d700 1 -- 172.21.15.130:0/3488148358 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f0012570 con 0x559f4ab05e00 2023-09-12T16:05:24.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f1406c7e700 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:05:24.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f140647d700 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab6b440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f13fc000c00 tx=0x7f13fc0092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3a64554e38e0392 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:24.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f13e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:05:24.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f13e9ffb700 1 -- 172.21.15.130:0/3488148358 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc02b070 con 0x559f4ab05e00 2023-09-12T16:05:24.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f14148a7b80 1 -- 172.21.15.130:0/3488148358 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13fc01e040 con 0x559f4ab05e00 2023-09-12T16:05:24.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.871+0000 7f13e9ffb700 1 -- 172.21.15.130:0/3488148358 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f13fc017c90 con 0x559f4ab05e00 2023-09-12T16:05:24.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f13e9ffb700 1 -- 172.21.15.130:0/3488148358 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13fc0246e0 con 0x559f4ab05e00 2023-09-12T16:05:24.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f14148a7b80 10 client.25373 fetch_fsmap learned FSMap version 28 2023-09-12T16:05:24.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f14148a7b80 1 -- 172.21.15.130:0/3488148358 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559f4a86cc60 con 0x559f4ab05e00 2023-09-12T16:05:24.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f13e9ffb700 1 -- 172.21.15.130:0/3488148358 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13fc01e040 con 0x559f4ab05e00 2023-09-12T16:05:24.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f13e9ffb700 1 -- 172.21.15.130:0/3488148358 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f13fc041030 con 0x559f4ab05e00 2023-09-12T16:05:24.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f14148a7b80 10 client.25373 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:05:24.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f14148a7b80 10 client.25373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:05:24.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f14148a7b80 20 client.25373 populate_metadata read hostname 'smithi130' 2023-09-12T16:05:24.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f4ab74820 0x559f4ab76c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:24.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f14148a7b80 4 client.25373 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:05:24.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f14148a7b80 1 -- 172.21.15.130:0/3488148358 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559f4aa50920 con 0x559f4ab74820 2023-09-12T16:05:24.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.872+0000 7f1405c7c700 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f4ab74820 0x559f4ab76c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:24.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f1405c7c700 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f4ab74820 0x559f4ab76c40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7423e827f26361a server_cookie=c6324b6d0725f04c in_seq=0 out_seq=0 2023-09-12T16:05:24.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f13e9ffb700 10 client.25373 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:05:24.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f13e9ffb700 1 -- 172.21.15.130:0/3488148358 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559f4aa50920 con 0x559f4ab74820 2023-09-12T16:05:24.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f13e9ffb700 10 client.25373 handle_command_reply: tid=0 2023-09-12T16:05:24.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 client.25373 shutdown 2023-09-12T16:05:24.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 -- 172.21.15.130:0/3488148358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 msgr2=0x559f4ab6b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:24.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab6b440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f13fc000c00 tx=0x7f13fc0092b0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 -- 172.21.15.130:0/3488148358 shutdown_connections 2023-09-12T16:05:24.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f4ab74820 0x559f4ab76c40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f4ab075a0 0x559f4ab729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f13e9ffb700 0 client.25373 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:05:24.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f4ab05e00 0x559f4ab6b440 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 --2- 172.21.15.130:0/3488148358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f4ab06c00 0x559f4ab6b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 -- 172.21.15.130:0/3488148358 >> 172.21.15.130:0/3488148358 conn(0x559f4aa7a090 msgr2=0x559f4ab17370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:24.895 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 -- 172.21.15.130:0/3488148358 shutdown_connections 2023-09-12T16:05:24.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 1 -- 172.21.15.130:0/3488148358 wait complete. 2023-09-12T16:05:24.896 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14148a7b80 20 client.25373 trim_cache size 0 max 16384 2023-09-12T16:05:24.896 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:05:24.896 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:05:24.897 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:05:24.897 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:05:24.897 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:05:24.897 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:05:24.897 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:05:24.898 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:05:24.898 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:05:24.898 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:05:24.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.873+0000 7f14086e1700 1 -- 172.21.15.130:0/1453967224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f13ec0a0640 msgr2=0x7f13ec0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:24.899 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f13ec0a0640 0x7f13ec0a2b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00b2e0 tx=0x7f13fc005960 comp rx=0 tx=0).stop 2023-09-12T16:05:24.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 -- 172.21.15.130:0/1453967224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14001062d0 msgr2=0x7f140019d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:24.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14001062d0 0x7f140019d870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f13f800efc0 tx=0x7f13f800c3a0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 -- 172.21.15.130:0/1453967224 shutdown_connections 2023-09-12T16:05:24.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f13ec0a0640 0x7f13ec0a2b00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14001062d0 0x7f140019d870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1400101070 0x7f14001962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 --2- 172.21.15.130:0/1453967224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f14001039a0 0x7f1400196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:24.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 -- 172.21.15.130:0/1453967224 >> 172.21.15.130:0/1453967224 conn(0x7f14000faa60 msgr2=0x7f1400101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:24.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 -- 172.21.15.130:0/1453967224 shutdown_connections 2023-09-12T16:05:24.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:24.874+0000 7f14086e1700 1 -- 172.21.15.130:0/1453967224 wait complete. 2023-09-12T16:05:24.902 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:05:24.903 INFO:teuthology.orchestra.run.smithi130.stderr:set primary keys... 2023-09-12T16:05:26.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:26 smithi130 ceph-mon[98670]: pgmap v367: 129 pgs: 129 active+clean; 7.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 331 MiB/s wr, 132 op/s 2023-09-12T16:05:26.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:26 smithi159 ceph-mon[102522]: pgmap v367: 129 pgs: 129 active+clean; 7.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 331 MiB/s wr, 132 op/s 2023-09-12T16:05:26.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:26 smithi203 ceph-mon[103431]: pgmap v367: 129 pgs: 129 active+clean; 7.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 331 MiB/s wr, 132 op/s 2023-09-12T16:05:27.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:27 smithi159 ceph-mon[102522]: pgmap v368: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 358 MiB/s wr, 138 op/s 2023-09-12T16:05:27.987 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:27 smithi130 ceph-mon[98670]: pgmap v368: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 358 MiB/s wr, 138 op/s 2023-09-12T16:05:27.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:27 smithi203 ceph-mon[103431]: pgmap v368: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 358 MiB/s wr, 138 op/s 2023-09-12T16:05:30.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:29 smithi130 ceph-mon[98670]: pgmap v369: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 349 MiB/s wr, 136 op/s 2023-09-12T16:05:30.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:29 smithi159 ceph-mon[102522]: pgmap v369: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 349 MiB/s wr, 136 op/s 2023-09-12T16:05:30.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:29 smithi203 ceph-mon[103431]: pgmap v369: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 349 MiB/s wr, 136 op/s 2023-09-12T16:05:32.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:31 smithi159 ceph-mon[102522]: pgmap v370: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 350 MiB/s wr, 104 op/s 2023-09-12T16:05:32.243 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:31 smithi130 ceph-mon[98670]: pgmap v370: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 350 MiB/s wr, 104 op/s 2023-09-12T16:05:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:31 smithi203 ceph-mon[103431]: pgmap v370: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 350 MiB/s wr, 104 op/s 2023-09-12T16:05:34.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:33 smithi130 ceph-mon[98670]: pgmap v371: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 281 MiB/s wr, 87 op/s 2023-09-12T16:05:34.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:33 smithi159 ceph-mon[102522]: pgmap v371: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 281 MiB/s wr, 87 op/s 2023-09-12T16:05:34.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:33 smithi203 ceph-mon[103431]: pgmap v371: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 281 MiB/s wr, 87 op/s 2023-09-12T16:05:36.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:35 smithi130 ceph-mon[98670]: pgmap v372: 129 pgs: 129 active+clean; 7.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 179 MiB/s wr, 58 op/s 2023-09-12T16:05:36.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:35 smithi159 ceph-mon[102522]: pgmap v372: 129 pgs: 129 active+clean; 7.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 179 MiB/s wr, 58 op/s 2023-09-12T16:05:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:35 smithi203 ceph-mon[103431]: pgmap v372: 129 pgs: 129 active+clean; 7.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 179 MiB/s wr, 58 op/s 2023-09-12T16:05:37.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:37 smithi130 ceph-mon[98670]: pgmap v373: 129 pgs: 129 active+clean; 7.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 287 MiB/s wr, 87 op/s 2023-09-12T16:05:37.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:37 smithi159 ceph-mon[102522]: pgmap v373: 129 pgs: 129 active+clean; 7.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 287 MiB/s wr, 87 op/s 2023-09-12T16:05:37.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:37 smithi203 ceph-mon[103431]: pgmap v373: 129 pgs: 129 active+clean; 7.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 287 MiB/s wr, 87 op/s 2023-09-12T16:05:40.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:39 smithi130 ceph-mon[98670]: pgmap v374: 129 pgs: 129 active+clean; 8.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 282 MiB/s wr, 86 op/s 2023-09-12T16:05:40.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:39 smithi159 ceph-mon[102522]: pgmap v374: 129 pgs: 129 active+clean; 8.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 282 MiB/s wr, 86 op/s 2023-09-12T16:05:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:39 smithi203 ceph-mon[103431]: pgmap v374: 129 pgs: 129 active+clean; 8.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 282 MiB/s wr, 86 op/s 2023-09-12T16:05:42.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:41 smithi130 ceph-mon[98670]: pgmap v375: 129 pgs: 129 active+clean; 8.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 299 MiB/s wr, 115 op/s 2023-09-12T16:05:42.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:41 smithi159 ceph-mon[102522]: pgmap v375: 129 pgs: 129 active+clean; 8.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 299 MiB/s wr, 115 op/s 2023-09-12T16:05:42.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:41 smithi203 ceph-mon[103431]: pgmap v375: 129 pgs: 129 active+clean; 8.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 299 MiB/s wr, 115 op/s 2023-09-12T16:05:44.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:43 smithi130 ceph-mon[98670]: pgmap v376: 129 pgs: 129 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 294 MiB/s wr, 164 op/s 2023-09-12T16:05:44.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:43 smithi159 ceph-mon[102522]: pgmap v376: 129 pgs: 129 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 294 MiB/s wr, 164 op/s 2023-09-12T16:05:44.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:43 smithi203 ceph-mon[103431]: pgmap v376: 129 pgs: 129 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 294 MiB/s wr, 164 op/s 2023-09-12T16:05:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:45 smithi130 ceph-mon[98670]: pgmap v377: 129 pgs: 129 active+clean; 8.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 282 MiB/s wr, 159 op/s 2023-09-12T16:05:46.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:45 smithi159 ceph-mon[102522]: pgmap v377: 129 pgs: 129 active+clean; 8.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 282 MiB/s wr, 159 op/s 2023-09-12T16:05:46.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:45 smithi203 ceph-mon[103431]: pgmap v377: 129 pgs: 129 active+clean; 8.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 282 MiB/s wr, 159 op/s 2023-09-12T16:05:47.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:47 smithi130 ceph-mon[98670]: pgmap v378: 129 pgs: 129 active+clean; 8.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 275 MiB/s wr, 242 op/s 2023-09-12T16:05:47.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:47 smithi159 ceph-mon[102522]: pgmap v378: 129 pgs: 129 active+clean; 8.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 275 MiB/s wr, 242 op/s 2023-09-12T16:05:47.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:47 smithi203 ceph-mon[103431]: pgmap v378: 129 pgs: 129 active+clean; 8.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 275 MiB/s wr, 242 op/s 2023-09-12T16:05:49.945 INFO:teuthology.orchestra.run.smithi130.stderr:done. 2023-09-12T16:05:49.949 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-09-12T16:05:50.086 INFO:teuthology.orchestra.run.smithi130.stderr:starting vacuum...end. 2023-09-12T16:05:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:49 smithi130 ceph-mon[98670]: pgmap v379: 129 pgs: 129 active+clean; 8.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 121 MiB/s wr, 220 op/s 2023-09-12T16:05:50.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:49 smithi159 ceph-mon[102522]: pgmap v379: 129 pgs: 129 active+clean; 8.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 121 MiB/s wr, 220 op/s 2023-09-12T16:05:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:49 smithi203 ceph-mon[103431]: pgmap v379: 129 pgs: 129 active+clean; 8.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 121 MiB/s wr, 220 op/s 2023-09-12T16:05:52.224 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:51 smithi130 ceph-mon[98670]: pgmap v380: 129 pgs: 129 active+clean; 8.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 126 MiB/s wr, 260 op/s 2023-09-12T16:05:52.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:51 smithi159 ceph-mon[102522]: pgmap v380: 129 pgs: 129 active+clean; 8.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 126 MiB/s wr, 260 op/s 2023-09-12T16:05:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:51 smithi203 ceph-mon[103431]: pgmap v380: 129 pgs: 129 active+clean; 8.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 126 MiB/s wr, 260 op/s 2023-09-12T16:05:54.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:53 smithi159 ceph-mon[102522]: pgmap v381: 129 pgs: 129 active+clean; 8.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 149 MiB/s wr, 284 op/s 2023-09-12T16:05:54.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:53 smithi130 ceph-mon[98670]: pgmap v381: 129 pgs: 129 active+clean; 8.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 149 MiB/s wr, 284 op/s 2023-09-12T16:05:54.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:53 smithi203 ceph-mon[103431]: pgmap v381: 129 pgs: 129 active+clean; 8.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 149 MiB/s wr, 284 op/s 2023-09-12T16:05:54.903 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:05:55.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 5.0 s, 4265.2 tps, lat 23.067 ms stddev 36.259 2023-09-12T16:05:55.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.150+0000 7fc08f5ec700 1 -- 172.21.15.130:0/3428242322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc088107600 msgr2=0x7fc088109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:55.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.150+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/3428242322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc088107600 0x7fc088109a30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc08000b0a0 tx=0x7fc08000b3b0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.152+0000 7fc08f5ec700 1 -- 172.21.15.130:0/3428242322 shutdown_connections 2023-09-12T16:05:55.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.152+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/3428242322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc08810c940 0x7fc08810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.152+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/3428242322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc088107600 0x7fc088109a30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.152+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/3428242322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc088109f70 0x7fc08810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.152+0000 7fc08f5ec700 1 -- 172.21.15.130:0/3428242322 >> 172.21.15.130:0/3428242322 conn(0x7fc08806d3b0 msgr2=0x7fc08806f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:55.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.156+0000 7fc08f5ec700 1 -- 172.21.15.130:0/3428242322 shutdown_connections 2023-09-12T16:05:55.158 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.159+0000 7fc08f5ec700 1 -- 172.21.15.130:0/3428242322 wait complete. 2023-09-12T16:05:55.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08f5ec700 1 Processor -- start 2023-09-12T16:05:55.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08f5ec700 1 -- start start 2023-09-12T16:05:55.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08f5ec700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc088107600 0x7fc088116c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08f5ec700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc088109f70 0x7fc088117190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08f5ec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc08810c940 0x7fc0881181d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08f5ec700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881187a0 con 0x7fc088107600 2023-09-12T16:05:55.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08f5ec700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc088118910 con 0x7fc088109f70 2023-09-12T16:05:55.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08f5ec700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc088118ad0 con 0x7fc08810c940 2023-09-12T16:05:55.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08d388700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc088107600 0x7fc088116c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08cb87700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc088109f70 0x7fc088117190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08d388700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc088107600 0x7fc088116c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55772/0 (socket says 172.21.15.130:55772) 2023-09-12T16:05:55.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08d388700 1 -- 172.21.15.130:0/2025891017 learned_addr learned my addr 172.21.15.130:0/2025891017 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:55.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08db89700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc08810c940 0x7fc0881181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.160+0000 7fc08cb87700 1 -- 172.21.15.130:0/2025891017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc08810c940 msgr2=0x7fc0881181d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:55.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.161+0000 7fc08cb87700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc08810c940 0x7fc0881181d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.161+0000 7fc08cb87700 1 -- 172.21.15.130:0/2025891017 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc088107600 msgr2=0x7fc088116c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:55.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.161+0000 7fc08cb87700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc088107600 0x7fc088116c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.161+0000 7fc08cb87700 1 -- 172.21.15.130:0/2025891017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080009d00 con 0x7fc088109f70 2023-09-12T16:05:55.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.161+0000 7fc08cb87700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc088109f70 0x7fc088117190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc084006fd0 tx=0x7fc08400b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:55.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.161+0000 7fc07e7fc700 1 -- 172.21.15.130:0/2025891017 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084011830 con 0x7fc088109f70 2023-09-12T16:05:55.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.161+0000 7fc07e7fc700 1 -- 172.21.15.130:0/2025891017 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc084011e50 con 0x7fc088109f70 2023-09-12T16:05:55.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.161+0000 7fc07e7fc700 1 -- 172.21.15.130:0/2025891017 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084010d20 con 0x7fc088109f70 2023-09-12T16:05:55.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.162+0000 7fc08f5ec700 1 -- 172.21.15.130:0/2025891017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc088118d50 con 0x7fc088109f70 2023-09-12T16:05:55.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.162+0000 7fc08f5ec700 1 -- 172.21.15.130:0/2025891017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc088119290 con 0x7fc088109f70 2023-09-12T16:05:55.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.168+0000 7fc09b7b2b80 1 Processor -- start 2023-09-12T16:05:55.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc07e7fc700 1 -- 172.21.15.130:0/2025891017 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fc084010380 con 0x7fc088109f70 2023-09-12T16:05:55.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc09b7b2b80 1 -- start start 2023-09-12T16:05:55.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acc2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acc0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3ac19730 con 0x559f3acc0e00 2023-09-12T16:05:55.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3ac17050 con 0x559f3acc25a0 2023-09-12T16:05:55.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3ac12940 con 0x559f3acc1c00 2023-09-12T16:05:55.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc08db89700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acc2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc08db89700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acc2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35390/0 (socket says 172.21.15.130:35390) 2023-09-12T16:05:55.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.169+0000 7fc08db89700 1 -- 172.21.15.130:0/3742324481 learned_addr learned my addr 172.21.15.130:0/3742324481 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:55.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc08cb87700 1 --2- 172.21.15.130:0/3742324481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc08d388700 1 --2- 172.21.15.130:0/3742324481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acc0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc08db89700 1 -- 172.21.15.130:0/3742324481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 msgr2=0x559f3acc0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:55.172 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc08db89700 1 --2- 172.21.15.130:0/3742324481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acc0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc08db89700 1 -- 172.21.15.130:0/3742324481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 msgr2=0x559f3acc12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:55.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc08db89700 1 --2- 172.21.15.130:0/3742324481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc08db89700 1 -- 172.21.15.130:0/3742324481 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f3aa3ede0 con 0x559f3acc1c00 2023-09-12T16:05:55.173 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc07e7fc700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc0740a0690 0x7fc0740a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc07e7fc700 1 -- 172.21.15.130:0/2025891017 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc0840cb900 con 0x7fc088109f70 2023-09-12T16:05:55.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.170+0000 7fc08db89700 1 --2- 172.21.15.130:0/3742324481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acc2020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc078009a70 tx=0x7fc078009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2be4724096ac0f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:55.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.171+0000 7fc08d388700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc0740a0690 0x7fc0740a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.171+0000 7fc0737fe700 1 -- 172.21.15.130:0/3742324481 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078009570 con 0x559f3acc1c00 2023-09-12T16:05:55.174 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.171+0000 7fc0737fe700 1 -- 172.21.15.130:0/3742324481 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07800ed90 con 0x559f3acc1c00 2023-09-12T16:05:55.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.171+0000 7fc0737fe700 1 -- 172.21.15.130:0/3742324481 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780123d0 con 0x559f3acc1c00 2023-09-12T16:05:55.175 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.172+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/3742324481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 msgr2=0x559f3acc2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:55.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.172+0000 7fc08d388700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc0740a0690 0x7fc0740a2b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc080009fd0 tx=0x7fc0800096e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:55.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.172+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/3742324481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acc2020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc078009a70 tx=0x7fc078009d80 comp rx=0 tx=0).stop 2023-09-12T16:05:55.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.173+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/3742324481 shutdown_connections 2023-09-12T16:05:55.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.173+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/3742324481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.173+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/3742324481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acc2020 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.173+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/3742324481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acc0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.173+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/3742324481 >> 172.21.15.130:0/3742324481 conn(0x559f3ac35090 msgr2=0x559f3acbb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:55.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.174+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/3742324481 shutdown_connections 2023-09-12T16:05:55.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.179+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/3742324481 wait complete. 2023-09-12T16:05:55.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.179+0000 7fc09b7b2b80 1 -- start start 2023-09-12T16:05:55.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.180+0000 7fc09b7b2b80 1 Processor -- start 2023-09-12T16:05:55.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.184+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc0e00 0x559f3acca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.184+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3accaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.184+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acd1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.184+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3abb0b70 con 0x559f3acc1c00 2023-09-12T16:05:55.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.184+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3abb0870 con 0x559f3acc25a0 2023-09-12T16:05:55.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.184+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3aa94920 con 0x559f3acc0e00 2023-09-12T16:05:55.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.184+0000 7fc09b7b2b80 10 client.0 _ll_register_callbacks cb 0x559f3aafaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:05:55.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08d388700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc0e00 0x559f3acca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08d388700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc0e00 0x559f3acca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35400/0 (socket says 172.21.15.130:35400) 2023-09-12T16:05:55.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08d388700 1 -- 172.21.15.130:0/1960023278 learned_addr learned my addr 172.21.15.130:0/1960023278 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:55.186 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08cb87700 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3accaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08d388700 1 -- 172.21.15.130:0/1960023278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 msgr2=0x559f3acd1ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:05:55.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08d388700 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acd1ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08d388700 1 -- 172.21.15.130:0/1960023278 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 msgr2=0x559f3accaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:55.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08d388700 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3accaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.187 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08d388700 1 -- 172.21.15.130:0/1960023278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0780096d0 con 0x559f3acc0e00 2023-09-12T16:05:55.188 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.185+0000 7fc08d388700 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc0e00 0x559f3acca560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc08000b070 tx=0x7fc0800099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2aa3a2da0fdd44a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:55.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.191+0000 7fc070ff9700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:05:55.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.192+0000 7fc070ff9700 1 -- 172.21.15.130:0/1960023278 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000e040 con 0x559f3acc0e00 2023-09-12T16:05:55.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.192+0000 7fc070ff9700 1 -- 172.21.15.130:0/1960023278 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc080022070 con 0x559f3acc0e00 2023-09-12T16:05:55.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.192+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1960023278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f3ab6a4f0 con 0x559f3acc0e00 2023-09-12T16:05:55.193 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.192+0000 7fc070ff9700 1 -- 172.21.15.130:0/1960023278 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0800052c0 con 0x559f3acc0e00 2023-09-12T16:05:55.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.194+0000 7fc09b7b2b80 10 client.25579 fetch_fsmap learned FSMap version 28 2023-09-12T16:05:55.194 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.194+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1960023278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559f3aa27c60 con 0x559f3acc0e00 2023-09-12T16:05:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.195+0000 7fc070ff9700 1 -- 172.21.15.130:0/1960023278 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f3ab6a4f0 con 0x559f3acc0e00 2023-09-12T16:05:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.196+0000 7fc070ff9700 1 -- 172.21.15.130:0/1960023278 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc080029b00 con 0x559f3acc0e00 2023-09-12T16:05:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.196+0000 7fc09b7b2b80 10 client.25579 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:05:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.196+0000 7fc09b7b2b80 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:05:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.196+0000 7fc09b7b2b80 20 client.25579 populate_metadata read hostname 'smithi130' 2023-09-12T16:05:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.196+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f3acd6380 0x559f3acd87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.196+0000 7fc09b7b2b80 4 client.25579 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:05:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.196+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1960023278 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f3ac278a0 con 0x559f3acd6380 2023-09-12T16:05:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.197+0000 7fc08cb87700 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f3acd6380 0x559f3acd87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.197+0000 7fc08cb87700 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f3acd6380 0x559f3acd87a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e5a7a7bb2e2612 server_cookie=f92abc38e4b02a3 in_seq=0 out_seq=0 2023-09-12T16:05:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.197+0000 7fc070ff9700 10 client.25579 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:05:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.198+0000 7fc070ff9700 1 -- 172.21.15.130:0/1960023278 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559f3ac278a0 con 0x559f3acd6380 2023-09-12T16:05:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.198+0000 7fc070ff9700 10 client.25579 handle_command_reply: tid=0 2023-09-12T16:05:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.198+0000 7fc09b7b2b80 1 client.25579 shutdown 2023-09-12T16:05:55.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.198+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1960023278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc0e00 msgr2=0x559f3acca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:55.199 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.198+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc0e00 0x559f3acca560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc08000b070 tx=0x7fc0800099e0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.198+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1960023278 shutdown_connections 2023-09-12T16:05:55.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.198+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f3acd6380 0x559f3acd87a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.198+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acd1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.199+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc0e00 0x559f3acca560 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.199+0000 7fc070ff9700 0 client.25579 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:05:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.199+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1960023278 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3accaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.199+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1960023278 >> 172.21.15.130:0/1960023278 conn(0x559f3ac35090 msgr2=0x559f3acbadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.199+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1960023278 shutdown_connections 2023-09-12T16:05:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.199+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1960023278 wait complete. 2023-09-12T16:05:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.199+0000 7fc09b7b2b80 20 client.25579 trim_cache size 0 max 16384 2023-09-12T16:05:55.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.246+0000 7fc09b7b2b80 1 Processor -- start 2023-09-12T16:05:55.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.246+0000 7fc09b7b2b80 1 -- start start 2023-09-12T16:05:55.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.246+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.246+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acd1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.246+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acc4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.246+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3acd3780 con 0x559f3acc0e00 2023-09-12T16:05:55.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.246+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3acc0c50 con 0x559f3acc25a0 2023-09-12T16:05:55.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.246+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3ac85880 con 0x559f3acc1c00 2023-09-12T16:05:55.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.246+0000 7fc08d388700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08d388700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55794/0 (socket says 172.21.15.130:55794) 2023-09-12T16:05:55.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08db89700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acc4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08d388700 1 -- 172.21.15.130:0/1990506810 learned_addr learned my addr 172.21.15.130:0/1990506810 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:55.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08cb87700 1 --2- 172.21.15.130:0/1990506810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acd1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08d388700 1 -- 172.21.15.130:0/1990506810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 msgr2=0x559f3acd1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:55.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08d388700 1 --2- 172.21.15.130:0/1990506810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acd1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08d388700 1 -- 172.21.15.130:0/1990506810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 msgr2=0x559f3acc4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:55.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08d388700 1 --2- 172.21.15.130:0/1990506810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acc4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08d388700 1 -- 172.21.15.130:0/1990506810 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f3aa65250 con 0x559f3acc0e00 2023-09-12T16:05:55.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08db89700 1 --2- 172.21.15.130:0/1990506810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acc4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:05:55.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc08d388700 1 --2- 172.21.15.130:0/1990506810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc0710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc080029e30 tx=0x7fc080007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6045f2e1a356de server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:55.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc073fff700 1 -- 172.21.15.130:0/1990506810 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000e070 con 0x559f3acc0e00 2023-09-12T16:05:55.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc073fff700 1 -- 172.21.15.130:0/1990506810 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc080020070 con 0x559f3acc0e00 2023-09-12T16:05:55.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.247+0000 7fc073fff700 1 -- 172.21.15.130:0/1990506810 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080029b90 con 0x559f3acc0e00 2023-09-12T16:05:55.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.248+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1990506810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 msgr2=0x559f3acc0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:55.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.248+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1990506810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc0710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc080029e30 tx=0x7fc080007ee0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.248+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1990506810 shutdown_connections 2023-09-12T16:05:55.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.248+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1990506810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acc4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.248+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1990506810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc0e00 0x559f3acc0710 secure :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc080029e30 tx=0x7fc080007ee0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.248+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/1990506810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acc1c00 0x559f3acd1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.248+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1990506810 >> 172.21.15.130:0/1990506810 conn(0x559f3ac35090 msgr2=0x559f3acd2310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:55.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.250+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1990506810 shutdown_connections 2023-09-12T16:05:55.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.252+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/1990506810 wait complete. 2023-09-12T16:05:55.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.252+0000 7fc09b7b2b80 1 -- start start 2023-09-12T16:05:55.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.252+0000 7fc09b7b2b80 1 Processor -- start 2023-09-12T16:05:55.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3acd1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc09b7b2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acce6e0 0x559f3ad2a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3acd1e80 con 0x559f3acc1c00 2023-09-12T16:05:55.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3acceb90 con 0x559f3acc25a0 2023-09-12T16:05:55.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc09b7b2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3acced50 con 0x559f3acce6e0 2023-09-12T16:05:55.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc09b7b2b80 10 client.0 _ll_register_callbacks cb 0x7fc080003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:05:55.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08d388700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3acd1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08db89700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acce6e0 0x559f3ad2a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08d388700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3acd1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55800/0 (socket says 172.21.15.130:55800) 2023-09-12T16:05:55.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08cb87700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08d388700 1 -- 172.21.15.130:0/2030240846 learned_addr learned my addr 172.21.15.130:0/2030240846 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:05:55.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08d388700 1 -- 172.21.15.130:0/2030240846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acce6e0 msgr2=0x559f3ad2a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:55.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08d388700 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acce6e0 0x559f3ad2a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08d388700 1 -- 172.21.15.130:0/2030240846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 msgr2=0x559f3acce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:05:55.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08d388700 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08d388700 1 -- 172.21.15.130:0/2030240846 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f3aa3ede0 con 0x559f3acc1c00 2023-09-12T16:05:55.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08cb87700 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:05:55.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc08d388700 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3acd1700 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc080007ba0 tx=0x7fc080029f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34990caf85bc327b server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:05:55.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc070ff9700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:05:55.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc070ff9700 1 -- 172.21.15.130:0/2030240846 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000e040 con 0x559f3acc1c00 2023-09-12T16:05:55.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.253+0000 7fc070ff9700 1 -- 172.21.15.130:0/2030240846 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc080022070 con 0x559f3acc1c00 2023-09-12T16:05:55.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/2030240846 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc07800ed90 con 0x559f3acc1c00 2023-09-12T16:05:55.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc070ff9700 1 -- 172.21.15.130:0/2030240846 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc080029c20 con 0x559f3acc1c00 2023-09-12T16:05:55.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc09b7b2b80 10 client.15924 fetch_fsmap learned FSMap version 28 2023-09-12T16:05:55.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/2030240846 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559f3aa27c60 con 0x559f3acc1c00 2023-09-12T16:05:55.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc070ff9700 1 -- 172.21.15.130:0/2030240846 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08004d080 con 0x559f3acc1c00 2023-09-12T16:05:55.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc070ff9700 1 -- 172.21.15.130:0/2030240846 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 28) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc080041030 con 0x559f3acc1c00 2023-09-12T16:05:55.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc09b7b2b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 28 2023-09-12T16:05:55.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc09b7b2b80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:05:55.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc09b7b2b80 20 client.15924 populate_metadata read hostname 'smithi130' 2023-09-12T16:05:55.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f3ad2baa0 0x559f3ad2dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:05:55.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc09b7b2b80 4 client.15924 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:05:55.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/2030240846 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559f3ac0b920 con 0x559f3ad2baa0 2023-09-12T16:05:55.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.254+0000 7fc08cb87700 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f3ad2baa0 0x559f3ad2dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:05:55.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.255+0000 7fc070ff9700 10 client.15924 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:05:55.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.255+0000 7fc08cb87700 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f3ad2baa0 0x559f3ad2dec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28f2caab93e50560 server_cookie=7a144d3dad66d364 in_seq=0 out_seq=0 2023-09-12T16:05:55.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.255+0000 7fc070ff9700 1 -- 172.21.15.130:0/2030240846 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559f3ac0b920 con 0x559f3ad2baa0 2023-09-12T16:05:55.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.255+0000 7fc070ff9700 10 client.15924 handle_command_reply: tid=0 2023-09-12T16:05:55.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.255+0000 7fc09b7b2b80 1 client.15924 shutdown 2023-09-12T16:05:55.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.258+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/2030240846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 msgr2=0x559f3acd1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:55.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.258+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3acd1700 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc080007ba0 tx=0x7fc080029f40 comp rx=0 tx=0).stop 2023-09-12T16:05:55.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.259+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/2030240846 shutdown_connections 2023-09-12T16:05:55.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.259+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559f3ad2baa0 0x559f3ad2dec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.259+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f3acce6e0 0x559f3ad2a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.259+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f3acc1c00 0x559f3acd1700 secure :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc080007ba0 tx=0x7fc080029f40 comp rx=0 tx=0).stop 2023-09-12T16:05:55.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.259+0000 7fc09b7b2b80 1 --2- 172.21.15.130:0/2030240846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f3acc25a0 0x559f3acce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.259+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/2030240846 >> 172.21.15.130:0/2030240846 conn(0x559f3ac35090 msgr2=0x559f3acc4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:55.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.259+0000 7fc070ff9700 0 client.15924 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:05:55.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.259+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/2030240846 shutdown_connections 2023-09-12T16:05:55.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.260+0000 7fc09b7b2b80 1 -- 172.21.15.130:0/2030240846 wait complete. 2023-09-12T16:05:55.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.260+0000 7fc09b7b2b80 20 client.15924 trim_cache size 0 max 16384 2023-09-12T16:05:55.268 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:05:55.269 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:05:55.269 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:05:55.269 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:05:55.269 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:05:55.269 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:05:55.269 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:05:55.270 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:05:55.270 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:05:55.270 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:05:55.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 -- 172.21.15.130:0/2025891017 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc0740a0690 msgr2=0x7fc0740a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:55.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc0740a0690 0x7fc0740a2b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc080009fd0 tx=0x7fc0800096e0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 -- 172.21.15.130:0/2025891017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc088109f70 msgr2=0x7fc088117190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:05:55.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc088109f70 0x7fc088117190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc084006fd0 tx=0x7fc08400b730 comp rx=0 tx=0).stop 2023-09-12T16:05:55.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 -- 172.21.15.130:0/2025891017 shutdown_connections 2023-09-12T16:05:55.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc0740a0690 0x7fc0740a2b50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc08810c940 0x7fc0881181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc088107600 0x7fc088116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 --2- 172.21.15.130:0/2025891017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc088109f70 0x7fc088117190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:05:55.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 -- 172.21.15.130:0/2025891017 >> 172.21.15.130:0/2025891017 conn(0x7fc08806d3b0 msgr2=0x7fc08806f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:05:55.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 -- 172.21.15.130:0/2025891017 shutdown_connections 2023-09-12T16:05:55.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:05:55.261+0000 7fc08f5ec700 1 -- 172.21.15.130:0/2025891017 wait complete. 2023-09-12T16:05:55.285 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:05:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:55 smithi130 ceph-mon[98670]: pgmap v382: 129 pgs: 129 active+clean; 9.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 145 MiB/s wr, 255 op/s 2023-09-12T16:05:56.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:55 smithi159 ceph-mon[102522]: pgmap v382: 129 pgs: 129 active+clean; 9.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 145 MiB/s wr, 255 op/s 2023-09-12T16:05:56.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:55 smithi203 ceph-mon[103431]: pgmap v382: 129 pgs: 129 active+clean; 9.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 145 MiB/s wr, 255 op/s 2023-09-12T16:05:57.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:57 smithi159 ceph-mon[102522]: pgmap v383: 129 pgs: 129 active+clean; 9.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 156 MiB/s wr, 579 op/s 2023-09-12T16:05:57.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:57 smithi130 ceph-mon[98670]: pgmap v383: 129 pgs: 129 active+clean; 9.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 156 MiB/s wr, 579 op/s 2023-09-12T16:05:57.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:57 smithi203 ceph-mon[103431]: pgmap v383: 129 pgs: 129 active+clean; 9.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 156 MiB/s wr, 579 op/s 2023-09-12T16:06:00.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 10.0 s, 2211.7 tps, lat 43.433 ms stddev 137.429 2023-09-12T16:06:00.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:05:59 smithi159 ceph-mon[102522]: pgmap v384: 129 pgs: 129 active+clean; 9.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 142 MiB/s wr, 612 op/s 2023-09-12T16:06:00.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:05:59 smithi130 ceph-mon[98670]: pgmap v384: 129 pgs: 129 active+clean; 9.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 142 MiB/s wr, 612 op/s 2023-09-12T16:06:00.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:05:59 smithi203 ceph-mon[103431]: pgmap v384: 129 pgs: 129 active+clean; 9.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 142 MiB/s wr, 612 op/s 2023-09-12T16:06:02.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:01 smithi130 ceph-mon[98670]: pgmap v385: 129 pgs: 129 active+clean; 9.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 165 MiB/s wr, 1.61k op/s 2023-09-12T16:06:02.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:01 smithi159 ceph-mon[102522]: pgmap v385: 129 pgs: 129 active+clean; 9.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 165 MiB/s wr, 1.61k op/s 2023-09-12T16:06:02.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:01 smithi203 ceph-mon[103431]: pgmap v385: 129 pgs: 129 active+clean; 9.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 165 MiB/s wr, 1.61k op/s 2023-09-12T16:06:04.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:03 smithi130 ceph-mon[98670]: pgmap v386: 129 pgs: 129 active+clean; 8.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 139 MiB/s wr, 3.01k op/s 2023-09-12T16:06:04.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:03 smithi159 ceph-mon[102522]: pgmap v386: 129 pgs: 129 active+clean; 8.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 139 MiB/s wr, 3.01k op/s 2023-09-12T16:06:04.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:03 smithi203 ceph-mon[103431]: pgmap v386: 129 pgs: 129 active+clean; 8.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 139 MiB/s wr, 3.01k op/s 2023-09-12T16:06:05.086 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 15.0 s, 2863.7 tps, lat 36.393 ms stddev 75.430 2023-09-12T16:06:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:06 smithi159 ceph-mon[102522]: pgmap v387: 129 pgs: 129 active+clean; 8.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 95 MiB/s wr, 3.29k op/s 2023-09-12T16:06:06.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:06 smithi130 ceph-mon[98670]: pgmap v387: 129 pgs: 129 active+clean; 8.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 95 MiB/s wr, 3.29k op/s 2023-09-12T16:06:06.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:06 smithi203 ceph-mon[103431]: pgmap v387: 129 pgs: 129 active+clean; 8.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 95 MiB/s wr, 3.29k op/s 2023-09-12T16:06:07.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:07 smithi130 ceph-mon[98670]: pgmap v388: 129 pgs: 129 active+clean; 8.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 92 MiB/s wr, 3.85k op/s 2023-09-12T16:06:07.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:07 smithi159 ceph-mon[102522]: pgmap v388: 129 pgs: 129 active+clean; 8.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 92 MiB/s wr, 3.85k op/s 2023-09-12T16:06:07.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:07 smithi203 ceph-mon[103431]: pgmap v388: 129 pgs: 129 active+clean; 8.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 92 MiB/s wr, 3.85k op/s 2023-09-12T16:06:10.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 20.0 s, 2089.5 tps, lat 47.715 ms stddev 86.256 2023-09-12T16:06:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:09 smithi130 ceph-mon[98670]: pgmap v389: 129 pgs: 129 active+clean; 8.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 89 MiB/s wr, 3.83k op/s 2023-09-12T16:06:10.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:09 smithi159 ceph-mon[102522]: pgmap v389: 129 pgs: 129 active+clean; 8.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 89 MiB/s wr, 3.83k op/s 2023-09-12T16:06:10.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:09 smithi203 ceph-mon[103431]: pgmap v389: 129 pgs: 129 active+clean; 8.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 89 MiB/s wr, 3.83k op/s 2023-09-12T16:06:12.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:11 smithi203 ceph-mon[103431]: pgmap v390: 129 pgs: 129 active+clean; 8.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 98 MiB/s wr, 4.50k op/s 2023-09-12T16:06:12.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:11 smithi130 ceph-mon[98670]: pgmap v390: 129 pgs: 129 active+clean; 8.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 98 MiB/s wr, 4.50k op/s 2023-09-12T16:06:12.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:11 smithi159 ceph-mon[102522]: pgmap v390: 129 pgs: 129 active+clean; 8.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 98 MiB/s wr, 4.50k op/s 2023-09-12T16:06:14.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:14 smithi130 ceph-mon[98670]: pgmap v391: 129 pgs: 129 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 85 MiB/s wr, 4.62k op/s 2023-09-12T16:06:14.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:14 smithi159 ceph-mon[102522]: pgmap v391: 129 pgs: 129 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 85 MiB/s wr, 4.62k op/s 2023-09-12T16:06:14.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:14 smithi203 ceph-mon[103431]: pgmap v391: 129 pgs: 129 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 85 MiB/s wr, 4.62k op/s 2023-09-12T16:06:15.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 25.0 s, 1974.4 tps, lat 50.827 ms stddev 115.199 2023-09-12T16:06:15.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:15 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:06:15.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:15 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:06:15.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:15 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:06:15.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:15 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:06:15.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:15 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:06:15.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:15 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:06:16.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:16 smithi159 ceph-mon[102522]: pgmap v392: 129 pgs: 129 active+clean; 8.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 65 MiB/s wr, 3.59k op/s 2023-09-12T16:06:16.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:16 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:16 smithi130 ceph-mon[98670]: pgmap v392: 129 pgs: 129 active+clean; 8.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 65 MiB/s wr, 3.59k op/s 2023-09-12T16:06:16.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:16 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:16 smithi203 ceph-mon[103431]: pgmap v392: 129 pgs: 129 active+clean; 8.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 65 MiB/s wr, 3.59k op/s 2023-09-12T16:06:16.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:16.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:16 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:17.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:06:17.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:06:17.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:17.479 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:06:17.479 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:06:17.480 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:17.501 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:06:17.501 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:06:17.501 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:06:18.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:18 smithi130 ceph-mon[98670]: pgmap v393: 129 pgs: 129 active+clean; 8.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 72 MiB/s wr, 4.14k op/s 2023-09-12T16:06:18.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:18 smithi159 ceph-mon[102522]: pgmap v393: 129 pgs: 129 active+clean; 8.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 72 MiB/s wr, 4.14k op/s 2023-09-12T16:06:18.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:18 smithi203 ceph-mon[103431]: pgmap v393: 129 pgs: 129 active+clean; 8.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 72 MiB/s wr, 4.14k op/s 2023-09-12T16:06:20.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 30.0 s, 2702.7 tps, lat 36.670 ms stddev 70.637 2023-09-12T16:06:20.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:20 smithi159 ceph-mon[102522]: pgmap v394: 129 pgs: 129 active+clean; 8.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 73 MiB/s wr, 4.16k op/s 2023-09-12T16:06:20.479 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:20 smithi130 ceph-mon[98670]: pgmap v394: 129 pgs: 129 active+clean; 8.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 73 MiB/s wr, 4.16k op/s 2023-09-12T16:06:20.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:20 smithi203 ceph-mon[103431]: pgmap v394: 129 pgs: 129 active+clean; 8.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 73 MiB/s wr, 4.16k op/s 2023-09-12T16:06:22.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:22 smithi130 ceph-mon[98670]: pgmap v395: 129 pgs: 129 active+clean; 8.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 71 MiB/s wr, 4.13k op/s 2023-09-12T16:06:22.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:22 smithi159 ceph-mon[102522]: pgmap v395: 129 pgs: 129 active+clean; 8.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 71 MiB/s wr, 4.13k op/s 2023-09-12T16:06:22.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:22 smithi203 ceph-mon[103431]: pgmap v395: 129 pgs: 129 active+clean; 8.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 71 MiB/s wr, 4.13k op/s 2023-09-12T16:06:24.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:24 smithi130 ceph-mon[98670]: pgmap v396: 129 pgs: 129 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 67 MiB/s wr, 3.62k op/s 2023-09-12T16:06:24.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:24 smithi159 ceph-mon[102522]: pgmap v396: 129 pgs: 129 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 67 MiB/s wr, 3.62k op/s 2023-09-12T16:06:24.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:24 smithi203 ceph-mon[103431]: pgmap v396: 129 pgs: 129 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 67 MiB/s wr, 3.62k op/s 2023-09-12T16:06:25.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 35.0 s, 1635.8 tps, lat 61.502 ms stddev 250.929 2023-09-12T16:06:25.287 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:06:25.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.617+0000 7ff9df1a3700 1 -- 172.21.15.130:0/1171985218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8107600 msgr2=0x7ff9d8109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:25.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.617+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/1171985218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8107600 0x7ff9d8109a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009f90 tx=0x7ff9c800b330 comp rx=0 tx=0).stop 2023-09-12T16:06:25.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.618+0000 7ff9df1a3700 1 -- 172.21.15.130:0/1171985218 shutdown_connections 2023-09-12T16:06:25.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.618+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/1171985218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d810c940 0x7ff9d810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.618+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/1171985218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8107600 0x7ff9d8109a30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.618+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/1171985218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9d8109f70 0x7ff9d810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.618+0000 7ff9df1a3700 1 -- 172.21.15.130:0/1171985218 >> 172.21.15.130:0/1171985218 conn(0x7ff9d806d3b0 msgr2=0x7ff9d806f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:25.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.629+0000 7ff9df1a3700 1 -- 172.21.15.130:0/1171985218 shutdown_connections 2023-09-12T16:06:25.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.635+0000 7ff9df1a3700 1 -- 172.21.15.130:0/1171985218 wait complete. 2023-09-12T16:06:25.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.636+0000 7ff9df1a3700 1 Processor -- start 2023-09-12T16:06:25.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9df1a3700 1 -- start start 2023-09-12T16:06:25.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9df1a3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8107600 0x7ff9d8113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9df1a3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9d8109f70 0x7ff9d8114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9df1a3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d810c940 0x7ff9d81b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9df1a3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d8117160 con 0x7ff9d8109f70 2023-09-12T16:06:25.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9df1a3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d81172d0 con 0x7ff9d8107600 2023-09-12T16:06:25.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9df1a3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d81b58d0 con 0x7ff9d810c940 2023-09-12T16:06:25.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9dd740700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d810c940 0x7ff9d81b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9dd740700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d810c940 0x7ff9d81b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35424/0 (socket says 172.21.15.130:35424) 2023-09-12T16:06:25.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.637+0000 7ff9dd740700 1 -- 172.21.15.130:0/927363041 learned_addr learned my addr 172.21.15.130:0/927363041 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:25.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.639+0000 7ff9dd740700 1 -- 172.21.15.130:0/927363041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8107600 msgr2=0x7ff9d8113b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-12T16:06:25.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.639+0000 7ff9dd740700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8107600 0x7ff9d8113b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.639+0000 7ff9dd740700 1 -- 172.21.15.130:0/927363041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9d8109f70 msgr2=0x7ff9d8114080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:25.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.639+0000 7ff9dd740700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9d8109f70 0x7ff9d8114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.639+0000 7ff9dd740700 1 -- 172.21.15.130:0/927363041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8009c40 con 0x7ff9d810c940 2023-09-12T16:06:25.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.639+0000 7ff9dd740700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d810c940 0x7ff9d81b5390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000e970 tx=0x7ff9d000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:25.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.640+0000 7ff9d5ffb700 1 -- 172.21.15.130:0/927363041 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000cbd0 con 0x7ff9d810c940 2023-09-12T16:06:25.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.640+0000 7ff9df1a3700 1 -- 172.21.15.130:0/927363041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d81b5a70 con 0x7ff9d810c940 2023-09-12T16:06:25.643 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.640+0000 7ff9df1a3700 1 -- 172.21.15.130:0/927363041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d81b6000 con 0x7ff9d810c940 2023-09-12T16:06:25.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.646+0000 7ff9d5ffb700 1 -- 172.21.15.130:0/927363041 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9d000ee50 con 0x7ff9d810c940 2023-09-12T16:06:25.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.646+0000 7ff9d5ffb700 1 -- 172.21.15.130:0/927363041 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d00055c0 con 0x7ff9d810c940 2023-09-12T16:06:25.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.647+0000 7ff9eb369b80 1 Processor -- start 2023-09-12T16:06:25.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.651+0000 7ff9d5ffb700 1 -- 172.21.15.130:0/927363041 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7ff9d0021440 con 0x7ff9d810c940 2023-09-12T16:06:25.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.652+0000 7ff9eb369b80 1 -- start start 2023-09-12T16:06:25.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.652+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb537020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.652+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 0x55b0bb535800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.652+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb5362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.652+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb48e730 con 0x55b0bb536c00 2023-09-12T16:06:25.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.652+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb48c050 con 0x55b0bb5375a0 2023-09-12T16:06:25.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.652+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb487940 con 0x55b0bb535e00 2023-09-12T16:06:25.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.652+0000 7ff9d5ffb700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9c40a0790 0x7ff9c40a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.652+0000 7ff9d5ffb700 1 -- 172.21.15.130:0/927363041 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff9d0014070 con 0x7ff9d810c940 2023-09-12T16:06:25.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.654+0000 7ff9dcf3f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 0x55b0bb535800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.654+0000 7ff9dcf3f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 0x55b0bb535800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39076/0 (socket says 172.21.15.130:39076) 2023-09-12T16:06:25.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.654+0000 7ff9dcf3f700 1 -- 172.21.15.130:0/3784011089 learned_addr learned my addr 172.21.15.130:0/3784011089 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:25.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.659+0000 7ff9dd740700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9c40a0790 0x7ff9c40a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.659+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/3784011089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb537020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.659+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/3784011089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb5362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.667+0000 7ff9d7fff700 1 -- 172.21.15.130:0/3784011089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 msgr2=0x55b0bb535800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:25.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.667+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/3784011089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 0x55b0bb535800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.667+0000 7ff9d7fff700 1 -- 172.21.15.130:0/3784011089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 msgr2=0x55b0bb537020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:25.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.667+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/3784011089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb537020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.667+0000 7ff9d7fff700 1 -- 172.21.15.130:0/3784011089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0bb2b3de0 con 0x55b0bb535e00 2023-09-12T16:06:25.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.667+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/3784011089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb5362b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000d750 tx=0x7ff9c00069f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cff5a2d84f5941e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:25.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.671+0000 7ff9ceffd700 1 -- 172.21.15.130:0/3784011089 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c000de30 con 0x55b0bb535e00 2023-09-12T16:06:25.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.672+0000 7ff9ceffd700 1 -- 172.21.15.130:0/3784011089 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c0004de0 con 0x55b0bb535e00 2023-09-12T16:06:25.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.672+0000 7ff9ceffd700 1 -- 172.21.15.130:0/3784011089 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0006370 con 0x55b0bb535e00 2023-09-12T16:06:25.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.673+0000 7ff9eb369b80 1 -- 172.21.15.130:0/3784011089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 msgr2=0x55b0bb5362b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:25.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.673+0000 7ff9dd740700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9c40a0790 0x7ff9c40a2c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0096a10 tx=0x7ff9d0009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:25.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.673+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/3784011089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb5362b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000d750 tx=0x7ff9c00069f0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.673+0000 7ff9eb369b80 1 -- 172.21.15.130:0/3784011089 shutdown_connections 2023-09-12T16:06:25.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.673+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/3784011089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb5362b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.673+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/3784011089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb537020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.673+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/3784011089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 0x55b0bb535800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.673+0000 7ff9eb369b80 1 -- 172.21.15.130:0/3784011089 >> 172.21.15.130:0/3784011089 conn(0x55b0bb4aa090 msgr2=0x55b0bb530720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:25.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.673+0000 7ff9eb369b80 1 -- 172.21.15.130:0/3784011089 shutdown_connections 2023-09-12T16:06:25.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.674+0000 7ff9eb369b80 1 -- 172.21.15.130:0/3784011089 wait complete. 2023-09-12T16:06:25.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.675+0000 7ff9eb369b80 1 -- start start 2023-09-12T16:06:25.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.675+0000 7ff9eb369b80 1 Processor -- start 2023-09-12T16:06:25.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.677+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.677+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb536c00 0x55b0bb53fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.677+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb5375a0 0x55b0bb546c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.677+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb425b70 con 0x55b0bb5375a0 2023-09-12T16:06:25.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.677+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb425870 con 0x55b0bb536c00 2023-09-12T16:06:25.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.677+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb309920 con 0x55b0bb535e00 2023-09-12T16:06:25.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.677+0000 7ff9eb369b80 10 client.0 _ll_register_callbacks cb 0x55b0bb36fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:06:25.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.678+0000 7ff9d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.678+0000 7ff9d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35438/0 (socket says 172.21.15.130:35438) 2023-09-12T16:06:25.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.678+0000 7ff9d7fff700 1 -- 172.21.15.130:0/1630678221 learned_addr learned my addr 172.21.15.130:0/1630678221 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:25.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.681+0000 7ff9d7fff700 1 -- 172.21.15.130:0/1630678221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb536c00 msgr2=0x55b0bb53fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:25.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.681+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb536c00 0x55b0bb53fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.681+0000 7ff9d7fff700 1 -- 172.21.15.130:0/1630678221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb5375a0 msgr2=0x55b0bb546c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:25.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.681+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb5375a0 0x55b0bb546c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.681+0000 7ff9d7fff700 1 -- 172.21.15.130:0/1630678221 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c0015040 con 0x55b0bb535e00 2023-09-12T16:06:25.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.681+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53f680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000b830 tx=0x7ff9c0006980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d51ba9ecc21bbe0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:25.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.682+0000 7ff9b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:06:25.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.682+0000 7ff9b7fff700 1 -- 172.21.15.130:0/1630678221 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0004330 con 0x55b0bb535e00 2023-09-12T16:06:25.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.682+0000 7ff9b7fff700 1 -- 172.21.15.130:0/1630678221 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff9c0000e40 con 0x55b0bb535e00 2023-09-12T16:06:25.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.682+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1630678221 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0bb3df4f0 con 0x55b0bb535e00 2023-09-12T16:06:25.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.682+0000 7ff9b7fff700 1 -- 172.21.15.130:0/1630678221 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c0004c70 con 0x55b0bb535e00 2023-09-12T16:06:25.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.683+0000 7ff9eb369b80 10 client.25597 fetch_fsmap learned FSMap version 29 2023-09-12T16:06:25.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.683+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1630678221 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b0bb29cc60 con 0x55b0bb535e00 2023-09-12T16:06:25.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.688+0000 7ff9b7fff700 1 -- 172.21.15.130:0/1630678221 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0bb3df4f0 con 0x55b0bb535e00 2023-09-12T16:06:25.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.688+0000 7ff9b7fff700 1 -- 172.21.15.130:0/1630678221 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff9c000a070 con 0x55b0bb535e00 2023-09-12T16:06:25.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.688+0000 7ff9eb369b80 10 client.25597 fetch_fsmap finished waiting for FSMap version 29 2023-09-12T16:06:25.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.688+0000 7ff9eb369b80 10 client.25597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:06:25.694 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.688+0000 7ff9eb369b80 20 client.25597 populate_metadata read hostname 'smithi130' 2023-09-12T16:06:25.694 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.688+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b0bb54b340 0x55b0bb54d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.694 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.688+0000 7ff9eb369b80 4 client.25597 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:06:25.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.688+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1630678221 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0bb49c8a0 con 0x55b0bb54b340 2023-09-12T16:06:25.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.691+0000 7ff9dcf3f700 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b0bb54b340 0x55b0bb54d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.691+0000 7ff9dcf3f700 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b0bb54b340 0x55b0bb54d760 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c16bb352060978f server_cookie=e476b9774b3d3de1 in_seq=0 out_seq=0 2023-09-12T16:06:25.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.691+0000 7ff9b7fff700 10 client.25597 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:06:25.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.697+0000 7ff9b7fff700 1 -- 172.21.15.130:0/1630678221 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b0bb49c8a0 con 0x55b0bb54b340 2023-09-12T16:06:25.698 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.697+0000 7ff9b7fff700 10 client.25597 handle_command_reply: tid=0 2023-09-12T16:06:25.698 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.697+0000 7ff9eb369b80 1 client.25597 shutdown 2023-09-12T16:06:25.699 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.699+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1630678221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 msgr2=0x55b0bb53f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:25.700 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.699+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53f680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000b830 tx=0x7ff9c0006980 comp rx=0 tx=0).stop 2023-09-12T16:06:25.700 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.700+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1630678221 shutdown_connections 2023-09-12T16:06:25.700 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.700+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b0bb54b340 0x55b0bb54d760 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.700+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb5375a0 0x55b0bb546c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.700+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53f680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.700+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1630678221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb536c00 0x55b0bb53fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.700+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1630678221 >> 172.21.15.130:0/1630678221 conn(0x55b0bb4aa090 msgr2=0x55b0bb52fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:25.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.700+0000 7ff9b7fff700 0 client.25597 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:06:25.704 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.704+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1630678221 shutdown_connections 2023-09-12T16:06:25.705 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.704+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1630678221 wait complete. 2023-09-12T16:06:25.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.713+0000 7ff9eb369b80 20 client.25597 trim_cache size 0 max 16384 2023-09-12T16:06:25.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.749+0000 7ff9eb369b80 1 Processor -- start 2023-09-12T16:06:25.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.749+0000 7ff9eb369b80 1 -- start start 2023-09-12T16:06:25.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.750+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.750+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb546bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.750+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 0x55b0bb539f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.750+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb53df00 con 0x55b0bb536c00 2023-09-12T16:06:25.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.750+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb53f1f0 con 0x55b0bb5375a0 2023-09-12T16:06:25.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.750+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb5319e0 con 0x55b0bb535e00 2023-09-12T16:06:25.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.750+0000 7ff9dcf3f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb546bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.750+0000 7ff9dcf3f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb546bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55834/0 (socket says 172.21.15.130:55834) 2023-09-12T16:06:25.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.750+0000 7ff9dcf3f700 1 -- 172.21.15.130:0/1759912835 learned_addr learned my addr 172.21.15.130:0/1759912835 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:25.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.751+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/1759912835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.751+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/1759912835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 0x55b0bb539f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.751+0000 7ff9dcf3f700 1 -- 172.21.15.130:0/1759912835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 msgr2=0x55b0bb53d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:25.755 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.751+0000 7ff9dcf3f700 1 --2- 172.21.15.130:0/1759912835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.751+0000 7ff9dcf3f700 1 -- 172.21.15.130:0/1759912835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 msgr2=0x55b0bb539f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:25.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.751+0000 7ff9dcf3f700 1 --2- 172.21.15.130:0/1759912835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 0x55b0bb539f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.751+0000 7ff9dcf3f700 1 -- 172.21.15.130:0/1759912835 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0bb552370 con 0x55b0bb536c00 2023-09-12T16:06:25.756 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.752+0000 7ff9dcf3f700 1 --2- 172.21.15.130:0/1759912835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb546bc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8007a70 tx=0x7ff9c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2fda992cb27eef server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:25.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.752+0000 7ff9cf7fe700 1 -- 172.21.15.130:0/1759912835 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8021070 con 0x55b0bb536c00 2023-09-12T16:06:25.757 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.752+0000 7ff9cf7fe700 1 -- 172.21.15.130:0/1759912835 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c8003b80 con 0x55b0bb536c00 2023-09-12T16:06:25.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.752+0000 7ff9cf7fe700 1 -- 172.21.15.130:0/1759912835 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800e040 con 0x55b0bb536c00 2023-09-12T16:06:25.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.753+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1759912835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 msgr2=0x55b0bb546bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:25.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.753+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1759912835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb546bc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8007a70 tx=0x7ff9c800b330 comp rx=0 tx=0).stop 2023-09-12T16:06:25.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.753+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1759912835 shutdown_connections 2023-09-12T16:06:25.758 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.753+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1759912835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb5375a0 0x55b0bb539f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.753+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1759912835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb535e00 0x55b0bb53d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.753+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/1759912835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb536c00 0x55b0bb546bc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.753+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1759912835 >> 172.21.15.130:0/1759912835 conn(0x55b0bb4aa090 msgr2=0x55b0bb53e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:25.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.753+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1759912835 shutdown_connections 2023-09-12T16:06:25.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.755+0000 7ff9eb369b80 1 -- 172.21.15.130:0/1759912835 wait complete. 2023-09-12T16:06:25.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.755+0000 7ff9eb369b80 1 -- start start 2023-09-12T16:06:25.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.755+0000 7ff9eb369b80 1 Processor -- start 2023-09-12T16:06:25.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb536c00 0x55b0bb5428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb542e00 0x55b0bb545e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9eb369b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb543280 0x55b0bb5463c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb546ad0 con 0x55b0bb543280 2023-09-12T16:06:25.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb546c40 con 0x55b0bb542e00 2023-09-12T16:06:25.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9eb369b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bb546e00 con 0x55b0bb536c00 2023-09-12T16:06:25.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9eb369b80 10 client.0 _ll_register_callbacks cb 0x7ff9c000be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:06:25.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb536c00 0x55b0bb5428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb536c00 0x55b0bb5428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35452/0 (socket says 172.21.15.130:35452) 2023-09-12T16:06:25.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9d7fff700 1 -- 172.21.15.130:0/459859383 learned_addr learned my addr 172.21.15.130:0/459859383 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:25.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.756+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb543280 0x55b0bb5463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.757+0000 7ff9dcf3f700 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb542e00 0x55b0bb545e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.757+0000 7ff9d7fff700 1 -- 172.21.15.130:0/459859383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb542e00 msgr2=0x55b0bb545e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:25.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.757+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb542e00 0x55b0bb545e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.757+0000 7ff9d7fff700 1 -- 172.21.15.130:0/459859383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb543280 msgr2=0x55b0bb5463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:25.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.757+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb543280 0x55b0bb5463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.757+0000 7ff9d7fff700 1 -- 172.21.15.130:0/459859383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d810d940 con 0x55b0bb536c00 2023-09-12T16:06:25.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.759+0000 7ff9d7fff700 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb536c00 0x55b0bb5428c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0004f90 tx=0x7ff9c0006980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4496ab0fed17adee server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:25.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.760+0000 7ff9b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:06:25.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.760+0000 7ff9b7fff700 1 -- 172.21.15.130:0/459859383 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0003890 con 0x55b0bb536c00 2023-09-12T16:06:25.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.760+0000 7ff9b7fff700 1 -- 172.21.15.130:0/459859383 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff9c0000e40 con 0x55b0bb536c00 2023-09-12T16:06:25.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.760+0000 7ff9eb369b80 1 -- 172.21.15.130:0/459859383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9c0004de0 con 0x55b0bb536c00 2023-09-12T16:06:25.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.760+0000 7ff9b7fff700 1 -- 172.21.15.130:0/459859383 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c00042d0 con 0x55b0bb536c00 2023-09-12T16:06:25.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.760+0000 7ff9eb369b80 10 client.25603 fetch_fsmap learned FSMap version 29 2023-09-12T16:06:25.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.760+0000 7ff9eb369b80 1 -- 172.21.15.130:0/459859383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b0bb2da250 con 0x55b0bb536c00 2023-09-12T16:06:25.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.760+0000 7ff9b7fff700 1 -- 172.21.15.130:0/459859383 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c0004de0 con 0x55b0bb536c00 2023-09-12T16:06:25.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.761+0000 7ff9b7fff700 1 -- 172.21.15.130:0/459859383 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff9c000da60 con 0x55b0bb536c00 2023-09-12T16:06:25.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.761+0000 7ff9eb369b80 10 client.25603 fetch_fsmap finished waiting for FSMap version 29 2023-09-12T16:06:25.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.761+0000 7ff9eb369b80 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:06:25.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.761+0000 7ff9eb369b80 20 client.25603 populate_metadata read hostname 'smithi130' 2023-09-12T16:06:25.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.761+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b0bb5a0530 0x55b0bb5a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:25.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.761+0000 7ff9eb369b80 4 client.25603 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:06:25.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.761+0000 7ff9eb369b80 1 -- 172.21.15.130:0/459859383 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b0bb480920 con 0x55b0bb5a0530 2023-09-12T16:06:25.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.761+0000 7ff9dcf3f700 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b0bb5a0530 0x55b0bb5a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:25.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.762+0000 7ff9dcf3f700 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b0bb5a0530 0x55b0bb5a2950 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71d4bf04c567bfb server_cookie=e1e341e0166a6497 in_seq=0 out_seq=0 2023-09-12T16:06:25.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.762+0000 7ff9b7fff700 10 client.25603 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:06:25.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9b7fff700 1 -- 172.21.15.130:0/459859383 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b0bb480920 con 0x55b0bb5a0530 2023-09-12T16:06:25.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9b7fff700 10 client.25603 handle_command_reply: tid=0 2023-09-12T16:06:25.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9eb369b80 1 client.25603 shutdown 2023-09-12T16:06:25.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9eb369b80 1 -- 172.21.15.130:0/459859383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb536c00 msgr2=0x55b0bb5428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:25.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb536c00 0x55b0bb5428c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0004f90 tx=0x7ff9c0006980 comp rx=0 tx=0).stop 2023-09-12T16:06:25.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9eb369b80 1 -- 172.21.15.130:0/459859383 shutdown_connections 2023-09-12T16:06:25.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55b0bb5a0530 0x55b0bb5a2950 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0bb543280 0x55b0bb5463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9b7fff700 0 client.25603 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:06:25.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0bb536c00 0x55b0bb5428c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9eb369b80 1 --2- 172.21.15.130:0/459859383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0bb542e00 0x55b0bb545e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.763+0000 7ff9eb369b80 1 -- 172.21.15.130:0/459859383 >> 172.21.15.130:0/459859383 conn(0x55b0bb4aa090 msgr2=0x55b0bb53d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:25.772 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.764+0000 7ff9eb369b80 1 -- 172.21.15.130:0/459859383 shutdown_connections 2023-09-12T16:06:25.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.765+0000 7ff9eb369b80 1 -- 172.21.15.130:0/459859383 wait complete. 2023-09-12T16:06:25.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.786+0000 7ff9eb369b80 20 client.25603 trim_cache size 0 max 16384 2023-09-12T16:06:25.791 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:06:25.791 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:06:25.791 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:06:25.792 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:06:25.792 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:06:25.792 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:06:25.792 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:06:25.792 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:06:25.793 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:06:25.793 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:06:25.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.809+0000 7ff9df1a3700 1 -- 172.21.15.130:0/927363041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9c40a0790 msgr2=0x7ff9c40a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:25.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.809+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9c40a0790 0x7ff9c40a2c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0096a10 tx=0x7ff9d0009550 comp rx=0 tx=0).stop 2023-09-12T16:06:25.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.809+0000 7ff9df1a3700 1 -- 172.21.15.130:0/927363041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d810c940 msgr2=0x7ff9d81b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:25.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.809+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d810c940 0x7ff9d81b5390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000e970 tx=0x7ff9d000ed30 comp rx=0 tx=0).stop 2023-09-12T16:06:25.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.815+0000 7ff9df1a3700 1 -- 172.21.15.130:0/927363041 shutdown_connections 2023-09-12T16:06:25.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.815+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff9c40a0790 0x7ff9c40a2c50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.815+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d810c940 0x7ff9d81b5390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.815+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8107600 0x7ff9d8113b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.815+0000 7ff9df1a3700 1 --2- 172.21.15.130:0/927363041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9d8109f70 0x7ff9d8114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:25.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.815+0000 7ff9df1a3700 1 -- 172.21.15.130:0/927363041 >> 172.21.15.130:0/927363041 conn(0x7ff9d806d3b0 msgr2=0x7ff9d810ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:25.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.819+0000 7ff9df1a3700 1 -- 172.21.15.130:0/927363041 shutdown_connections 2023-09-12T16:06:25.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:25.824+0000 7ff9df1a3700 1 -- 172.21.15.130:0/927363041 wait complete. 2023-09-12T16:06:25.853 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:06:26.121 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:26 smithi130 ceph-mon[98670]: pgmap v397: 129 pgs: 129 active+clean; 8.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 MiB/s wr, 2.81k op/s 2023-09-12T16:06:26.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:26 smithi159 ceph-mon[102522]: pgmap v397: 129 pgs: 129 active+clean; 8.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 MiB/s wr, 2.81k op/s 2023-09-12T16:06:26.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:26 smithi203 ceph-mon[103431]: pgmap v397: 129 pgs: 129 active+clean; 8.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 MiB/s wr, 2.81k op/s 2023-09-12T16:06:27.974 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:27 smithi130 ceph-mon[98670]: pgmap v398: 129 pgs: 129 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 63 MiB/s wr, 3.37k op/s 2023-09-12T16:06:27.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:27 smithi159 ceph-mon[102522]: pgmap v398: 129 pgs: 129 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 63 MiB/s wr, 3.37k op/s 2023-09-12T16:06:27.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:27 smithi203 ceph-mon[103431]: pgmap v398: 129 pgs: 129 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 63 MiB/s wr, 3.37k op/s 2023-09-12T16:06:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 40.0 s, 2785.8 tps, lat 35.979 ms stddev 67.312 2023-09-12T16:06:30.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:29 smithi159 ceph-mon[102522]: pgmap v399: 129 pgs: 129 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 68 MiB/s wr, 4.12k op/s 2023-09-12T16:06:30.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:29 smithi130 ceph-mon[98670]: pgmap v399: 129 pgs: 129 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 68 MiB/s wr, 4.12k op/s 2023-09-12T16:06:30.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:29 smithi203 ceph-mon[103431]: pgmap v399: 129 pgs: 129 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 68 MiB/s wr, 4.12k op/s 2023-09-12T16:06:32.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:31 smithi130 ceph-mon[98670]: pgmap v400: 129 pgs: 129 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 68 MiB/s wr, 4.13k op/s 2023-09-12T16:06:32.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:31 smithi159 ceph-mon[102522]: pgmap v400: 129 pgs: 129 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 68 MiB/s wr, 4.13k op/s 2023-09-12T16:06:32.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:31 smithi203 ceph-mon[103431]: pgmap v400: 129 pgs: 129 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 68 MiB/s wr, 4.13k op/s 2023-09-12T16:06:34.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:33 smithi159 ceph-mon[102522]: pgmap v401: 129 pgs: 129 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 65 MiB/s wr, 4.05k op/s 2023-09-12T16:06:34.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:33 smithi130 ceph-mon[98670]: pgmap v401: 129 pgs: 129 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 65 MiB/s wr, 4.05k op/s 2023-09-12T16:06:34.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:33 smithi203 ceph-mon[103431]: pgmap v401: 129 pgs: 129 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 65 MiB/s wr, 4.05k op/s 2023-09-12T16:06:35.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 45.0 s, 2151.1 tps, lat 46.406 ms stddev 89.874 2023-09-12T16:06:36.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:35 smithi130 ceph-mon[98670]: pgmap v402: 129 pgs: 129 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 MiB/s wr, 3.85k op/s 2023-09-12T16:06:36.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:35 smithi159 ceph-mon[102522]: pgmap v402: 129 pgs: 129 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 MiB/s wr, 3.85k op/s 2023-09-12T16:06:36.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:36 smithi203 ceph-mon[103431]: pgmap v402: 129 pgs: 129 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 MiB/s wr, 3.85k op/s 2023-09-12T16:06:37.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:37 smithi203 ceph-mon[103431]: pgmap v403: 129 pgs: 129 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 58 MiB/s wr, 3.72k op/s 2023-09-12T16:06:38.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:37 smithi130 ceph-mon[98670]: pgmap v403: 129 pgs: 129 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 58 MiB/s wr, 3.72k op/s 2023-09-12T16:06:38.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:37 smithi159 ceph-mon[102522]: pgmap v403: 129 pgs: 129 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 58 MiB/s wr, 3.72k op/s 2023-09-12T16:06:40.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 50.0 s, 825.1 tps, lat 104.576 ms stddev 263.938 2023-09-12T16:06:40.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:39 smithi159 ceph-mon[102522]: pgmap v404: 129 pgs: 129 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 61 MiB/s wr, 3.46k op/s 2023-09-12T16:06:40.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:39 smithi130 ceph-mon[98670]: pgmap v404: 129 pgs: 129 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 61 MiB/s wr, 3.46k op/s 2023-09-12T16:06:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:39 smithi203 ceph-mon[103431]: pgmap v404: 129 pgs: 129 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 61 MiB/s wr, 3.46k op/s 2023-09-12T16:06:42.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:41 smithi130 ceph-mon[98670]: pgmap v405: 129 pgs: 129 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 61 MiB/s wr, 3.32k op/s 2023-09-12T16:06:42.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:41 smithi159 ceph-mon[102522]: pgmap v405: 129 pgs: 129 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 61 MiB/s wr, 3.32k op/s 2023-09-12T16:06:42.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:41 smithi203 ceph-mon[103431]: pgmap v405: 129 pgs: 129 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 61 MiB/s wr, 3.32k op/s 2023-09-12T16:06:44.254 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:43 smithi203 ceph-mon[103431]: pgmap v406: 129 pgs: 129 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 MiB/s wr, 3.83k op/s 2023-09-12T16:06:44.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:43 smithi159 ceph-mon[102522]: pgmap v406: 129 pgs: 129 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 MiB/s wr, 3.83k op/s 2023-09-12T16:06:44.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:43 smithi130 ceph-mon[98670]: pgmap v406: 129 pgs: 129 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 MiB/s wr, 3.83k op/s 2023-09-12T16:06:45.092 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 55.0 s, 3184.1 tps, lat 35.284 ms stddev 72.598 2023-09-12T16:06:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:45 smithi203 ceph-mon[103431]: pgmap v407: 129 pgs: 129 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 57 MiB/s wr, 3.80k op/s 2023-09-12T16:06:46.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:45 smithi130 ceph-mon[98670]: pgmap v407: 129 pgs: 129 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 57 MiB/s wr, 3.80k op/s 2023-09-12T16:06:46.484 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:45 smithi159 ceph-mon[102522]: pgmap v407: 129 pgs: 129 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 57 MiB/s wr, 3.80k op/s 2023-09-12T16:06:48.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:47 smithi159 ceph-mon[102522]: pgmap v408: 129 pgs: 129 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 61 MiB/s wr, 3.97k op/s 2023-09-12T16:06:48.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:47 smithi130 ceph-mon[98670]: pgmap v408: 129 pgs: 129 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 61 MiB/s wr, 3.97k op/s 2023-09-12T16:06:48.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:47 smithi203 ceph-mon[103431]: pgmap v408: 129 pgs: 129 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 61 MiB/s wr, 3.97k op/s 2023-09-12T16:06:50.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 60.0 s, 1723.5 tps, lat 58.930 ms stddev 118.055 2023-09-12T16:06:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:49 smithi130 ceph-mon[98670]: pgmap v409: 129 pgs: 129 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 66 MiB/s wr, 4.00k op/s 2023-09-12T16:06:50.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:49 smithi159 ceph-mon[102522]: pgmap v409: 129 pgs: 129 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 66 MiB/s wr, 4.00k op/s 2023-09-12T16:06:50.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:49 smithi203 ceph-mon[103431]: pgmap v409: 129 pgs: 129 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 66 MiB/s wr, 4.00k op/s 2023-09-12T16:06:51.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:50 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:06:51.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:50 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:06:51.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:50 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:06:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:51 smithi203 ceph-mon[103431]: pgmap v410: 129 pgs: 129 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 MiB/s wr, 3.50k op/s 2023-09-12T16:06:52.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:51 smithi159 ceph-mon[102522]: pgmap v410: 129 pgs: 129 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 MiB/s wr, 3.50k op/s 2023-09-12T16:06:52.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:51 smithi130 ceph-mon[98670]: pgmap v410: 129 pgs: 129 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 MiB/s wr, 3.50k op/s 2023-09-12T16:06:54.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:54 smithi130 ceph-mon[98670]: pgmap v411: 129 pgs: 129 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.52k op/s 2023-09-12T16:06:54.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:54 smithi159 ceph-mon[102522]: pgmap v411: 129 pgs: 129 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.52k op/s 2023-09-12T16:06:54.502 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:54 smithi203 ceph-mon[103431]: pgmap v411: 129 pgs: 129 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.52k op/s 2023-09-12T16:06:55.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 65.0 s, 1023.7 tps, lat 85.369 ms stddev 188.880 2023-09-12T16:06:55.854 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:06:56.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.160+0000 7ff2e2087700 1 -- 172.21.15.130:0/1910349840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc10c940 msgr2=0x7ff2dc10cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:56.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.160+0000 7ff2e2087700 1 --2- 172.21.15.130:0/1910349840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc10c940 0x7ff2dc10cdc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009990 tx=0x7ff2cc009ca0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.183+0000 7ff2e2087700 1 -- 172.21.15.130:0/1910349840 shutdown_connections 2023-09-12T16:06:56.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.183+0000 7ff2e2087700 1 --2- 172.21.15.130:0/1910349840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc10c940 0x7ff2dc10cdc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.183+0000 7ff2e2087700 1 --2- 172.21.15.130:0/1910349840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2dc107600 0x7ff2dc109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.183+0000 7ff2e2087700 1 --2- 172.21.15.130:0/1910349840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2dc109f70 0x7ff2dc10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.183+0000 7ff2e2087700 1 -- 172.21.15.130:0/1910349840 >> 172.21.15.130:0/1910349840 conn(0x7ff2dc06d3b0 msgr2=0x7ff2dc06f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:56.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.184+0000 7ff2e2087700 1 -- 172.21.15.130:0/1910349840 shutdown_connections 2023-09-12T16:06:56.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.196+0000 7ff2e2087700 1 -- 172.21.15.130:0/1910349840 wait complete. 2023-09-12T16:06:56.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.196+0000 7ff2e2087700 1 Processor -- start 2023-09-12T16:06:56.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.201+0000 7ff2e2087700 1 -- start start 2023-09-12T16:06:56.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.202+0000 7ff2e2087700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2dc107600 0x7ff2dc116b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.202+0000 7ff2e2087700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc109f70 0x7ff2dc117090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.202+0000 7ff2e2087700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2dc10c940 0x7ff2dc1180d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.202+0000 7ff2e2087700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1186a0 con 0x7ff2dc107600 2023-09-12T16:06:56.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.202+0000 7ff2e2087700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc118810 con 0x7ff2dc10c940 2023-09-12T16:06:56.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.202+0000 7ff2e2087700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1189d0 con 0x7ff2dc109f70 2023-09-12T16:06:56.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.202+0000 7ff2daffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc109f70 0x7ff2dc117090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.202+0000 7ff2daffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc109f70 0x7ff2dc117090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35466/0 (socket says 172.21.15.130:35466) 2023-09-12T16:06:56.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.202+0000 7ff2daffd700 1 -- 172.21.15.130:0/3959861767 learned_addr learned my addr 172.21.15.130:0/3959861767 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:56.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.204+0000 7ff2db7fe700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2dc107600 0x7ff2dc116b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.205+0000 7ff2daffd700 1 -- 172.21.15.130:0/3959861767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2dc10c940 msgr2=0x7ff2dc1180d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:56.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.205+0000 7ff2daffd700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2dc10c940 0x7ff2dc1180d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.205+0000 7ff2daffd700 1 -- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2dc107600 msgr2=0x7ff2dc116b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:56.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.205+0000 7ff2daffd700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2dc107600 0x7ff2dc116b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.205+0000 7ff2daffd700 1 -- 172.21.15.130:0/3959861767 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc0095f0 con 0x7ff2dc109f70 2023-09-12T16:06:56.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.210+0000 7ff2daffd700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc109f70 0x7ff2dc117090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40060b0 tx=0x7ff2d400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:56.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.213+0000 7ff2d8ff9700 1 -- 172.21.15.130:0/3959861767 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400ddd0 con 0x7ff2dc109f70 2023-09-12T16:06:56.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.213+0000 7ff2d8ff9700 1 -- 172.21.15.130:0/3959861767 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2d400f040 con 0x7ff2dc109f70 2023-09-12T16:06:56.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.213+0000 7ff2d8ff9700 1 -- 172.21.15.130:0/3959861767 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4014600 con 0x7ff2dc109f70 2023-09-12T16:06:56.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.215+0000 7ff2e2087700 1 -- 172.21.15.130:0/3959861767 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc118c50 con 0x7ff2dc109f70 2023-09-12T16:06:56.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.215+0000 7ff2e2087700 1 -- 172.21.15.130:0/3959861767 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc1190e0 con 0x7ff2dc109f70 2023-09-12T16:06:56.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.220+0000 7ff2ee24db80 1 Processor -- start 2023-09-12T16:06:56.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.221+0000 7ff2ee24db80 1 -- start start 2023-09-12T16:06:56.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.222+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7103020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.222+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7101800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.222+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 0x557cb71022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.222+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb705a730 con 0x557cb7102c00 2023-09-12T16:06:56.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.222+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb7058050 con 0x557cb71035a0 2023-09-12T16:06:56.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.222+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb7053940 con 0x557cb7101e00 2023-09-12T16:06:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:56 smithi130 ceph-mon[98670]: pgmap v412: 129 pgs: 129 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 MiB/s wr, 1.81k op/s 2023-09-12T16:06:56.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:56 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:06:56.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.226+0000 7ff2db7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.226 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.226+0000 7ff2db7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7101800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39122/0 (socket says 172.21.15.130:39122) 2023-09-12T16:06:56.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.226+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/1361390038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.226+0000 7ff2db7fe700 1 -- 172.21.15.130:0/1361390038 learned_addr learned my addr 172.21.15.130:0/1361390038 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:56.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.226+0000 7ff2dbfff700 1 -- 172.21.15.130:0/1361390038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 msgr2=0x557cb71022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:56.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.226+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/1361390038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 0x557cb71022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.227 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.226+0000 7ff2dbfff700 1 -- 172.21.15.130:0/1361390038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 msgr2=0x557cb7101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:56.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.226+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/1361390038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.226+0000 7ff2dbfff700 1 -- 172.21.15.130:0/1361390038 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cb6e7fde0 con 0x557cb7102c00 2023-09-12T16:06:56.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.227+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/1361390038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7103020 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0098e0 tx=0x7ff2cc009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffbe0bda2d2a092 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:56.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.227+0000 7ff2c1ffb700 1 -- 172.21.15.130:0/1361390038 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc01d070 con 0x557cb7102c00 2023-09-12T16:06:56.228 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.227+0000 7ff2c1ffb700 1 -- 172.21.15.130:0/1361390038 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2cc00bd90 con 0x557cb7102c00 2023-09-12T16:06:56.229 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.227+0000 7ff2c1ffb700 1 -- 172.21.15.130:0/1361390038 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc021410 con 0x557cb7102c00 2023-09-12T16:06:56.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.228+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1361390038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 msgr2=0x557cb7103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:56.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.228+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/1361390038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7103020 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0098e0 tx=0x7ff2cc009550 comp rx=0 tx=0).stop 2023-09-12T16:06:56.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.231+0000 7ff2d8ff9700 1 -- 172.21.15.130:0/3959861767 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7ff2d4015400 con 0x7ff2dc109f70 2023-09-12T16:06:56.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.232+0000 7ff2d8ff9700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2c40a07c0 0x7ff2c40a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.232+0000 7ff2d8ff9700 1 -- 172.21.15.130:0/3959861767 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff2d40cb320 con 0x7ff2dc109f70 2023-09-12T16:06:56.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.236+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1361390038 shutdown_connections 2023-09-12T16:06:56.236 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.236+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/1361390038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 0x557cb71022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.236+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/1361390038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7103020 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.236+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/1361390038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.236+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1361390038 >> 172.21.15.130:0/1361390038 conn(0x557cb7076090 msgr2=0x557cb70fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:56.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.236+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1361390038 shutdown_connections 2023-09-12T16:06:56.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.237+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1361390038 wait complete. 2023-09-12T16:06:56.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.237+0000 7ff2ee24db80 1 -- start start 2023-09-12T16:06:56.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.237+0000 7ff2ee24db80 1 Processor -- start 2023-09-12T16:06:56.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.238+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 0x557cb710b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.238+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb710bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.238+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7112b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.238+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb6ff1b70 con 0x557cb7102c00 2023-09-12T16:06:56.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.238+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb6ff1870 con 0x557cb71035a0 2023-09-12T16:06:56.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.238+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb6ed5920 con 0x557cb7101e00 2023-09-12T16:06:56.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.238+0000 7ff2ee24db80 10 client.0 _ll_register_callbacks cb 0x557cb6f3ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:06:56.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.241+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2c40a07c0 0x7ff2c40a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.241+0000 7ff2daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb710bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.241+0000 7ff2daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb710bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55870/0 (socket says 172.21.15.130:55870) 2023-09-12T16:06:56.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.241+0000 7ff2daffd700 1 -- 172.21.15.130:0/3308183975 learned_addr learned my addr 172.21.15.130:0/3308183975 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:56.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.241+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.242+0000 7ff2daffd700 1 -- 172.21.15.130:0/3308183975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 msgr2=0x557cb710b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:56.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.242+0000 7ff2daffd700 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 0x557cb710b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.242+0000 7ff2daffd700 1 -- 172.21.15.130:0/3308183975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 msgr2=0x557cb7112b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:56.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.242+0000 7ff2daffd700 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7112b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.242+0000 7ff2daffd700 1 -- 172.21.15.130:0/3308183975 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc10d940 con 0x557cb7102c00 2023-09-12T16:06:56.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.242+0000 7ff2daffd700 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb710bb00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4095330 tx=0x7ff2d40982d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6565ccc3e736eb43 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:56.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.242+0000 7ff2baffd700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:06:56.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.242+0000 7ff2baffd700 1 -- 172.21.15.130:0/3308183975 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4097fb0 con 0x557cb7102c00 2023-09-12T16:06:56.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2baffd700 1 -- 172.21.15.130:0/3308183975 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff2d40988c0 con 0x557cb7102c00 2023-09-12T16:06:56.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2ee24db80 1 -- 172.21.15.130:0/3308183975 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cb6fab4f0 con 0x557cb7102c00 2023-09-12T16:06:56.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2baffd700 1 -- 172.21.15.130:0/3308183975 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d409c6f0 con 0x557cb7102c00 2023-09-12T16:06:56.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2ee24db80 10 client.15945 fetch_fsmap learned FSMap version 31 2023-09-12T16:06:56.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2ee24db80 1 -- 172.21.15.130:0/3308183975 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557cb6e68c60 con 0x557cb7102c00 2023-09-12T16:06:56.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2baffd700 1 -- 172.21.15.130:0/3308183975 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557cb6fab4f0 con 0x557cb7102c00 2023-09-12T16:06:56.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2baffd700 1 -- 172.21.15.130:0/3308183975 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff2d4015840 con 0x557cb7102c00 2023-09-12T16:06:56.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2ee24db80 10 client.15945 fetch_fsmap finished waiting for FSMap version 31 2023-09-12T16:06:56.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2ee24db80 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:06:56.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2ee24db80 20 client.15945 populate_metadata read hostname 'smithi130' 2023-09-12T16:06:56.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557cb7117420 0x557cb7119840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2ee24db80 4 client.15945 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:06:56.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.243+0000 7ff2ee24db80 1 -- 172.21.15.130:0/3308183975 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cb70688a0 con 0x557cb7117420 2023-09-12T16:06:56.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.251+0000 7ff2db7fe700 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557cb7117420 0x557cb7119840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.252+0000 7ff2db7fe700 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557cb7117420 0x557cb7119840 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5230907f9bfc5e42 server_cookie=ffa622dfe9e28d1e in_seq=0 out_seq=0 2023-09-12T16:06:56.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.252+0000 7ff2baffd700 10 client.15945 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:06:56.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.252+0000 7ff2baffd700 1 -- 172.21.15.130:0/3308183975 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557cb70688a0 con 0x557cb7117420 2023-09-12T16:06:56.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.252+0000 7ff2baffd700 10 client.15945 handle_command_reply: tid=0 2023-09-12T16:06:56.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.252+0000 7ff2ee24db80 1 client.15945 shutdown 2023-09-12T16:06:56.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.256+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2c40a07c0 0x7ff2c40a2c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00c010 tx=0x7ff2cc00bf60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:56.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.259+0000 7ff2ee24db80 1 -- 172.21.15.130:0/3308183975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 msgr2=0x557cb710bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:56.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.259+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb710bb00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4095330 tx=0x7ff2d40982d0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.261+0000 7ff2ee24db80 1 -- 172.21.15.130:0/3308183975 shutdown_connections 2023-09-12T16:06:56.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.261+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557cb7117420 0x557cb7119840 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.261+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7112b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.261+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 0x557cb710b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.261+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/3308183975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb710bb00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.261+0000 7ff2ee24db80 1 -- 172.21.15.130:0/3308183975 >> 172.21.15.130:0/3308183975 conn(0x557cb7076090 msgr2=0x557cb70fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:56.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.262+0000 7ff2baffd700 0 client.15945 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:06:56.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.263+0000 7ff2ee24db80 1 -- 172.21.15.130:0/3308183975 shutdown_connections 2023-09-12T16:06:56.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.268+0000 7ff2ee24db80 1 -- 172.21.15.130:0/3308183975 wait complete. 2023-09-12T16:06:56.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.280+0000 7ff2ee24db80 20 client.15945 trim_cache size 0 max 16384 2023-09-12T16:06:56.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.316+0000 7ff2ee24db80 1 Processor -- start 2023-09-12T16:06:56.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2ee24db80 1 -- start start 2023-09-12T16:06:56.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 0x557cb7101710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7112b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7119430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb710c740 con 0x557cb7102c00 2023-09-12T16:06:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb710aa70 con 0x557cb71035a0 2023-09-12T16:06:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb7101c50 con 0x557cb7101e00 2023-09-12T16:06:56.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2daffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7112b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55876/0 (socket says 172.21.15.130:55876) 2023-09-12T16:06:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.323+0000 7ff2daffd700 1 -- 172.21.15.130:0/1667926533 learned_addr learned my addr 172.21.15.130:0/1667926533 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.324+0000 7ff2daffd700 1 -- 172.21.15.130:0/1667926533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 msgr2=0x557cb7101710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:06:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.324+0000 7ff2daffd700 1 --2- 172.21.15.130:0/1667926533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 0x557cb7101710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.324+0000 7ff2daffd700 1 -- 172.21.15.130:0/1667926533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 msgr2=0x557cb7119430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:06:56.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.324+0000 7ff2daffd700 1 --2- 172.21.15.130:0/1667926533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7119430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.324+0000 7ff2daffd700 1 -- 172.21.15.130:0/1667926533 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cb6ea6250 con 0x557cb7102c00 2023-09-12T16:06:56.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.324+0000 7ff2daffd700 1 --2- 172.21.15.130:0/1667926533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7112b00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40a0400 tx=0x7ff2d400df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c0498f913a6956 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:56.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.326+0000 7ff2c27fc700 1 -- 172.21.15.130:0/1667926533 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40a06a0 con 0x557cb7102c00 2023-09-12T16:06:56.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.326+0000 7ff2c27fc700 1 -- 172.21.15.130:0/1667926533 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2d40988c0 con 0x557cb7102c00 2023-09-12T16:06:56.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.326+0000 7ff2c27fc700 1 -- 172.21.15.130:0/1667926533 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4098010 con 0x557cb7102c00 2023-09-12T16:06:56.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.327+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1667926533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 msgr2=0x557cb7112b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:56.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.327+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/1667926533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7112b00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40a0400 tx=0x7ff2d400df30 comp rx=0 tx=0).stop 2023-09-12T16:06:56.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.331+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1667926533 shutdown_connections 2023-09-12T16:06:56.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.331+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/1667926533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7119430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.331+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/1667926533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7101e00 0x557cb7101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.331+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/1667926533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7102c00 0x557cb7112b00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.331+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1667926533 >> 172.21.15.130:0/1667926533 conn(0x557cb7076090 msgr2=0x557cb70fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:56.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.344+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1667926533 shutdown_connections 2023-09-12T16:06:56.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.347+0000 7ff2ee24db80 1 -- 172.21.15.130:0/1667926533 wait complete. 2023-09-12T16:06:56.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.347+0000 7ff2ee24db80 1 -- start start 2023-09-12T16:06:56.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2ee24db80 1 Processor -- start 2023-09-12T16:06:56.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7101e00 0x557cb71124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7102c00 0x557cb7112a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2ee24db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7110af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.350 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb7111200 con 0x557cb7101e00 2023-09-12T16:06:56.350 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb7111370 con 0x557cb71035a0 2023-09-12T16:06:56.350 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2ee24db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb7111530 con 0x557cb7102c00 2023-09-12T16:06:56.351 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2ee24db80 10 client.0 _ll_register_callbacks cb 0x7ff2d4098690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:06:56.351 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2dbfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7110af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.351 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2dbfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7110af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39138/0 (socket says 172.21.15.130:39138) 2023-09-12T16:06:56.351 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2dbfff700 1 -- 172.21.15.130:0/162934884 learned_addr learned my addr 172.21.15.130:0/162934884 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:06:56.351 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.348+0000 7ff2daffd700 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7102c00 0x557cb7112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.349+0000 7ff2dbfff700 1 -- 172.21.15.130:0/162934884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7102c00 msgr2=0x557cb7112a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:56.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.349+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7102c00 0x557cb7112a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.349+0000 7ff2dbfff700 1 -- 172.21.15.130:0/162934884 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7101e00 msgr2=0x557cb71124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:06:56.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.349+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7101e00 0x557cb71124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.349+0000 7ff2dbfff700 1 -- 172.21.15.130:0/162934884 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cb6e68c60 con 0x557cb71035a0 2023-09-12T16:06:56.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.349+0000 7ff2dbfff700 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7110af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc023000 tx=0x7ff2cc009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d182cd967f06d866 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:06:56.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.355+0000 7ff2baffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:06:56.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.355+0000 7ff2baffd700 1 -- 172.21.15.130:0/162934884 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc032070 con 0x557cb71035a0 2023-09-12T16:06:56.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.355+0000 7ff2baffd700 1 -- 172.21.15.130:0/162934884 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff2cc02d7d0 con 0x557cb71035a0 2023-09-12T16:06:56.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.355+0000 7ff2ee24db80 1 -- 172.21.15.130:0/162934884 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2cc00bd90 con 0x557cb71035a0 2023-09-12T16:06:56.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.355+0000 7ff2baffd700 1 -- 172.21.15.130:0/162934884 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2cc01d040 con 0x557cb71035a0 2023-09-12T16:06:56.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.355+0000 7ff2ee24db80 10 client.25409 fetch_fsmap learned FSMap version 31 2023-09-12T16:06:56.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.355+0000 7ff2ee24db80 1 -- 172.21.15.130:0/162934884 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557cb6e7e2b0 con 0x557cb71035a0 2023-09-12T16:06:56.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.356+0000 7ff2baffd700 1 -- 172.21.15.130:0/162934884 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2cc00bd90 con 0x557cb71035a0 2023-09-12T16:06:56.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.356+0000 7ff2baffd700 1 -- 172.21.15.130:0/162934884 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff2cc038ab0 con 0x557cb71035a0 2023-09-12T16:06:56.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.356+0000 7ff2ee24db80 10 client.25409 fetch_fsmap finished waiting for FSMap version 31 2023-09-12T16:06:56.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.356+0000 7ff2ee24db80 10 client.25409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:06:56.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.356+0000 7ff2ee24db80 20 client.25409 populate_metadata read hostname 'smithi130' 2023-09-12T16:06:56.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.356+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557cb716c280 0x557cb716e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:06:56.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.356+0000 7ff2ee24db80 4 client.25409 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:06:56.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.356+0000 7ff2ee24db80 1 -- 172.21.15.130:0/162934884 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557cb704c920 con 0x557cb716c280 2023-09-12T16:06:56.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.357+0000 7ff2db7fe700 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557cb716c280 0x557cb716e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:06:56.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.357+0000 7ff2baffd700 10 client.25409 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:06:56.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.357+0000 7ff2db7fe700 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557cb716c280 0x557cb716e6a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ee5c8f3a45a029 server_cookie=8170d5ae27f81e9d in_seq=0 out_seq=0 2023-09-12T16:06:56.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.357+0000 7ff2baffd700 1 -- 172.21.15.130:0/162934884 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557cb704c920 con 0x557cb716c280 2023-09-12T16:06:56.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.357+0000 7ff2baffd700 10 client.25409 handle_command_reply: tid=0 2023-09-12T16:06:56.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.357+0000 7ff2ee24db80 1 client.25409 shutdown 2023-09-12T16:06:56.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.366+0000 7ff2ee24db80 1 -- 172.21.15.130:0/162934884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 msgr2=0x557cb7110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:56.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.366+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7110af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc023000 tx=0x7ff2cc009550 comp rx=0 tx=0).stop 2023-09-12T16:06:56.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2ee24db80 1 -- 172.21.15.130:0/162934884 shutdown_connections 2023-09-12T16:06:56.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557cb716c280 0x557cb716e6a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb71035a0 0x557cb7110af0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557cb7101e00 0x557cb71124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.370 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2ee24db80 1 --2- 172.21.15.130:0/162934884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557cb7102c00 0x557cb7112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2ee24db80 1 -- 172.21.15.130:0/162934884 >> 172.21.15.130:0/162934884 conn(0x557cb7076090 msgr2=0x557cb7109470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:56.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2baffd700 0 client.25409 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:06:56.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2ee24db80 1 -- 172.21.15.130:0/162934884 shutdown_connections 2023-09-12T16:06:56.371 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2ee24db80 1 -- 172.21.15.130:0/162934884 wait complete. 2023-09-12T16:06:56.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.369+0000 7ff2ee24db80 20 client.25409 trim_cache size 0 max 16384 2023-09-12T16:06:56.372 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:06:56.372 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:06:56.372 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:06:56.372 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:06:56.373 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:06:56.373 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:06:56.373 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:06:56.373 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:06:56.373 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:06:56.374 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:06:56.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.385+0000 7ff2e2087700 1 -- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2c40a07c0 msgr2=0x7ff2c40a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:56.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.385+0000 7ff2e2087700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2c40a07c0 0x7ff2c40a2c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00c010 tx=0x7ff2cc00bf60 comp rx=0 tx=0).stop 2023-09-12T16:06:56.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.385+0000 7ff2e2087700 1 -- 172.21.15.130:0/3959861767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc109f70 msgr2=0x7ff2dc117090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:06:56.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.385+0000 7ff2e2087700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc109f70 0x7ff2dc117090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40060b0 tx=0x7ff2d400d3b0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.390+0000 7ff2e2087700 1 -- 172.21.15.130:0/3959861767 shutdown_connections 2023-09-12T16:06:56.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.390+0000 7ff2e2087700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff2c40a07c0 0x7ff2c40a2c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.390+0000 7ff2e2087700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2dc10c940 0x7ff2dc1180d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.390+0000 7ff2e2087700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2dc107600 0x7ff2dc116b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.390+0000 7ff2e2087700 1 --2- 172.21.15.130:0/3959861767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2dc109f70 0x7ff2dc117090 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:06:56.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.390+0000 7ff2e2087700 1 -- 172.21.15.130:0/3959861767 >> 172.21.15.130:0/3959861767 conn(0x7ff2dc06d3b0 msgr2=0x7ff2dc06f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:06:56.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.395+0000 7ff2e2087700 1 -- 172.21.15.130:0/3959861767 shutdown_connections 2023-09-12T16:06:56.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:06:56.399+0000 7ff2e2087700 1 -- 172.21.15.130:0/3959861767 wait complete. 2023-09-12T16:06:56.430 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:06:56.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:56 smithi159 ceph-mon[102522]: pgmap v412: 129 pgs: 129 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 MiB/s wr, 1.81k op/s 2023-09-12T16:06:56.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:56 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:06:56.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:56 smithi203 ceph-mon[103431]: pgmap v412: 129 pgs: 129 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 MiB/s wr, 1.81k op/s 2023-09-12T16:06:56.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:56 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:06:57.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:57 smithi203 ceph-mon[103431]: pgmap v413: 129 pgs: 129 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 52 MiB/s wr, 2.80k op/s 2023-09-12T16:06:58.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:57 smithi130 ceph-mon[98670]: pgmap v413: 129 pgs: 129 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 52 MiB/s wr, 2.80k op/s 2023-09-12T16:06:58.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:57 smithi159 ceph-mon[102522]: pgmap v413: 129 pgs: 129 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 52 MiB/s wr, 2.80k op/s 2023-09-12T16:07:00.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 70.0 s, 2454.2 tps, lat 45.866 ms stddev 112.319 2023-09-12T16:07:00.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:59 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:07:00.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:06:59 smithi130 ceph-mon[98670]: pgmap v414: 129 pgs: 129 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 58 MiB/s wr, 3.68k op/s 2023-09-12T16:07:00.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:59 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:07:00.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:06:59 smithi159 ceph-mon[102522]: pgmap v414: 129 pgs: 129 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 58 MiB/s wr, 3.68k op/s 2023-09-12T16:07:00.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:59 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:07:00.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:06:59 smithi203 ceph-mon[103431]: pgmap v414: 129 pgs: 129 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 58 MiB/s wr, 3.68k op/s 2023-09-12T16:07:02.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:01 smithi203 ceph-mon[103431]: pgmap v415: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 58 MiB/s wr, 3.84k op/s 2023-09-12T16:07:02.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:01 smithi130 ceph-mon[98670]: pgmap v415: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 58 MiB/s wr, 3.84k op/s 2023-09-12T16:07:02.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:01 smithi159 ceph-mon[102522]: pgmap v415: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 58 MiB/s wr, 3.84k op/s 2023-09-12T16:07:04.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:03 smithi203 ceph-mon[103431]: pgmap v416: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 58 MiB/s wr, 3.81k op/s 2023-09-12T16:07:04.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:03 smithi130 ceph-mon[98670]: pgmap v416: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 58 MiB/s wr, 3.81k op/s 2023-09-12T16:07:04.482 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:03 smithi159 ceph-mon[102522]: pgmap v416: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 58 MiB/s wr, 3.81k op/s 2023-09-12T16:07:05.092 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 75.0 s, 1809.2 tps, lat 55.110 ms stddev 126.433 2023-09-12T16:07:06.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:05 smithi130 ceph-mon[98670]: pgmap v417: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 MiB/s wr, 3.37k op/s 2023-09-12T16:07:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:05 smithi159 ceph-mon[102522]: pgmap v417: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 MiB/s wr, 3.37k op/s 2023-09-12T16:07:06.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:05 smithi203 ceph-mon[103431]: pgmap v417: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 MiB/s wr, 3.37k op/s 2023-09-12T16:07:08.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:07 smithi130 ceph-mon[98670]: pgmap v418: 129 pgs: 129 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 52 MiB/s wr, 3.16k op/s 2023-09-12T16:07:08.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:07 smithi159 ceph-mon[102522]: pgmap v418: 129 pgs: 129 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 52 MiB/s wr, 3.16k op/s 2023-09-12T16:07:08.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:07 smithi203 ceph-mon[103431]: pgmap v418: 129 pgs: 129 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 52 MiB/s wr, 3.16k op/s 2023-09-12T16:07:10.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 80.0 s, 1396.9 tps, lat 64.818 ms stddev 187.675 2023-09-12T16:07:10.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:10 smithi130 ceph-mon[98670]: pgmap v419: 129 pgs: 129 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 MiB/s wr, 2.49k op/s 2023-09-12T16:07:10.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:10 smithi159 ceph-mon[102522]: pgmap v419: 129 pgs: 129 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 MiB/s wr, 2.49k op/s 2023-09-12T16:07:10.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:10 smithi203 ceph-mon[103431]: pgmap v419: 129 pgs: 129 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 MiB/s wr, 2.49k op/s 2023-09-12T16:07:12.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:12 smithi159 ceph-mon[102522]: pgmap v420: 129 pgs: 129 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 48 MiB/s wr, 2.30k op/s 2023-09-12T16:07:12.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:12 smithi130 ceph-mon[98670]: pgmap v420: 129 pgs: 129 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 48 MiB/s wr, 2.30k op/s 2023-09-12T16:07:12.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:12 smithi203 ceph-mon[103431]: pgmap v420: 129 pgs: 129 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 48 MiB/s wr, 2.30k op/s 2023-09-12T16:07:14.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:14 smithi159 ceph-mon[102522]: pgmap v421: 129 pgs: 129 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 MiB/s wr, 2.52k op/s 2023-09-12T16:07:14.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:14 smithi130 ceph-mon[98670]: pgmap v421: 129 pgs: 129 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 MiB/s wr, 2.52k op/s 2023-09-12T16:07:14.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:14 smithi203 ceph-mon[103431]: pgmap v421: 129 pgs: 129 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 MiB/s wr, 2.52k op/s 2023-09-12T16:07:15.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 85.0 s, 1529.1 tps, lat 70.887 ms stddev 182.852 2023-09-12T16:07:16.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:16 smithi130 ceph-mon[98670]: pgmap v422: 129 pgs: 129 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 49 MiB/s wr, 3.41k op/s 2023-09-12T16:07:16.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:16 smithi159 ceph-mon[102522]: pgmap v422: 129 pgs: 129 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 49 MiB/s wr, 3.41k op/s 2023-09-12T16:07:16.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:16 smithi203 ceph-mon[103431]: pgmap v422: 129 pgs: 129 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 49 MiB/s wr, 3.41k op/s 2023-09-12T16:07:17.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:07:17.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:07:17.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:17 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:07:17.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:07:17.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:07:17.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:17 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:07:17.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:07:17.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:07:17.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:17 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:07:18.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:18 smithi159 ceph-mon[102522]: pgmap v423: 129 pgs: 129 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 MiB/s wr, 3.74k op/s 2023-09-12T16:07:18.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:07:18.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:07:18.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:07:18.479 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:18 smithi130 ceph-mon[98670]: pgmap v423: 129 pgs: 129 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 MiB/s wr, 3.74k op/s 2023-09-12T16:07:18.480 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:07:18.480 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:07:18.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:07:18.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:18 smithi203 ceph-mon[103431]: pgmap v423: 129 pgs: 129 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 MiB/s wr, 3.74k op/s 2023-09-12T16:07:18.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:07:18.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:07:18.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:07:20.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 90.0 s, 2059.2 tps, lat 48.686 ms stddev 95.382 2023-09-12T16:07:20.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:20 smithi130 ceph-mon[98670]: pgmap v424: 129 pgs: 129 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 56 MiB/s wr, 3.72k op/s 2023-09-12T16:07:20.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:20 smithi159 ceph-mon[102522]: pgmap v424: 129 pgs: 129 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 56 MiB/s wr, 3.72k op/s 2023-09-12T16:07:20.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:20 smithi203 ceph-mon[103431]: pgmap v424: 129 pgs: 129 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 56 MiB/s wr, 3.72k op/s 2023-09-12T16:07:22.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:22 smithi159 ceph-mon[102522]: pgmap v425: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 54 MiB/s wr, 3.42k op/s 2023-09-12T16:07:22.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:22 smithi130 ceph-mon[98670]: pgmap v425: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 54 MiB/s wr, 3.42k op/s 2023-09-12T16:07:22.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:22 smithi203 ceph-mon[103431]: pgmap v425: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 54 MiB/s wr, 3.42k op/s 2023-09-12T16:07:24.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:24 smithi159 ceph-mon[102522]: pgmap v426: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 44 MiB/s wr, 2.61k op/s 2023-09-12T16:07:24.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:24 smithi130 ceph-mon[98670]: pgmap v426: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 44 MiB/s wr, 2.61k op/s 2023-09-12T16:07:24.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:24 smithi203 ceph-mon[103431]: pgmap v426: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 44 MiB/s wr, 2.61k op/s 2023-09-12T16:07:25.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 95.0 s, 2239.0 tps, lat 41.587 ms stddev 40.608 2023-09-12T16:07:26.431 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:07:26.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:26 smithi203 ceph-mon[103431]: pgmap v427: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 MiB/s wr, 2.17k op/s 2023-09-12T16:07:26.539 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:26 smithi130 ceph-mon[98670]: pgmap v427: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 MiB/s wr, 2.17k op/s 2023-09-12T16:07:26.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:26 smithi159 ceph-mon[102522]: pgmap v427: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 MiB/s wr, 2.17k op/s 2023-09-12T16:07:26.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.752+0000 7f29beb9a700 1 -- 172.21.15.130:0/2989436858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b8106c40 msgr2=0x7f29b8107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:07:26.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.752+0000 7f29beb9a700 1 --2- 172.21.15.130:0/2989436858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b8106c40 0x7f29b8107060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29a00098f0 tx=0x7f29a0009c00 comp rx=0 tx=0).stop 2023-09-12T16:07:26.773 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.773+0000 7f29beb9a700 1 -- 172.21.15.130:0/2989436858 shutdown_connections 2023-09-12T16:07:26.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.773+0000 7f29beb9a700 1 --2- 172.21.15.130:0/2989436858 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29b81087d0 0x7f29b810f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.773+0000 7f29beb9a700 1 --2- 172.21.15.130:0/2989436858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b8106c40 0x7f29b8107060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.773+0000 7f29beb9a700 1 --2- 172.21.15.130:0/2989436858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29b8107e30 0x7f29b8108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.773+0000 7f29beb9a700 1 -- 172.21.15.130:0/2989436858 >> 172.21.15.130:0/2989436858 conn(0x7f29b8075bd0 msgr2=0x7f29b8078010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:07:26.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.777+0000 7f29beb9a700 1 -- 172.21.15.130:0/2989436858 shutdown_connections 2023-09-12T16:07:26.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.777+0000 7f29beb9a700 1 -- 172.21.15.130:0/2989436858 wait complete. 2023-09-12T16:07:26.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29beb9a700 1 Processor -- start 2023-09-12T16:07:26.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29beb9a700 1 -- start start 2023-09-12T16:07:26.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29beb9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29b8106c40 0x7f29b8196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29beb9a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29b8107e30 0x7f29b8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29beb9a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b81087d0 0x7f29b819b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29beb9a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b819bd40 con 0x7f29b8107e30 2023-09-12T16:07:26.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29beb9a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b819beb0 con 0x7f29b8106c40 2023-09-12T16:07:26.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29beb9a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b819c070 con 0x7f29b81087d0 2023-09-12T16:07:26.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29bc936700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29b8106c40 0x7f29b8196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29bc936700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29b8106c40 0x7f29b8196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39150/0 (socket says 172.21.15.130:39150) 2023-09-12T16:07:26.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.778+0000 7f29bc936700 1 -- 172.21.15.130:0/1486752931 learned_addr learned my addr 172.21.15.130:0/1486752931 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:07:26.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.779+0000 7f29b7fff700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29b8107e30 0x7f29b8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29bd137700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b81087d0 0x7f29b819b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29bc936700 1 -- 172.21.15.130:0/1486752931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b81087d0 msgr2=0x7f29b819b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:07:26.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29bc936700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b81087d0 0x7f29b819b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29bc936700 1 -- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29b8107e30 msgr2=0x7f29b8198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:07:26.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29bc936700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29b8107e30 0x7f29b8198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29bc936700 1 -- 172.21.15.130:0/1486752931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009520 con 0x7f29b8106c40 2023-09-12T16:07:26.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29bd137700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b81087d0 0x7f29b819b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:07:26.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29bc936700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29b8106c40 0x7f29b8196940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f29a0000c00 tx=0x7f29a000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:07:26.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29b5ffb700 1 -- 172.21.15.130:0/1486752931 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a001c070 con 0x7f29b8106c40 2023-09-12T16:07:26.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29b5ffb700 1 -- 172.21.15.130:0/1486752931 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29a000ba90 con 0x7f29b8106c40 2023-09-12T16:07:26.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29beb9a700 1 -- 172.21.15.130:0/1486752931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b819c2f0 con 0x7f29b8106c40 2023-09-12T16:07:26.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.781+0000 7f29b5ffb700 1 -- 172.21.15.130:0/1486752931 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0003b40 con 0x7f29b8106c40 2023-09-12T16:07:26.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.782+0000 7f29beb9a700 1 -- 172.21.15.130:0/1486752931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b819c6c0 con 0x7f29b8106c40 2023-09-12T16:07:26.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.786+0000 7f29b5ffb700 1 -- 172.21.15.130:0/1486752931 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f29a0020030 con 0x7f29b8106c40 2023-09-12T16:07:26.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.787+0000 7f29b5ffb700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f29a40a0710 0x7f29a40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.787+0000 7f29b5ffb700 1 -- 172.21.15.130:0/1486752931 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f29a00cd270 con 0x7f29b8106c40 2023-09-12T16:07:26.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.790+0000 7f29b7fff700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f29a40a0710 0x7f29a40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.790+0000 7f29b7fff700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f29a40a0710 0x7f29a40a2bd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f29b810ef40 tx=0x7f29b001c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:07:26.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.794+0000 7f29cad60b80 1 Processor -- start 2023-09-12T16:07:26.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.794+0000 7f29cad60b80 1 -- start start 2023-09-12T16:07:26.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.794+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97eccc70 0x557d97ecd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.794+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ecb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.794+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ecc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.794+0000 7f29cad60b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97e13730 con 0x557d97ecbe30 2023-09-12T16:07:26.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.794+0000 7f29cad60b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97e11050 con 0x557d97eccc70 2023-09-12T16:07:26.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.794+0000 7f29cad60b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97e0c940 con 0x557d97ecd5d0 2023-09-12T16:07:26.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.796+0000 7f29b7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ecb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.796+0000 7f29b7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ecb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35512/0 (socket says 172.21.15.130:35512) 2023-09-12T16:07:26.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.796+0000 7f29b7fff700 1 -- 172.21.15.130:0/781063685 learned_addr learned my addr 172.21.15.130:0/781063685 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:07:26.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.797+0000 7f29b7fff700 1 -- 172.21.15.130:0/781063685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97eccc70 msgr2=0x557d97ecd090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:07:26.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.797+0000 7f29b7fff700 1 --2- 172.21.15.130:0/781063685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97eccc70 0x557d97ecd090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.797+0000 7f29b7fff700 1 -- 172.21.15.130:0/781063685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 msgr2=0x557d97ecc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:07:26.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.797+0000 7f29b7fff700 1 --2- 172.21.15.130:0/781063685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ecc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.797+0000 7f29b7fff700 1 -- 172.21.15.130:0/781063685 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d97c38de0 con 0x557d97ecd5d0 2023-09-12T16:07:26.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.797+0000 7f29b7fff700 1 --2- 172.21.15.130:0/781063685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ecb850 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29b00264f0 tx=0x7f29b0024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd01fba6265bc0b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:07:26.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.798+0000 7f29aeffd700 1 -- 172.21.15.130:0/781063685 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b001f050 con 0x557d97ecd5d0 2023-09-12T16:07:26.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.798+0000 7f29aeffd700 1 -- 172.21.15.130:0/781063685 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29b0020040 con 0x557d97ecd5d0 2023-09-12T16:07:26.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.798+0000 7f29aeffd700 1 -- 172.21.15.130:0/781063685 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0027a50 con 0x557d97ecd5d0 2023-09-12T16:07:26.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.799+0000 7f29cad60b80 1 -- 172.21.15.130:0/781063685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 msgr2=0x557d97ecb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:07:26.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.799+0000 7f29cad60b80 1 --2- 172.21.15.130:0/781063685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ecb850 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29b00264f0 tx=0x7f29b0024000 comp rx=0 tx=0).stop 2023-09-12T16:07:26.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.801+0000 7f29cad60b80 1 -- 172.21.15.130:0/781063685 shutdown_connections 2023-09-12T16:07:26.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.801+0000 7f29cad60b80 1 --2- 172.21.15.130:0/781063685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ecc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.801+0000 7f29cad60b80 1 --2- 172.21.15.130:0/781063685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97eccc70 0x557d97ecd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.801+0000 7f29cad60b80 1 --2- 172.21.15.130:0/781063685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ecb850 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.801+0000 7f29cad60b80 1 -- 172.21.15.130:0/781063685 >> 172.21.15.130:0/781063685 conn(0x557d97e2f090 msgr2=0x557d97eb5720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:07:26.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.805+0000 7f29cad60b80 1 -- 172.21.15.130:0/781063685 shutdown_connections 2023-09-12T16:07:26.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.806+0000 7f29cad60b80 1 -- 172.21.15.130:0/781063685 wait complete. 2023-09-12T16:07:26.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.806+0000 7f29cad60b80 1 -- start start 2023-09-12T16:07:26.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.807+0000 7f29cad60b80 1 Processor -- start 2023-09-12T16:07:26.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.809+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ecbe30 0x557d97ebbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.809+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ebc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.809+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ec1500 0x557d97ec3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.809+0000 7f29cad60b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97daab70 con 0x557d97ec1500 2023-09-12T16:07:26.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.809+0000 7f29cad60b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97daa870 con 0x557d97ecbe30 2023-09-12T16:07:26.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.809+0000 7f29cad60b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97c8e920 con 0x557d97ecd5d0 2023-09-12T16:07:26.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.809+0000 7f29cad60b80 10 client.0 _ll_register_callbacks cb 0x557d97cf4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:07:26.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.810+0000 7f29bd137700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ec1500 0x557d97ec3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.810+0000 7f29bd137700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ec1500 0x557d97ec3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55908/0 (socket says 172.21.15.130:55908) 2023-09-12T16:07:26.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.810+0000 7f29bd137700 1 -- 172.21.15.130:0/3872367689 learned_addr learned my addr 172.21.15.130:0/3872367689 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:07:26.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.813+0000 7f29b7fff700 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ebc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.813+0000 7f29bc936700 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ecbe30 0x557d97ebbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.813+0000 7f29b7fff700 1 -- 172.21.15.130:0/3872367689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ecbe30 msgr2=0x557d97ebbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:07:26.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.814+0000 7f29b7fff700 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ecbe30 0x557d97ebbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.814+0000 7f29b7fff700 1 -- 172.21.15.130:0/3872367689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ec1500 msgr2=0x557d97ec3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:07:26.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.814+0000 7f29b7fff700 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ec1500 0x557d97ec3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.814+0000 7f29b7fff700 1 -- 172.21.15.130:0/3872367689 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b0018dd0 con 0x557d97ecd5d0 2023-09-12T16:07:26.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.814+0000 7f29b7fff700 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ebc4d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29b00264f0 tx=0x7f29b0028a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d5c674e05fdb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:07:26.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.814+0000 7f2997fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:07:26.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.815+0000 7f2997fff700 1 -- 172.21.15.130:0/3872367689 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b001f070 con 0x557d97ecd5d0 2023-09-12T16:07:26.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.815+0000 7f2997fff700 1 -- 172.21.15.130:0/3872367689 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f29b0050040 con 0x557d97ecd5d0 2023-09-12T16:07:26.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.815+0000 7f29cad60b80 1 -- 172.21.15.130:0/3872367689 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d97d644f0 con 0x557d97ecd5d0 2023-09-12T16:07:26.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.815+0000 7f2997fff700 1 -- 172.21.15.130:0/3872367689 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b0028e30 con 0x557d97ecd5d0 2023-09-12T16:07:26.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.815+0000 7f29cad60b80 10 client.25621 fetch_fsmap learned FSMap version 32 2023-09-12T16:07:26.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.815+0000 7f29cad60b80 1 -- 172.21.15.130:0/3872367689 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d97c21c60 con 0x557d97ecd5d0 2023-09-12T16:07:26.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.815+0000 7f2997fff700 1 -- 172.21.15.130:0/3872367689 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d97d644f0 con 0x557d97ecd5d0 2023-09-12T16:07:26.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.816+0000 7f2997fff700 1 -- 172.21.15.130:0/3872367689 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f29b0067630 con 0x557d97ecd5d0 2023-09-12T16:07:26.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.816+0000 7f29cad60b80 10 client.25621 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:07:26.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.816+0000 7f29cad60b80 10 client.25621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:07:26.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.816+0000 7f29cad60b80 20 client.25621 populate_metadata read hostname 'smithi130' 2023-09-12T16:07:26.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.816+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557d97ec8070 0x557d97eca490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.816+0000 7f29cad60b80 4 client.25621 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:07:26.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.816+0000 7f29cad60b80 1 -- 172.21.15.130:0/3872367689 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d97e218a0 con 0x557d97ec8070 2023-09-12T16:07:26.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.818+0000 7f29bc936700 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557d97ec8070 0x557d97eca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.833+0000 7f29bc936700 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557d97ec8070 0x557d97eca490 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b6add5efc32511 server_cookie=75fe1eeb2548f234 in_seq=0 out_seq=0 2023-09-12T16:07:26.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.833+0000 7f2997fff700 10 client.25621 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:07:26.845 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.845+0000 7f2997fff700 1 -- 172.21.15.130:0/3872367689 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d97e218a0 con 0x557d97ec8070 2023-09-12T16:07:26.845 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.845+0000 7f2997fff700 10 client.25621 handle_command_reply: tid=0 2023-09-12T16:07:26.846 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.845+0000 7f29cad60b80 1 client.25621 shutdown 2023-09-12T16:07:26.852 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.852+0000 7f29cad60b80 1 -- 172.21.15.130:0/3872367689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 msgr2=0x557d97ebc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:07:26.853 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.852+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ebc4d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29b00264f0 tx=0x7f29b0028a10 comp rx=0 tx=0).stop 2023-09-12T16:07:26.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.857+0000 7f29cad60b80 1 -- 172.21.15.130:0/3872367689 shutdown_connections 2023-09-12T16:07:26.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.857+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557d97ec8070 0x557d97eca490 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.857+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ec1500 0x557d97ec3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.857+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ecbe30 0x557d97ebbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.857+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3872367689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97ebc4d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.857+0000 7f29cad60b80 1 -- 172.21.15.130:0/3872367689 >> 172.21.15.130:0/3872367689 conn(0x557d97e2f090 msgr2=0x557d97eb4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:07:26.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.861+0000 7f2997fff700 0 client.25621 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:07:26.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.861+0000 7f29cad60b80 1 -- 172.21.15.130:0/3872367689 shutdown_connections 2023-09-12T16:07:26.865 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.865+0000 7f29cad60b80 1 -- 172.21.15.130:0/3872367689 wait complete. 2023-09-12T16:07:26.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.866+0000 7f29cad60b80 20 client.25621 trim_cache size 0 max 16384 2023-09-12T16:07:26.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.907+0000 7f29cad60b80 1 Processor -- start 2023-09-12T16:07:26.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.909+0000 7f29cad60b80 1 -- start start 2023-09-12T16:07:26.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.909+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ec1500 0x557d97ebaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.909+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ee0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.909+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ecd5d0 0x557d97ecad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.909+0000 7f29cad60b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97eb6870 con 0x557d97ecbe30 2023-09-12T16:07:26.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.909+0000 7f29cad60b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97ec7bc0 con 0x557d97ecd5d0 2023-09-12T16:07:26.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.909+0000 7f29cad60b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97ebd110 con 0x557d97ec1500 2023-09-12T16:07:26.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.910+0000 7f29bc936700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ec1500 0x557d97ebaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.910+0000 7f29bc936700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ec1500 0x557d97ebaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35524/0 (socket says 172.21.15.130:35524) 2023-09-12T16:07:26.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.910+0000 7f29bc936700 1 -- 172.21.15.130:0/3754720556 learned_addr learned my addr 172.21.15.130:0/3754720556 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:07:26.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29b7fff700 1 --2- 172.21.15.130:0/3754720556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ee0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29bc936700 1 -- 172.21.15.130:0/3754720556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ecd5d0 msgr2=0x557d97ecad90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:07:26.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29bc936700 1 --2- 172.21.15.130:0/3754720556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ecd5d0 0x557d97ecad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29bc936700 1 -- 172.21.15.130:0/3754720556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 msgr2=0x557d97ee0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:07:26.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29bc936700 1 --2- 172.21.15.130:0/3754720556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ee0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29bc936700 1 -- 172.21.15.130:0/3754720556 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d97c5f250 con 0x557d97ec1500 2023-09-12T16:07:26.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29b7fff700 1 --2- 172.21.15.130:0/3754720556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ee0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:07:26.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29bc936700 1 --2- 172.21.15.130:0/3754720556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ec1500 0x557d97ebaa80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29a00098c0 tx=0x7f29a00ca870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ef12f13fccf8b3b server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:07:26.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29af7fe700 1 -- 172.21.15.130:0/3754720556 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a009ae20 con 0x557d97ec1500 2023-09-12T16:07:26.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29af7fe700 1 -- 172.21.15.130:0/3754720556 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29a009a720 con 0x557d97ec1500 2023-09-12T16:07:26.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.911+0000 7f29af7fe700 1 -- 172.21.15.130:0/3754720556 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a00cab90 con 0x557d97ec1500 2023-09-12T16:07:26.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.912+0000 7f29cad60b80 1 -- 172.21.15.130:0/3754720556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ec1500 msgr2=0x557d97ebaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:07:26.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.912+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3754720556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ec1500 0x557d97ebaa80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29a00098c0 tx=0x7f29a00ca870 comp rx=0 tx=0).stop 2023-09-12T16:07:26.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.920+0000 7f29cad60b80 1 -- 172.21.15.130:0/3754720556 shutdown_connections 2023-09-12T16:07:26.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.920+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3754720556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ecd5d0 0x557d97ecad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.920+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3754720556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ec1500 0x557d97ebaa80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.920+0000 7f29cad60b80 1 --2- 172.21.15.130:0/3754720556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ee0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.920+0000 7f29cad60b80 1 -- 172.21.15.130:0/3754720556 >> 172.21.15.130:0/3754720556 conn(0x557d97e2f090 msgr2=0x557d97ed7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:07:26.928 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.927+0000 7f29cad60b80 1 -- 172.21.15.130:0/3754720556 shutdown_connections 2023-09-12T16:07:26.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.932+0000 7f29cad60b80 1 -- 172.21.15.130:0/3754720556 wait complete. 2023-09-12T16:07:26.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.932+0000 7f29cad60b80 1 -- start start 2023-09-12T16:07:26.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.932+0000 7f29cad60b80 1 Processor -- start 2023-09-12T16:07:26.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.933+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ec1500 0x557d97ede1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.933+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ede730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.933+0000 7f29cad60b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97edf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.933+0000 7f29cad60b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97edfc80 con 0x557d97ecbe30 2023-09-12T16:07:26.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.933+0000 7f29cad60b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97edfdf0 con 0x557d97ec1500 2023-09-12T16:07:26.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.933+0000 7f29cad60b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d97edffb0 con 0x557d97ecd5d0 2023-09-12T16:07:26.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.933+0000 7f29cad60b80 10 client.0 _ll_register_callbacks cb 0x7f29b0026440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:07:26.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.936+0000 7f29bd137700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97edf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.936+0000 7f29bd137700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97edf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35530/0 (socket says 172.21.15.130:35530) 2023-09-12T16:07:26.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.936+0000 7f29bd137700 1 -- 172.21.15.130:0/337231731 learned_addr learned my addr 172.21.15.130:0/337231731 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:07:26.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.941+0000 7f29bc936700 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ec1500 0x557d97ede1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.949+0000 7f29bc936700 1 -- 172.21.15.130:0/337231731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 msgr2=0x557d97edf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:07:26.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.949+0000 7f29bc936700 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97edf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.949+0000 7f29bc936700 1 -- 172.21.15.130:0/337231731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 msgr2=0x557d97ede730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:07:26.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.949+0000 7f29bc936700 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ede730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.949+0000 7f29bc936700 1 -- 172.21.15.130:0/337231731 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b819c6c0 con 0x557d97ec1500 2023-09-12T16:07:26.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.969+0000 7f29bc936700 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ec1500 0x557d97ede1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29a0096c20 tx=0x7f29a000bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff49f037214fae8b server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:07:26.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.971+0000 7f2997fff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:07:26.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.974+0000 7f2997fff700 1 -- 172.21.15.130:0/337231731 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0003ea0 con 0x557d97ec1500 2023-09-12T16:07:26.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.974+0000 7f2997fff700 1 -- 172.21.15.130:0/337231731 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f29a009ae20 con 0x557d97ec1500 2023-09-12T16:07:26.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.974+0000 7f29cad60b80 1 -- 172.21.15.130:0/337231731 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29b0020040 con 0x557d97ec1500 2023-09-12T16:07:26.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.982+0000 7f2997fff700 1 -- 172.21.15.130:0/337231731 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a009e550 con 0x557d97ec1500 2023-09-12T16:07:26.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.982+0000 7f29cad60b80 10 client.25418 fetch_fsmap learned FSMap version 32 2023-09-12T16:07:26.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.982+0000 7f29cad60b80 1 -- 172.21.15.130:0/337231731 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d97c372b0 con 0x557d97ec1500 2023-09-12T16:07:26.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.983+0000 7f2997fff700 1 -- 172.21.15.130:0/337231731 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29b0020040 con 0x557d97ec1500 2023-09-12T16:07:26.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.988+0000 7f2997fff700 1 -- 172.21.15.130:0/337231731 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f29a0099c10 con 0x557d97ec1500 2023-09-12T16:07:26.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.988+0000 7f29cad60b80 10 client.25418 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:07:26.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.988+0000 7f29cad60b80 10 client.25418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:07:26.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.988+0000 7f29cad60b80 20 client.25418 populate_metadata read hostname 'smithi130' 2023-09-12T16:07:26.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.988+0000 7f29cad60b80 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557d97f25780 0x557d97f27b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:07:26.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.988+0000 7f29cad60b80 4 client.25418 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:07:26.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.988+0000 7f29cad60b80 1 -- 172.21.15.130:0/337231731 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557d97e05920 con 0x557d97f25780 2023-09-12T16:07:26.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.988+0000 7f29b7fff700 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557d97f25780 0x557d97f27b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:07:26.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.993+0000 7f29b7fff700 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557d97f25780 0x557d97f27b70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ef517c29e026b3f server_cookie=c61e3123b7e2ba78 in_seq=0 out_seq=0 2023-09-12T16:07:26.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.993+0000 7f2997fff700 10 client.25418 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:07:26.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.993+0000 7f2997fff700 1 -- 172.21.15.130:0/337231731 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557d97e05920 con 0x557d97f25780 2023-09-12T16:07:26.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.993+0000 7f2997fff700 10 client.25418 handle_command_reply: tid=0 2023-09-12T16:07:26.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.993+0000 7f29cad60b80 1 client.25418 shutdown 2023-09-12T16:07:26.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.998+0000 7f29cad60b80 1 -- 172.21.15.130:0/337231731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ec1500 msgr2=0x557d97ede1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:07:26.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.998+0000 7f29cad60b80 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ec1500 0x557d97ede1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29a0096c20 tx=0x7f29a000bfa0 comp rx=0 tx=0).stop 2023-09-12T16:07:26.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.998+0000 7f29cad60b80 1 -- 172.21.15.130:0/337231731 shutdown_connections 2023-09-12T16:07:26.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.998+0000 7f29cad60b80 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x557d97f25780 0x557d97f27b70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:27.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.998+0000 7f29cad60b80 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d97ecd5d0 0x557d97edf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:27.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.998+0000 7f29cad60b80 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d97ec1500 0x557d97ede1f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:27.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.998+0000 7f2997fff700 0 client.25418 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:07:27.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.998+0000 7f29cad60b80 1 --2- 172.21.15.130:0/337231731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557d97ecbe30 0x557d97ede730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:27.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.998+0000 7f29cad60b80 1 -- 172.21.15.130:0/337231731 >> 172.21.15.130:0/337231731 conn(0x557d97e2f090 msgr2=0x557d97ebaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:07:27.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.999+0000 7f29cad60b80 1 -- 172.21.15.130:0/337231731 shutdown_connections 2023-09-12T16:07:27.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:26.999+0000 7f29cad60b80 1 -- 172.21.15.130:0/337231731 wait complete. 2023-09-12T16:07:27.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.010+0000 7f29cad60b80 20 client.25418 trim_cache size 0 max 16384 2023-09-12T16:07:27.015 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:07:27.015 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:07:27.015 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:07:27.015 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:07:27.016 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:07:27.016 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:07:27.016 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:07:27.016 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:07:27.016 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:07:27.017 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:07:27.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 -- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f29a40a0710 msgr2=0x7f29a40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:07:27.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f29a40a0710 0x7f29a40a2bd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f29b810ef40 tx=0x7f29b001c040 comp rx=0 tx=0).stop 2023-09-12T16:07:27.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 -- 172.21.15.130:0/1486752931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29b8106c40 msgr2=0x7f29b8196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:07:27.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29b8106c40 0x7f29b8196940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f29a0000c00 tx=0x7f29a000b840 comp rx=0 tx=0).stop 2023-09-12T16:07:27.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 -- 172.21.15.130:0/1486752931 shutdown_connections 2023-09-12T16:07:27.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f29a40a0710 0x7f29a40a2bd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:27.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b81087d0 0x7f29b819b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:27.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29b8106c40 0x7f29b8196940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:27.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 --2- 172.21.15.130:0/1486752931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29b8107e30 0x7f29b8198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:07:27.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.019+0000 7f29beb9a700 1 -- 172.21.15.130:0/1486752931 >> 172.21.15.130:0/1486752931 conn(0x7f29b8075bd0 msgr2=0x7f29b806f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:07:27.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.031+0000 7f29beb9a700 1 -- 172.21.15.130:0/1486752931 shutdown_connections 2023-09-12T16:07:27.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:07:27.035+0000 7f29beb9a700 1 -- 172.21.15.130:0/1486752931 wait complete. 2023-09-12T16:07:27.071 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:07:28.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:27 smithi130 ceph-mon[98670]: pgmap v428: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 MiB/s wr, 1.86k op/s 2023-09-12T16:07:28.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:27 smithi159 ceph-mon[102522]: pgmap v428: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 MiB/s wr, 1.86k op/s 2023-09-12T16:07:28.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:27 smithi203 ceph-mon[103431]: pgmap v428: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 MiB/s wr, 1.86k op/s 2023-09-12T16:07:30.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 100.0 s, 384.8 tps, lat 271.606 ms stddev 478.777 2023-09-12T16:07:30.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:30 smithi159 ceph-mon[102522]: pgmap v429: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 48 MiB/s wr, 2.51k op/s 2023-09-12T16:07:30.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:30 smithi130 ceph-mon[98670]: pgmap v429: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 48 MiB/s wr, 2.51k op/s 2023-09-12T16:07:30.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:30 smithi203 ceph-mon[103431]: pgmap v429: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 48 MiB/s wr, 2.51k op/s 2023-09-12T16:07:32.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:32 smithi159 ceph-mon[102522]: pgmap v430: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 48 MiB/s wr, 3.14k op/s 2023-09-12T16:07:32.479 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:32 smithi130 ceph-mon[98670]: pgmap v430: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 48 MiB/s wr, 3.14k op/s 2023-09-12T16:07:32.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:32 smithi203 ceph-mon[103431]: pgmap v430: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 48 MiB/s wr, 3.14k op/s 2023-09-12T16:07:34.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:34 smithi130 ceph-mon[98670]: pgmap v431: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 MiB/s wr, 3.44k op/s 2023-09-12T16:07:34.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:34 smithi159 ceph-mon[102522]: pgmap v431: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 MiB/s wr, 3.44k op/s 2023-09-12T16:07:34.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:34 smithi203 ceph-mon[103431]: pgmap v431: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 MiB/s wr, 3.44k op/s 2023-09-12T16:07:35.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 105.0 s, 2571.1 tps, lat 40.263 ms stddev 52.888 2023-09-12T16:07:36.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:36 smithi159 ceph-mon[102522]: pgmap v432: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 53 MiB/s wr, 3.72k op/s 2023-09-12T16:07:36.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:36 smithi130 ceph-mon[98670]: pgmap v432: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 53 MiB/s wr, 3.72k op/s 2023-09-12T16:07:36.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:36 smithi203 ceph-mon[103431]: pgmap v432: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 53 MiB/s wr, 3.72k op/s 2023-09-12T16:07:38.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:37 smithi130 ceph-mon[98670]: pgmap v433: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 MiB/s wr, 3.64k op/s 2023-09-12T16:07:38.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:37 smithi159 ceph-mon[102522]: pgmap v433: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 MiB/s wr, 3.64k op/s 2023-09-12T16:07:38.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:37 smithi203 ceph-mon[103431]: pgmap v433: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 MiB/s wr, 3.64k op/s 2023-09-12T16:07:40.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 110.0 s, 1362.9 tps, lat 72.275 ms stddev 101.223 2023-09-12T16:07:40.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:40 smithi130 ceph-mon[98670]: pgmap v434: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 50 MiB/s wr, 3.09k op/s 2023-09-12T16:07:40.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:40 smithi159 ceph-mon[102522]: pgmap v434: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 50 MiB/s wr, 3.09k op/s 2023-09-12T16:07:40.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:40 smithi203 ceph-mon[103431]: pgmap v434: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 50 MiB/s wr, 3.09k op/s 2023-09-12T16:07:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:42 smithi130 ceph-mon[98670]: pgmap v435: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 MiB/s wr, 2.11k op/s 2023-09-12T16:07:42.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:42 smithi159 ceph-mon[102522]: pgmap v435: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 MiB/s wr, 2.11k op/s 2023-09-12T16:07:42.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:42 smithi203 ceph-mon[103431]: pgmap v435: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 MiB/s wr, 2.11k op/s 2023-09-12T16:07:44.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:44 smithi130 ceph-mon[98670]: pgmap v436: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.84k op/s 2023-09-12T16:07:44.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:44 smithi159 ceph-mon[102522]: pgmap v436: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.84k op/s 2023-09-12T16:07:44.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:44 smithi203 ceph-mon[103431]: pgmap v436: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.84k op/s 2023-09-12T16:07:45.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 115.0 s, 1547.6 tps, lat 65.603 ms stddev 174.497 2023-09-12T16:07:46.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:46 smithi130 ceph-mon[98670]: pgmap v437: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 1.83k op/s 2023-09-12T16:07:46.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:46 smithi159 ceph-mon[102522]: pgmap v437: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 1.83k op/s 2023-09-12T16:07:46.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:46 smithi203 ceph-mon[103431]: pgmap v437: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 1.83k op/s 2023-09-12T16:07:48.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:47 smithi159 ceph-mon[102522]: pgmap v438: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 MiB/s wr, 1.90k op/s 2023-09-12T16:07:48.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:47 smithi130 ceph-mon[98670]: pgmap v438: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 MiB/s wr, 1.90k op/s 2023-09-12T16:07:48.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:47 smithi203 ceph-mon[103431]: pgmap v438: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 MiB/s wr, 1.90k op/s 2023-09-12T16:07:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 120.0 s, 547.4 tps, lat 182.407 ms stddev 320.920 2023-09-12T16:07:50.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:50 smithi159 ceph-mon[102522]: pgmap v439: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 42 MiB/s wr, 2.64k op/s 2023-09-12T16:07:50.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:50 smithi130 ceph-mon[98670]: pgmap v439: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 42 MiB/s wr, 2.64k op/s 2023-09-12T16:07:50.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:50 smithi203 ceph-mon[103431]: pgmap v439: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 42 MiB/s wr, 2.64k op/s 2023-09-12T16:07:52.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:52 smithi130 ceph-mon[98670]: pgmap v440: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 MiB/s wr, 2.79k op/s 2023-09-12T16:07:52.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:52 smithi159 ceph-mon[102522]: pgmap v440: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 MiB/s wr, 2.79k op/s 2023-09-12T16:07:52.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:52 smithi203 ceph-mon[103431]: pgmap v440: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 MiB/s wr, 2.79k op/s 2023-09-12T16:07:54.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:54 smithi130 ceph-mon[98670]: pgmap v441: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 MiB/s wr, 3.53k op/s 2023-09-12T16:07:54.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:54 smithi159 ceph-mon[102522]: pgmap v441: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 MiB/s wr, 3.53k op/s 2023-09-12T16:07:54.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:54 smithi203 ceph-mon[103431]: pgmap v441: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 MiB/s wr, 3.53k op/s 2023-09-12T16:07:55.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 125.0 s, 2431.6 tps, lat 41.155 ms stddev 49.781 2023-09-12T16:07:56.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:56 smithi159 ceph-mon[102522]: pgmap v442: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 47 MiB/s wr, 3.25k op/s 2023-09-12T16:07:56.501 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:56 smithi203 ceph-mon[103431]: pgmap v442: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 47 MiB/s wr, 3.25k op/s 2023-09-12T16:07:56.529 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:56 smithi130 ceph-mon[98670]: pgmap v442: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 47 MiB/s wr, 3.25k op/s 2023-09-12T16:07:57.072 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:07:58.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:07:57 smithi159 ceph-mon[102522]: pgmap v443: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 45 MiB/s wr, 2.98k op/s 2023-09-12T16:07:58.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:07:57 smithi203 ceph-mon[103431]: pgmap v443: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 45 MiB/s wr, 2.98k op/s 2023-09-12T16:07:58.265 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:07:57 smithi130 ceph-mon[98670]: pgmap v443: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 45 MiB/s wr, 2.98k op/s 2023-09-12T16:08:00.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 130.0 s, 508.9 tps, lat 189.991 ms stddev 544.255 2023-09-12T16:08:00.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:00 smithi130 ceph-mon[98670]: pgmap v444: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 50 MiB/s wr, 3.37k op/s 2023-09-12T16:08:00.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:00 smithi159 ceph-mon[102522]: pgmap v444: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 50 MiB/s wr, 3.37k op/s 2023-09-12T16:08:00.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:00 smithi203 ceph-mon[103431]: pgmap v444: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 50 MiB/s wr, 3.37k op/s 2023-09-12T16:08:02.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:02 smithi159 ceph-mon[102522]: pgmap v445: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 43 MiB/s wr, 2.90k op/s 2023-09-12T16:08:02.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:02 smithi130 ceph-mon[98670]: pgmap v445: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 43 MiB/s wr, 2.90k op/s 2023-09-12T16:08:02.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:02 smithi203 ceph-mon[103431]: pgmap v445: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 43 MiB/s wr, 2.90k op/s 2023-09-12T16:08:02.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.882+0000 7fd072c71700 1 -- 172.21.15.130:0/4140726368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 msgr2=0x7fd06c100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:02.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.882+0000 7fd072c71700 1 --2- 172.21.15.130:0/4140726368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 0x7fd06c100620 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd064009f90 tx=0x7fd06400b330 comp rx=0 tx=0).stop 2023-09-12T16:08:02.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.882+0000 7fd072c71700 1 -- 172.21.15.130:0/4140726368 shutdown_connections 2023-09-12T16:08:02.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.882+0000 7fd072c71700 1 --2- 172.21.15.130:0/4140726368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 0x7fd06c100620 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.882+0000 7fd072c71700 1 --2- 172.21.15.130:0/4140726368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd06c0fee80 0x7fd06c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.882+0000 7fd072c71700 1 --2- 172.21.15.130:0/4140726368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd06c0ff7e0 0x7fd06c0ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.882+0000 7fd072c71700 1 -- 172.21.15.130:0/4140726368 >> 172.21.15.130:0/4140726368 conn(0x7fd06c0faa60 msgr2=0x7fd06c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:02.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.910+0000 7fd072c71700 1 -- 172.21.15.130:0/4140726368 shutdown_connections 2023-09-12T16:08:02.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.916+0000 7fd072c71700 1 -- 172.21.15.130:0/4140726368 wait complete. 2023-09-12T16:08:02.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.917+0000 7fd072c71700 1 Processor -- start 2023-09-12T16:08:02.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.924+0000 7fd072c71700 1 -- start start 2023-09-12T16:08:02.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.924+0000 7fd072c71700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd06c0fee80 0x7fd06c1036e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.924+0000 7fd072c71700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd06c0ff7e0 0x7fd06c103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.924+0000 7fd072c71700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 0x7fd06c10ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.924+0000 7fd072c71700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c10b260 con 0x7fd06c1001a0 2023-09-12T16:08:02.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.924+0000 7fd072c71700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c06a020 con 0x7fd06c0ff7e0 2023-09-12T16:08:02.927 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.924+0000 7fd072c71700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c06a1b0 con 0x7fd06c0fee80 2023-09-12T16:08:02.929 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.928+0000 7fd07120e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 0x7fd06c10ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:02.929 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.928+0000 7fd07120e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 0x7fd06c10ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55934/0 (socket says 172.21.15.130:55934) 2023-09-12T16:08:02.929 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.928+0000 7fd07120e700 1 -- 172.21.15.130:0/268161591 learned_addr learned my addr 172.21.15.130:0/268161591 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:02.930 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.929+0000 7fd07120e700 1 -- 172.21.15.130:0/268161591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd06c0fee80 msgr2=0x7fd06c1036e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-12T16:08:02.930 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.929+0000 7fd07120e700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd06c0fee80 0x7fd06c1036e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.930 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.929+0000 7fd07120e700 1 -- 172.21.15.130:0/268161591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd06c0ff7e0 msgr2=0x7fd06c103c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:02.930 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.929+0000 7fd07120e700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd06c0ff7e0 0x7fd06c103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.931 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.929+0000 7fd07120e700 1 -- 172.21.15.130:0/268161591 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064009c40 con 0x7fd06c1001a0 2023-09-12T16:08:02.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.932+0000 7fd07120e700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 0x7fd06c10ac90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd064009b10 tx=0x7fd064004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:02.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.933+0000 7fd069ffb700 1 -- 172.21.15.130:0/268161591 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400e040 con 0x7fd06c1001a0 2023-09-12T16:08:02.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.933+0000 7fd069ffb700 1 -- 172.21.15.130:0/268161591 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd06401d930 con 0x7fd06c1001a0 2023-09-12T16:08:02.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.933+0000 7fd072c71700 1 -- 172.21.15.130:0/268161591 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c06a430 con 0x7fd06c1001a0 2023-09-12T16:08:02.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.933+0000 7fd069ffb700 1 -- 172.21.15.130:0/268161591 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06401b770 con 0x7fd06c1001a0 2023-09-12T16:08:02.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.934+0000 7fd072c71700 1 -- 172.21.15.130:0/268161591 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c06a960 con 0x7fd06c1001a0 2023-09-12T16:08:02.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.938+0000 7fd07ee37b80 1 Processor -- start 2023-09-12T16:08:02.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.938+0000 7fd07ee37b80 1 -- start start 2023-09-12T16:08:02.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653930ca80 0x55653930cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653930d3e0 0x55653930da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930e010 0x55653930e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556539265730 con 0x55653930ca80 2023-09-12T16:08:02.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556539263050 con 0x55653930d3e0 2023-09-12T16:08:02.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653925e940 con 0x55653930e010 2023-09-12T16:08:02.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930e010 0x55653930e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:02.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653930ca80 0x55653930cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:02.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653930ca80 0x55653930cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55936/0 (socket says 172.21.15.130:55936) 2023-09-12T16:08:02.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.939+0000 7fd06bfff700 1 -- 172.21.15.130:0/1843361581 learned_addr learned my addr 172.21.15.130:0/1843361581 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:02.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.940+0000 7fd06bfff700 1 -- 172.21.15.130:0/1843361581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653930d3e0 msgr2=0x55653930da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:02.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.940+0000 7fd06bfff700 1 --2- 172.21.15.130:0/1843361581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653930d3e0 0x55653930da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.940+0000 7fd06bfff700 1 -- 172.21.15.130:0/1843361581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653930ca80 msgr2=0x55653930cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:02.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.940+0000 7fd06bfff700 1 --2- 172.21.15.130:0/1843361581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653930ca80 0x55653930cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.940+0000 7fd06bfff700 1 -- 172.21.15.130:0/1843361581 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653908ade0 con 0x55653930e010 2023-09-12T16:08:02.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.940+0000 7fd06bfff700 1 --2- 172.21.15.130:0/1843361581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930e010 0x55653930e490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd06000af90 tx=0x7fd0600090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b4bb5dcc1f41d8a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:02.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.945+0000 7fd072c71700 1 -- 172.21.15.130:0/1843361581 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060018070 con 0x55653930e010 2023-09-12T16:08:02.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.945+0000 7fd072c71700 1 -- 172.21.15.130:0/1843361581 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd060012e20 con 0x55653930e010 2023-09-12T16:08:02.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.945+0000 7fd072c71700 1 -- 172.21.15.130:0/1843361581 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060006330 con 0x55653930e010 2023-09-12T16:08:02.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.946+0000 7fd069ffb700 1 -- 172.21.15.130:0/268161591 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fd06401d470 con 0x7fd06c1001a0 2023-09-12T16:08:02.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.946+0000 7fd07ee37b80 1 -- 172.21.15.130:0/1843361581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930e010 msgr2=0x55653930e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:02.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.946+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/1843361581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930e010 0x55653930e490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd06000af90 tx=0x7fd0600090e0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.946+0000 7fd07ee37b80 1 -- 172.21.15.130:0/1843361581 shutdown_connections 2023-09-12T16:08:02.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.946+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/1843361581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930e010 0x55653930e490 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd06000af90 tx=0x7fd0600090e0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.946+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/1843361581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653930ca80 0x55653930cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.946+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/1843361581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653930d3e0 0x55653930da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.946+0000 7fd07ee37b80 1 -- 172.21.15.130:0/1843361581 >> 172.21.15.130:0/1843361581 conn(0x556539281090 msgr2=0x556539307750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:02.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.947+0000 7fd069ffb700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0540a0790 0x7fd0540a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.947+0000 7fd07ee37b80 1 -- 172.21.15.130:0/1843361581 shutdown_connections 2023-09-12T16:08:02.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.947+0000 7fd069ffb700 1 -- 172.21.15.130:0/268161591 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd0640ccfa0 con 0x7fd06c1001a0 2023-09-12T16:08:02.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.947+0000 7fd070a0d700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0540a0790 0x7fd0540a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:02.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.948+0000 7fd070a0d700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0540a0790 0x7fd0540a2c50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd05c005fd0 tx=0x7fd05c0057d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:02.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.957+0000 7fd07ee37b80 1 -- 172.21.15.130:0/1843361581 wait complete. 2023-09-12T16:08:02.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.958+0000 7fd07ee37b80 1 -- start start 2023-09-12T16:08:02.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.958+0000 7fd07ee37b80 1 Processor -- start 2023-09-12T16:08:02.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.958+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653930ca80 0x5565393167d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.958+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930d3e0 0x556539316d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.958+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653931bd40 0x55653931e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.958+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565391fcb70 con 0x55653931bd40 2023-09-12T16:08:02.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.958+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565391fc870 con 0x55653930ca80 2023-09-12T16:08:02.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.958+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565390e0920 con 0x55653930d3e0 2023-09-12T16:08:02.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.958+0000 7fd07ee37b80 10 client.0 _ll_register_callbacks cb 0x556539146a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:08:02.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.959+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930d3e0 0x556539316d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:02.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.959+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930d3e0 0x556539316d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35556/0 (socket says 172.21.15.130:35556) 2023-09-12T16:08:02.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.959+0000 7fd06bfff700 1 -- 172.21.15.130:0/3865081673 learned_addr learned my addr 172.21.15.130:0/3865081673 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:02.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.965+0000 7fd06bfff700 1 -- 172.21.15.130:0/3865081673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653930ca80 msgr2=0x5565393167d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:08:02.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.965+0000 7fd06bfff700 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653930ca80 0x5565393167d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.965+0000 7fd06bfff700 1 -- 172.21.15.130:0/3865081673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653931bd40 msgr2=0x55653931e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:02.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.965+0000 7fd06bfff700 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653931bd40 0x55653931e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:02.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.965+0000 7fd06bfff700 1 -- 172.21.15.130:0/3865081673 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd060012360 con 0x55653930d3e0 2023-09-12T16:08:02.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.971+0000 7fd06bfff700 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930d3e0 0x556539316d10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd06000a200 tx=0x7fd060004c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8abe4aed7b5b8b03 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:02.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.971+0000 7fd04bfff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:08:02.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.971+0000 7fd04bfff700 1 -- 172.21.15.130:0/3865081673 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060018040 con 0x55653930d3e0 2023-09-12T16:08:02.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.971+0000 7fd04bfff700 1 -- 172.21.15.130:0/3865081673 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd060004430 con 0x55653930d3e0 2023-09-12T16:08:02.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.971+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3865081673 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565391b64f0 con 0x55653930d3e0 2023-09-12T16:08:02.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.972+0000 7fd04bfff700 1 -- 172.21.15.130:0/3865081673 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd060003680 con 0x55653930d3e0 2023-09-12T16:08:02.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.981+0000 7fd07ee37b80 10 client.25636 fetch_fsmap learned FSMap version 32 2023-09-12T16:08:02.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.981+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3865081673 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556539073c60 con 0x55653930d3e0 2023-09-12T16:08:02.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.981+0000 7fd04bfff700 1 -- 172.21.15.130:0/3865081673 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565391b64f0 con 0x55653930d3e0 2023-09-12T16:08:02.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.981+0000 7fd04bfff700 1 -- 172.21.15.130:0/3865081673 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd0600063d0 con 0x55653930d3e0 2023-09-12T16:08:02.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.993+0000 7fd07ee37b80 10 client.25636 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:08:02.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.993+0000 7fd07ee37b80 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:08:02.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.993+0000 7fd07ee37b80 20 client.25636 populate_metadata read hostname 'smithi130' 2023-09-12T16:08:02.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.993+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556539322810 0x556539324c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:02.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.993+0000 7fd07ee37b80 4 client.25636 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:08:02.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.993+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3865081673 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565392738a0 con 0x556539322810 2023-09-12T16:08:02.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.994+0000 7fd070a0d700 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556539322810 0x556539324c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:02.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.997+0000 7fd070a0d700 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556539322810 0x556539324c30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e64d070dfe2752 server_cookie=624550e3e06f794e in_seq=0 out_seq=0 2023-09-12T16:08:02.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.997+0000 7fd04bfff700 10 client.25636 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:08:02.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.998+0000 7fd04bfff700 1 -- 172.21.15.130:0/3865081673 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5565392738a0 con 0x556539322810 2023-09-12T16:08:02.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.998+0000 7fd04bfff700 10 client.25636 handle_command_reply: tid=0 2023-09-12T16:08:02.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:02.998+0000 7fd07ee37b80 1 client.25636 shutdown 2023-09-12T16:08:03.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.004+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3865081673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930d3e0 msgr2=0x556539316d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:03.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.004+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930d3e0 0x556539316d10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd06000a200 tx=0x7fd060004c00 comp rx=0 tx=0).stop 2023-09-12T16:08:03.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.008+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3865081673 shutdown_connections 2023-09-12T16:08:03.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.008+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556539322810 0x556539324c30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.008+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653931bd40 0x55653931e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.008+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653930ca80 0x5565393167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.008+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3865081673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930d3e0 0x556539316d10 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd06000a200 tx=0x7fd060004c00 comp rx=0 tx=0).stop 2023-09-12T16:08:03.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.008+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3865081673 >> 172.21.15.130:0/3865081673 conn(0x556539281090 msgr2=0x556539306df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:03.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.008+0000 7fd04bfff700 0 client.25636 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:08:03.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.012+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3865081673 shutdown_connections 2023-09-12T16:08:03.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.012+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3865081673 wait complete. 2023-09-12T16:08:03.015 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.015+0000 7fd07ee37b80 20 client.25636 trim_cache size 0 max 16384 2023-09-12T16:08:03.046 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.045+0000 7fd07ee37b80 1 Processor -- start 2023-09-12T16:08:03.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.045+0000 7fd07ee37b80 1 -- start start 2023-09-12T16:08:03.047 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.045+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539332450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:03.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.045+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653931bd40 0x55653931ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:03.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.045+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556539303820 0x556539310480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:03.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.045+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565393163a0 con 0x55653931bd40 2023-09-12T16:08:03.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.045+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653930c710 con 0x556539303820 2023-09-12T16:08:03.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.045+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565393043d0 con 0x55653930ca80 2023-09-12T16:08:03.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd070a0d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539332450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:03.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd070a0d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539332450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35564/0 (socket says 172.21.15.130:35564) 2023-09-12T16:08:03.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd070a0d700 1 -- 172.21.15.130:0/3372552409 learned_addr learned my addr 172.21.15.130:0/3372552409 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:03.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd07120e700 1 --2- 172.21.15.130:0/3372552409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556539303820 0x556539310480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:03.049 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd07120e700 1 -- 172.21.15.130:0/3372552409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 msgr2=0x556539332450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:03.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd07120e700 1 --2- 172.21.15.130:0/3372552409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539332450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd07120e700 1 -- 172.21.15.130:0/3372552409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653931bd40 msgr2=0x55653931ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:03.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd07120e700 1 --2- 172.21.15.130:0/3372552409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653931bd40 0x55653931ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd07120e700 1 -- 172.21.15.130:0/3372552409 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565390b1250 con 0x556539303820 2023-09-12T16:08:03.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.046+0000 7fd070a0d700 1 --2- 172.21.15.130:0/3372552409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539332450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:08:03.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.048+0000 7fd07120e700 1 --2- 172.21.15.130:0/3372552409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556539303820 0x556539310480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x556539317880 tx=0x7fd064099f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a12c4efe43582d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:03.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.048+0000 7fd052ffd700 1 -- 172.21.15.130:0/3372552409 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06409ab50 con 0x556539303820 2023-09-12T16:08:03.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.048+0000 7fd052ffd700 1 -- 172.21.15.130:0/3372552409 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd06409a450 con 0x556539303820 2023-09-12T16:08:03.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.048+0000 7fd052ffd700 1 -- 172.21.15.130:0/3372552409 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0640987f0 con 0x556539303820 2023-09-12T16:08:03.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.051+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3372552409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556539303820 msgr2=0x556539310480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:03.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.051+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3372552409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556539303820 0x556539310480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x556539317880 tx=0x7fd064099f90 comp rx=0 tx=0).stop 2023-09-12T16:08:03.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.052+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3372552409 shutdown_connections 2023-09-12T16:08:03.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.052+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3372552409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556539303820 0x556539310480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.052+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3372552409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539332450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.052+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3372552409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55653931bd40 0x55653931ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.052+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3372552409 >> 172.21.15.130:0/3372552409 conn(0x556539281090 msgr2=0x55653930d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:03.054 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.053+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3372552409 shutdown_connections 2023-09-12T16:08:03.055 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.053+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3372552409 wait complete. 2023-09-12T16:08:03.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.053+0000 7fd07ee37b80 1 -- start start 2023-09-12T16:08:03.056 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.053+0000 7fd07ee37b80 1 Processor -- start 2023-09-12T16:08:03.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556539303820 0x556539332450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:03.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539372d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:03.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd07ee37b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653931bd40 0x5565393752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:03.057 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556539375940 con 0x556539303820 2023-09-12T16:08:03.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556539375ab0 con 0x55653931bd40 2023-09-12T16:08:03.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd07ee37b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556539375c70 con 0x55653930ca80 2023-09-12T16:08:03.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd07ee37b80 10 client.0 _ll_register_callbacks cb 0x7fd060012580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:08:03.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd070a0d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556539303820 0x556539332450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:03.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd070a0d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556539303820 0x556539332450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55960/0 (socket says 172.21.15.130:55960) 2023-09-12T16:08:03.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd070a0d700 1 -- 172.21.15.130:0/3338943254 learned_addr learned my addr 172.21.15.130:0/3338943254 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:03.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd06bfff700 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539372d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:03.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.054+0000 7fd07120e700 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653931bd40 0x5565393752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:03.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd070a0d700 1 -- 172.21.15.130:0/3338943254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 msgr2=0x556539372d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:03.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd070a0d700 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539372d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd070a0d700 1 -- 172.21.15.130:0/3338943254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653931bd40 msgr2=0x5565393752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:03.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd070a0d700 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653931bd40 0x5565393752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd070a0d700 1 -- 172.21.15.130:0/3338943254 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c00e740 con 0x556539303820 2023-09-12T16:08:03.060 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd070a0d700 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556539303820 0x556539332450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00ec50 tx=0x7fd05c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841ef9df8ad3ec2c server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:03.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd04bfff700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:08:03.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd04bfff700 1 -- 172.21.15.130:0/3338943254 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c023810 con 0x556539303820 2023-09-12T16:08:03.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd04bfff700 1 -- 172.21.15.130:0/3338943254 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd05c023970 con 0x556539303820 2023-09-12T16:08:03.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.055+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3338943254 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd060012e20 con 0x556539303820 2023-09-12T16:08:03.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.057+0000 7fd04bfff700 1 -- 172.21.15.130:0/3338943254 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd05c023b30 con 0x556539303820 2023-09-12T16:08:03.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.067+0000 7fd07ee37b80 10 client.15969 fetch_fsmap learned FSMap version 32 2023-09-12T16:08:03.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.067+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3338943254 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fd06c06a960 con 0x556539303820 2023-09-12T16:08:03.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.068+0000 7fd04bfff700 1 -- 172.21.15.130:0/3338943254 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd060012e20 con 0x556539303820 2023-09-12T16:08:03.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.068+0000 7fd04bfff700 1 -- 172.21.15.130:0/3338943254 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd05c01c420 con 0x556539303820 2023-09-12T16:08:03.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.078+0000 7fd07ee37b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:08:03.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.078+0000 7fd07ee37b80 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:08:03.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.078+0000 7fd07ee37b80 20 client.15969 populate_metadata read hostname 'smithi130' 2023-09-12T16:08:03.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.078+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556539377680 0x556539379aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:03.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.078+0000 7fd07ee37b80 4 client.15969 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:08:03.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.078+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3338943254 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556539257920 con 0x556539377680 2023-09-12T16:08:03.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.078+0000 7fd06bfff700 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556539377680 0x556539379aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:03.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.086+0000 7fd06bfff700 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556539377680 0x556539379aa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac82d4b7d0ae6355 server_cookie=3dc71b6ebe987dc1 in_seq=0 out_seq=0 2023-09-12T16:08:03.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.086+0000 7fd04bfff700 10 client.15969 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:08:03.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.106+0000 7fd04bfff700 1 -- 172.21.15.130:0/3338943254 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556539257920 con 0x556539377680 2023-09-12T16:08:03.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.106+0000 7fd04bfff700 10 client.15969 handle_command_reply: tid=0 2023-09-12T16:08:03.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.108+0000 7fd07ee37b80 1 client.15969 shutdown 2023-09-12T16:08:03.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.109+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3338943254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556539303820 msgr2=0x556539332450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:03.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.109+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556539303820 0x556539332450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00ec50 tx=0x7fd05c009310 comp rx=0 tx=0).stop 2023-09-12T16:08:03.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.110+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3338943254 shutdown_connections 2023-09-12T16:08:03.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.110+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556539377680 0x556539379aa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.110+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653931bd40 0x5565393752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.110+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556539303820 0x556539332450 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.110+0000 7fd07ee37b80 1 --2- 172.21.15.130:0/3338943254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55653930ca80 0x556539372d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.110+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3338943254 >> 172.21.15.130:0/3338943254 conn(0x556539281090 msgr2=0x556539310480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:03.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.110+0000 7fd04bfff700 0 client.15969 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:08:03.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.110+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3338943254 shutdown_connections 2023-09-12T16:08:03.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.110+0000 7fd07ee37b80 1 -- 172.21.15.130:0/3338943254 wait complete. 2023-09-12T16:08:03.132 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:08:03.132 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:08:03.132 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:08:03.133 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:08:03.133 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:08:03.133 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:08:03.133 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:08:03.133 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:08:03.133 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:08:03.134 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:08:03.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.131+0000 7fd07ee37b80 20 client.15969 trim_cache size 0 max 16384 2023-09-12T16:08:03.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.133+0000 7fd0537fe700 1 -- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0540a0790 msgr2=0x7fd0540a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:03.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.133+0000 7fd0537fe700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0540a0790 0x7fd0540a2c50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd05c005fd0 tx=0x7fd05c0057d0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.133+0000 7fd0537fe700 1 -- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 msgr2=0x7fd06c10ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:03.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.133+0000 7fd0537fe700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 0x7fd06c10ac90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd064009b10 tx=0x7fd064004d30 comp rx=0 tx=0).stop 2023-09-12T16:08:03.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.134+0000 7fd0537fe700 1 -- 172.21.15.130:0/268161591 shutdown_connections 2023-09-12T16:08:03.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.134+0000 7fd0537fe700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0540a0790 0x7fd0540a2c50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.134+0000 7fd0537fe700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd06c1001a0 0x7fd06c10ac90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.134+0000 7fd0537fe700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd06c0fee80 0x7fd06c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.134+0000 7fd0537fe700 1 --2- 172.21.15.130:0/268161591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd06c0ff7e0 0x7fd06c103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:03.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.134+0000 7fd0537fe700 1 -- 172.21.15.130:0/268161591 >> 172.21.15.130:0/268161591 conn(0x7fd06c0faa60 msgr2=0x7fd06c0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:03.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.139+0000 7fd0537fe700 1 -- 172.21.15.130:0/268161591 shutdown_connections 2023-09-12T16:08:03.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:03.142+0000 7fd0537fe700 1 -- 172.21.15.130:0/268161591 wait complete. 2023-09-12T16:08:03.184 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:08:04.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:04 smithi130 ceph-mon[98670]: pgmap v446: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 45 MiB/s wr, 3.35k op/s 2023-09-12T16:08:04.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:04 smithi159 ceph-mon[102522]: pgmap v446: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 45 MiB/s wr, 3.35k op/s 2023-09-12T16:08:04.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:04 smithi203 ceph-mon[103431]: pgmap v446: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 45 MiB/s wr, 3.35k op/s 2023-09-12T16:08:05.094 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 135.0 s, 711.8 tps, lat 143.643 ms stddev 153.707 2023-09-12T16:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:06 smithi159 ceph-mon[102522]: pgmap v447: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 36 MiB/s wr, 2.65k op/s 2023-09-12T16:08:06.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:06 smithi130 ceph-mon[98670]: pgmap v447: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 36 MiB/s wr, 2.65k op/s 2023-09-12T16:08:06.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:06 smithi203 ceph-mon[103431]: pgmap v447: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 36 MiB/s wr, 2.65k op/s 2023-09-12T16:08:08.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:07 smithi130 ceph-mon[98670]: pgmap v448: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 34 MiB/s wr, 2.55k op/s 2023-09-12T16:08:08.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:07 smithi159 ceph-mon[102522]: pgmap v448: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 34 MiB/s wr, 2.55k op/s 2023-09-12T16:08:08.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:07 smithi203 ceph-mon[103431]: pgmap v448: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 34 MiB/s wr, 2.55k op/s 2023-09-12T16:08:10.098 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 140.0 s, 2042.6 tps, lat 49.457 ms stddev 42.393 2023-09-12T16:08:10.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:10 smithi130 ceph-mon[98670]: pgmap v449: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 33 MiB/s wr, 2.55k op/s 2023-09-12T16:08:10.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:10 smithi159 ceph-mon[102522]: pgmap v449: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 33 MiB/s wr, 2.55k op/s 2023-09-12T16:08:10.500 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:10 smithi203 ceph-mon[103431]: pgmap v449: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 33 MiB/s wr, 2.55k op/s 2023-09-12T16:08:12.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:12 smithi159 ceph-mon[102522]: pgmap v450: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2023-09-12T16:08:12.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:12 smithi130 ceph-mon[98670]: pgmap v450: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2023-09-12T16:08:12.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:12 smithi203 ceph-mon[103431]: pgmap v450: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2023-09-12T16:08:14.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:14 smithi130 ceph-mon[98670]: pgmap v451: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 30 MiB/s wr, 2.01k op/s 2023-09-12T16:08:14.482 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:14 smithi159 ceph-mon[102522]: pgmap v451: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 30 MiB/s wr, 2.01k op/s 2023-09-12T16:08:14.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:14 smithi203 ceph-mon[103431]: pgmap v451: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 30 MiB/s wr, 2.01k op/s 2023-09-12T16:08:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 145.0 s, 861.0 tps, lat 115.021 ms stddev 200.466 2023-09-12T16:08:16.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:16 smithi130 ceph-mon[98670]: pgmap v452: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 28 MiB/s wr, 1.69k op/s 2023-09-12T16:08:16.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:16 smithi159 ceph-mon[102522]: pgmap v452: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 28 MiB/s wr, 1.69k op/s 2023-09-12T16:08:16.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:16 smithi203 ceph-mon[103431]: pgmap v452: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 28 MiB/s wr, 1.69k op/s 2023-09-12T16:08:18.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:17 smithi130 ceph-mon[98670]: pgmap v453: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 32 MiB/s wr, 1.93k op/s 2023-09-12T16:08:18.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:17 smithi159 ceph-mon[102522]: pgmap v453: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 32 MiB/s wr, 1.93k op/s 2023-09-12T16:08:18.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:17 smithi203 ceph-mon[103431]: pgmap v453: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 32 MiB/s wr, 1.93k op/s 2023-09-12T16:08:19.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:18 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:08:19.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:18 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:08:19.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:18 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:08:20.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 150.0 s, 2244.2 tps, lat 45.010 ms stddev 37.186 2023-09-12T16:08:20.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:19 smithi159 ceph-mon[102522]: pgmap v454: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 37 MiB/s wr, 2.04k op/s 2023-09-12T16:08:20.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:08:20.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:08:20.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:19 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:08:20.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:19 smithi130 ceph-mon[98670]: pgmap v454: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 37 MiB/s wr, 2.04k op/s 2023-09-12T16:08:20.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:08:20.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:08:20.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:19 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:08:20.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:19 smithi203 ceph-mon[103431]: pgmap v454: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 37 MiB/s wr, 2.04k op/s 2023-09-12T16:08:20.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:08:20.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:08:20.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:19 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:08:22.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:22 smithi130 ceph-mon[98670]: pgmap v455: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 31 MiB/s wr, 1.85k op/s 2023-09-12T16:08:22.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:22 smithi159 ceph-mon[102522]: pgmap v455: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 31 MiB/s wr, 1.85k op/s 2023-09-12T16:08:22.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:22 smithi203 ceph-mon[103431]: pgmap v455: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 31 MiB/s wr, 1.85k op/s 2023-09-12T16:08:24.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:24 smithi130 ceph-mon[98670]: pgmap v456: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 37 MiB/s wr, 1.99k op/s 2023-09-12T16:08:24.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:24 smithi159 ceph-mon[102522]: pgmap v456: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 37 MiB/s wr, 1.99k op/s 2023-09-12T16:08:24.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:24 smithi203 ceph-mon[103431]: pgmap v456: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 37 MiB/s wr, 1.99k op/s 2023-09-12T16:08:25.097 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 155.0 s, 973.6 tps, lat 102.333 ms stddev 109.615 2023-09-12T16:08:26.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:26 smithi159 ceph-mon[102522]: pgmap v457: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 30 MiB/s wr, 1.56k op/s 2023-09-12T16:08:26.488 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:26 smithi130 ceph-mon[98670]: pgmap v457: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 30 MiB/s wr, 1.56k op/s 2023-09-12T16:08:26.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:26 smithi203 ceph-mon[103431]: pgmap v457: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 30 MiB/s wr, 1.56k op/s 2023-09-12T16:08:28.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:27 smithi130 ceph-mon[98670]: pgmap v458: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 28 MiB/s wr, 1.24k op/s 2023-09-12T16:08:28.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:27 smithi159 ceph-mon[102522]: pgmap v458: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 28 MiB/s wr, 1.24k op/s 2023-09-12T16:08:28.252 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:27 smithi203 ceph-mon[103431]: pgmap v458: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 28 MiB/s wr, 1.24k op/s 2023-09-12T16:08:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 160.0 s, 1469.2 tps, lat 67.855 ms stddev 51.852 2023-09-12T16:08:30.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:30 smithi159 ceph-mon[102522]: pgmap v459: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 28 MiB/s wr, 1.13k op/s 2023-09-12T16:08:30.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:30 smithi130 ceph-mon[98670]: pgmap v459: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 28 MiB/s wr, 1.13k op/s 2023-09-12T16:08:30.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:30 smithi203 ceph-mon[103431]: pgmap v459: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 28 MiB/s wr, 1.13k op/s 2023-09-12T16:08:32.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:32 smithi130 ceph-mon[98670]: pgmap v460: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 20 MiB/s wr, 858 op/s 2023-09-12T16:08:32.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:32 smithi159 ceph-mon[102522]: pgmap v460: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 20 MiB/s wr, 858 op/s 2023-09-12T16:08:32.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:32 smithi203 ceph-mon[103431]: pgmap v460: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 20 MiB/s wr, 858 op/s 2023-09-12T16:08:33.185 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:08:33.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.614+0000 7f2556268700 1 -- 172.21.15.130:0/2856825142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 msgr2=0x7f255010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:33.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.614+0000 7f2556268700 1 --2- 172.21.15.130:0/2856825142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 0x7f255010cdc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2548009f90 tx=0x7f254800b330 comp rx=0 tx=0).stop 2023-09-12T16:08:33.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.618+0000 7f2556268700 1 -- 172.21.15.130:0/2856825142 shutdown_connections 2023-09-12T16:08:33.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.618+0000 7f2556268700 1 --2- 172.21.15.130:0/2856825142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 0x7f255010cdc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.618+0000 7f2556268700 1 --2- 172.21.15.130:0/2856825142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2550107600 0x7f2550109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.618+0000 7f2556268700 1 --2- 172.21.15.130:0/2856825142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2550109f70 0x7f255010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.618+0000 7f2556268700 1 -- 172.21.15.130:0/2856825142 >> 172.21.15.130:0/2856825142 conn(0x7f255006d3b0 msgr2=0x7f255006f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:33.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.618+0000 7f2556268700 1 -- 172.21.15.130:0/2856825142 shutdown_connections 2023-09-12T16:08:33.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.618+0000 7f2556268700 1 -- 172.21.15.130:0/2856825142 wait complete. 2023-09-12T16:08:33.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.619+0000 7f2556268700 1 Processor -- start 2023-09-12T16:08:33.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.619+0000 7f2556268700 1 -- start start 2023-09-12T16:08:33.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.619+0000 7f2556268700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2550107600 0x7f255019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.619+0000 7f2556268700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2550109f70 0x7f255019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.619+0000 7f2556268700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 0x7f25501a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.619+0000 7f2556268700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501a4350 con 0x7f255010c940 2023-09-12T16:08:33.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.619+0000 7f2556268700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501a4490 con 0x7f2550107600 2023-09-12T16:08:33.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.619+0000 7f2556268700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501a4620 con 0x7f2550109f70 2023-09-12T16:08:33.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.619+0000 7f2554805700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 0x7f25501a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.620+0000 7f2554805700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 0x7f25501a3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:55974/0 (socket says 172.21.15.130:55974) 2023-09-12T16:08:33.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.620+0000 7f2554805700 1 -- 172.21.15.130:0/3170468888 learned_addr learned my addr 172.21.15.130:0/3170468888 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:33.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.620+0000 7f254f7fe700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2550109f70 0x7f255019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.620+0000 7f2554805700 1 -- 172.21.15.130:0/3170468888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2550109f70 msgr2=0x7f255019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:33.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.620+0000 7f2554805700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2550109f70 0x7f255019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.620+0000 7f2554805700 1 -- 172.21.15.130:0/3170468888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2550107600 msgr2=0x7f255019c7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-12T16:08:33.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.620+0000 7f2554805700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2550107600 0x7f255019c7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.620+0000 7f2554805700 1 -- 172.21.15.130:0/3170468888 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548009c40 con 0x7f255010c940 2023-09-12T16:08:33.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.621+0000 7f2554805700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 0x7f25501a3d80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f254800b770 tx=0x7f2548003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:33.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.624+0000 7f254d7fa700 1 -- 172.21.15.130:0/3170468888 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800e070 con 0x7f255010c940 2023-09-12T16:08:33.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.624+0000 7f254d7fa700 1 -- 172.21.15.130:0/3170468888 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254800ce90 con 0x7f255010c940 2023-09-12T16:08:33.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.624+0000 7f254d7fa700 1 -- 172.21.15.130:0/3170468888 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254803db30 con 0x7f255010c940 2023-09-12T16:08:33.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.626+0000 7f2556268700 1 -- 172.21.15.130:0/3170468888 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25501a48a0 con 0x7f255010c940 2023-09-12T16:08:33.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.626+0000 7f2556268700 1 -- 172.21.15.130:0/3170468888 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501a4cd0 con 0x7f255010c940 2023-09-12T16:08:33.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.632+0000 7f256242eb80 1 Processor -- start 2023-09-12T16:08:33.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.632+0000 7f254d7fa700 1 -- 172.21.15.130:0/3170468888 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f25480052c0 con 0x7f255010c940 2023-09-12T16:08:33.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.635+0000 7f254d7fa700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f25380a0840 0x7f25380a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.635+0000 7f254d7fa700 1 -- 172.21.15.130:0/3170468888 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f25480ef7e0 con 0x7f255010c940 2023-09-12T16:08:33.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f256242eb80 1 -- start start 2023-09-12T16:08:33.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f256242eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab8c00 0x563299ab9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f256242eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ab95a0 0x563299ab7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f256242eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 0x563299ab82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f256242eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x563299a10730 con 0x563299ab95a0 2023-09-12T16:08:33.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f256242eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563299a0e050 con 0x563299ab8c00 2023-09-12T16:08:33.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f256242eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563299a09940 con 0x563299ab7e00 2023-09-12T16:08:33.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f254ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab8c00 0x563299ab9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f254ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab8c00 0x563299ab9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39234/0 (socket says 172.21.15.130:39234) 2023-09-12T16:08:33.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f254ffff700 1 -- 172.21.15.130:0/1222877570 learned_addr learned my addr 172.21.15.130:0/1222877570 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:33.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.636+0000 7f2554805700 1 --2- 172.21.15.130:0/1222877570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 0x563299ab82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.637+0000 7f254f7fe700 1 --2- 172.21.15.130:0/1222877570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ab95a0 0x563299ab7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.637+0000 7f254f7fe700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f25380a0840 0x7f25380a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.637+0000 7f254ffff700 1 -- 172.21.15.130:0/1222877570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 msgr2=0x563299ab82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:33.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.637+0000 7f254ffff700 1 --2- 172.21.15.130:0/1222877570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 0x563299ab82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.637+0000 7f254ffff700 1 -- 172.21.15.130:0/1222877570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ab95a0 msgr2=0x563299ab7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:33.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.637+0000 7f254ffff700 1 --2- 172.21.15.130:0/1222877570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ab95a0 0x563299ab7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.637+0000 7f254ffff700 1 -- 172.21.15.130:0/1222877570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563299835de0 con 0x563299ab8c00 2023-09-12T16:08:33.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.638+0000 7f254ffff700 1 --2- 172.21.15.130:0/1222877570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab8c00 0x563299ab9020 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f254400aac0 tx=0x7f254400add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58865110525041ac server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:33.641 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.638+0000 7f254f7fe700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f25380a0840 0x7f25380a2d00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2540005950 tx=0x7f254000ab90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:33.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.645+0000 7f25367fc700 1 -- 172.21.15.130:0/1222877570 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400cc10 con 0x563299ab8c00 2023-09-12T16:08:33.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.645+0000 7f25367fc700 1 -- 172.21.15.130:0/1222877570 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254400cd70 con 0x563299ab8c00 2023-09-12T16:08:33.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.645+0000 7f25367fc700 1 -- 172.21.15.130:0/1222877570 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400f6b0 con 0x563299ab8c00 2023-09-12T16:08:33.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.646+0000 7f256242eb80 1 -- 172.21.15.130:0/1222877570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab8c00 msgr2=0x563299ab9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:33.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.646+0000 7f256242eb80 1 --2- 172.21.15.130:0/1222877570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab8c00 0x563299ab9020 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f254400aac0 tx=0x7f254400add0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.646+0000 7f256242eb80 1 -- 172.21.15.130:0/1222877570 shutdown_connections 2023-09-12T16:08:33.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.646+0000 7f256242eb80 1 --2- 172.21.15.130:0/1222877570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 0x563299ab82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.646+0000 7f256242eb80 1 --2- 172.21.15.130:0/1222877570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab8c00 0x563299ab9020 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f254400aac0 tx=0x7f254400add0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.646+0000 7f256242eb80 1 --2- 172.21.15.130:0/1222877570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ab95a0 0x563299ab7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.646+0000 7f256242eb80 1 -- 172.21.15.130:0/1222877570 >> 172.21.15.130:0/1222877570 conn(0x563299a2c090 msgr2=0x563299ab2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:33.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.648+0000 7f256242eb80 1 -- 172.21.15.130:0/1222877570 shutdown_connections 2023-09-12T16:08:33.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.648+0000 7f256242eb80 1 -- 172.21.15.130:0/1222877570 wait complete. 2023-09-12T16:08:33.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.649+0000 7f256242eb80 1 -- start start 2023-09-12T16:08:33.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.649+0000 7f256242eb80 1 Processor -- start 2023-09-12T16:08:33.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.649+0000 7f256242eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab7e00 0x563299ac1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.649+0000 7f256242eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab95a0 0x563299ac1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.649+0000 7f256242eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ac6de0 0x563299ac91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.649+0000 7f256242eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632999a7b70 con 0x563299ac6de0 2023-09-12T16:08:33.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.649+0000 7f256242eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632999a7870 con 0x563299ab7e00 2023-09-12T16:08:33.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.649+0000 7f256242eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56329988b920 con 0x563299ab95a0 2023-09-12T16:08:33.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.649+0000 7f256242eb80 10 client.0 _ll_register_callbacks cb 0x5632998f1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:08:33.653 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.650+0000 7f254ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab7e00 0x563299ac1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.650+0000 7f254ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab7e00 0x563299ac1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39244/0 (socket says 172.21.15.130:39244) 2023-09-12T16:08:33.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.650+0000 7f254ffff700 1 -- 172.21.15.130:0/3809031813 learned_addr learned my addr 172.21.15.130:0/3809031813 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:33.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.650+0000 7f2554805700 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ac6de0 0x563299ac91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.650+0000 7f254ffff700 1 -- 172.21.15.130:0/3809031813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab95a0 msgr2=0x563299ac1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:33.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.650+0000 7f254ffff700 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab95a0 0x563299ac1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.650+0000 7f254ffff700 1 -- 172.21.15.130:0/3809031813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ac6de0 msgr2=0x563299ac91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:33.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.650+0000 7f254ffff700 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ac6de0 0x563299ac91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.650+0000 7f254ffff700 1 -- 172.21.15.130:0/3809031813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254400a6f0 con 0x563299ab7e00 2023-09-12T16:08:33.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.651+0000 7f254ffff700 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab7e00 0x563299ac1870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f254400c5f0 tx=0x7f25440186b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecbe1c39b769dd7 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:33.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.651+0000 7f252f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:08:33.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.651+0000 7f252f7fe700 1 -- 172.21.15.130:0/3809031813 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254401c040 con 0x563299ab7e00 2023-09-12T16:08:33.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.651+0000 7f252f7fe700 1 -- 172.21.15.130:0/3809031813 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2544013070 con 0x563299ab7e00 2023-09-12T16:08:33.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.651+0000 7f256242eb80 1 -- 172.21.15.130:0/3809031813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632999614f0 con 0x563299ab7e00 2023-09-12T16:08:33.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.651+0000 7f252f7fe700 1 -- 172.21.15.130:0/3809031813 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2544004020 con 0x563299ab7e00 2023-09-12T16:08:33.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.651+0000 7f256242eb80 10 client.25433 fetch_fsmap learned FSMap version 32 2023-09-12T16:08:33.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.651+0000 7f256242eb80 1 -- 172.21.15.130:0/3809031813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56329981ec60 con 0x563299ab7e00 2023-09-12T16:08:33.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.652+0000 7f252f7fe700 1 -- 172.21.15.130:0/3809031813 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632999614f0 con 0x563299ab7e00 2023-09-12T16:08:33.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.652+0000 7f252f7fe700 1 -- 172.21.15.130:0/3809031813 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2544028050 con 0x563299ab7e00 2023-09-12T16:08:33.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.652+0000 7f256242eb80 10 client.25433 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:08:33.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.652+0000 7f256242eb80 10 client.25433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:08:33.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.652+0000 7f256242eb80 20 client.25433 populate_metadata read hostname 'smithi130' 2023-09-12T16:08:33.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.652+0000 7f256242eb80 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563299acd8d0 0x563299acfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.652+0000 7f256242eb80 4 client.25433 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:08:33.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.652+0000 7f256242eb80 1 -- 172.21.15.130:0/3809031813 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563299a1e8a0 con 0x563299acd8d0 2023-09-12T16:08:33.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.656+0000 7f254f7fe700 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563299acd8d0 0x563299acfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.664+0000 7f252f7fe700 10 client.25433 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:08:33.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.664+0000 7f254f7fe700 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563299acd8d0 0x563299acfd10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f4b7fa14e46e23 server_cookie=9512c438b1d847ef in_seq=0 out_seq=0 2023-09-12T16:08:33.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.664+0000 7f252f7fe700 1 -- 172.21.15.130:0/3809031813 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563299a1e8a0 con 0x563299acd8d0 2023-09-12T16:08:33.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.664+0000 7f252f7fe700 10 client.25433 handle_command_reply: tid=0 2023-09-12T16:08:33.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.664+0000 7f256242eb80 1 client.25433 shutdown 2023-09-12T16:08:33.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.672+0000 7f256242eb80 1 -- 172.21.15.130:0/3809031813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab7e00 msgr2=0x563299ac1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:33.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.672+0000 7f256242eb80 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab7e00 0x563299ac1870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f254400c5f0 tx=0x7f25440186b0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.685+0000 7f256242eb80 1 -- 172.21.15.130:0/3809031813 shutdown_connections 2023-09-12T16:08:33.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.685+0000 7f256242eb80 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563299acd8d0 0x563299acfd10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.685+0000 7f256242eb80 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ac6de0 0x563299ac91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.685+0000 7f256242eb80 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab7e00 0x563299ac1870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.685+0000 7f256242eb80 1 --2- 172.21.15.130:0/3809031813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab95a0 0x563299ac1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.685+0000 7f256242eb80 1 -- 172.21.15.130:0/3809031813 >> 172.21.15.130:0/3809031813 conn(0x563299a2c090 msgr2=0x563299ab1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:33.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.687+0000 7f252f7fe700 0 client.25433 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:08:33.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.688+0000 7f256242eb80 1 -- 172.21.15.130:0/3809031813 shutdown_connections 2023-09-12T16:08:33.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.690+0000 7f256242eb80 1 -- 172.21.15.130:0/3809031813 wait complete. 2023-09-12T16:08:33.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.690+0000 7f256242eb80 20 client.25433 trim_cache size 0 max 16384 2023-09-12T16:08:33.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.723+0000 7f256242eb80 1 Processor -- start 2023-09-12T16:08:33.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.723+0000 7f256242eb80 1 -- start start 2023-09-12T16:08:33.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.723+0000 7f256242eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ab7e00 0x563299ab25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.727 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.723+0000 7f256242eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299add530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.727 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.723+0000 7f256242eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ac6de0 0x563299ac9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.727 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.723+0000 7f256242eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x563299ac1440 con 0x563299ab7e00 2023-09-12T16:08:33.727 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.723+0000 7f256242eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563299ac0d20 con 0x563299ab95a0 2023-09-12T16:08:33.727 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.723+0000 7f256242eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563299ab4230 con 0x563299ac6de0 2023-09-12T16:08:33.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f254f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299add530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f254f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299add530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39250/0 (socket says 172.21.15.130:39250) 2023-09-12T16:08:33.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f254f7fe700 1 -- 172.21.15.130:0/575856280 learned_addr learned my addr 172.21.15.130:0/575856280 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:33.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f2554805700 1 --2- 172.21.15.130:0/575856280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ac6de0 0x563299ac9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.728 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f2554805700 1 -- 172.21.15.130:0/575856280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 msgr2=0x563299add530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:33.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f2554805700 1 --2- 172.21.15.130:0/575856280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299add530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f2554805700 1 -- 172.21.15.130:0/575856280 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ab7e00 msgr2=0x563299ab25b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:08:33.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f2554805700 1 --2- 172.21.15.130:0/575856280 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ab7e00 0x563299ab25b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f2554805700 1 -- 172.21.15.130:0/575856280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56329985c250 con 0x563299ac6de0 2023-09-12T16:08:33.729 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.724+0000 7f2554805700 1 --2- 172.21.15.130:0/575856280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ac6de0 0x563299ac9a90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25480bc990 tx=0x7f25480dc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b687a0042abd35 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:33.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.732+0000 7f2536ffd700 1 -- 172.21.15.130:0/575856280 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254803cea0 con 0x563299ac6de0 2023-09-12T16:08:33.734 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.732+0000 7f2536ffd700 1 -- 172.21.15.130:0/575856280 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25480bd390 con 0x563299ac6de0 2023-09-12T16:08:33.734 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.732+0000 7f2536ffd700 1 -- 172.21.15.130:0/575856280 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548005570 con 0x563299ac6de0 2023-09-12T16:08:33.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.736+0000 7f256242eb80 1 -- 172.21.15.130:0/575856280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ac6de0 msgr2=0x563299ac9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:33.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.736+0000 7f256242eb80 1 --2- 172.21.15.130:0/575856280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ac6de0 0x563299ac9a90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25480bc990 tx=0x7f25480dc040 comp rx=0 tx=0).stop 2023-09-12T16:08:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.743+0000 7f256242eb80 1 -- 172.21.15.130:0/575856280 shutdown_connections 2023-09-12T16:08:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.743+0000 7f256242eb80 1 --2- 172.21.15.130:0/575856280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ac6de0 0x563299ac9a90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.743+0000 7f256242eb80 1 --2- 172.21.15.130:0/575856280 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ab7e00 0x563299ab25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.743+0000 7f256242eb80 1 --2- 172.21.15.130:0/575856280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299add530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.743+0000 7f256242eb80 1 -- 172.21.15.130:0/575856280 >> 172.21.15.130:0/575856280 conn(0x563299a2c090 msgr2=0x563299abf650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:33.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.757+0000 7f256242eb80 1 -- 172.21.15.130:0/575856280 shutdown_connections 2023-09-12T16:08:33.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.757+0000 7f256242eb80 1 -- 172.21.15.130:0/575856280 wait complete. 2023-09-12T16:08:33.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.757+0000 7f256242eb80 1 -- start start 2023-09-12T16:08:33.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.757+0000 7f256242eb80 1 Processor -- start 2023-09-12T16:08:33.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f256242eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 0x563299adae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f256242eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299adb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f256242eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ac6de0 0x563299adc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f256242eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x563299adc8c0 con 0x563299ac6de0 2023-09-12T16:08:33.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f256242eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563299adca30 con 0x563299ab95a0 2023-09-12T16:08:33.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f256242eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563299adcbf0 con 0x563299ab7e00 2023-09-12T16:08:33.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f256242eb80 10 client.0 _ll_register_callbacks cb 0x7f254000a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:08:33.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f254ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 0x563299adae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f254ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 0x563299adae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35608/0 (socket says 172.21.15.130:35608) 2023-09-12T16:08:33.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f254ffff700 1 -- 172.21.15.130:0/1811932641 learned_addr learned my addr 172.21.15.130:0/1811932641 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:08:33.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f254f7fe700 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299adb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f254f7fe700 1 -- 172.21.15.130:0/1811932641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 msgr2=0x563299adae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:33.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f254f7fe700 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 0x563299adae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f254f7fe700 1 -- 172.21.15.130:0/1811932641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ac6de0 msgr2=0x563299adc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:08:33.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f254f7fe700 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ac6de0 0x563299adc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.758+0000 7f254f7fe700 1 -- 172.21.15.130:0/1811932641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501a4cd0 con 0x563299ab95a0 2023-09-12T16:08:33.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.761+0000 7f254f7fe700 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299adb3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f25400059d0 tx=0x7f254000a9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b696a625deb92ee server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:08:33.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.761+0000 7f252f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:08:33.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.767+0000 7f252f7fe700 1 -- 172.21.15.130:0/1811932641 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540024040 con 0x563299ab95a0 2023-09-12T16:08:33.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.767+0000 7f252f7fe700 1 -- 172.21.15.130:0/1811932641 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2540011660 con 0x563299ab95a0 2023-09-12T16:08:33.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.767+0000 7f256242eb80 1 -- 172.21.15.130:0/1811932641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f254400cd70 con 0x563299ab95a0 2023-09-12T16:08:33.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.776+0000 7f252f7fe700 1 -- 172.21.15.130:0/1811932641 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25400283c0 con 0x563299ab95a0 2023-09-12T16:08:33.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.776+0000 7f256242eb80 10 client.25439 fetch_fsmap learned FSMap version 32 2023-09-12T16:08:33.779 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.777+0000 7f256242eb80 1 -- 172.21.15.130:0/1811932641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5632998342b0 con 0x563299ab95a0 2023-09-12T16:08:33.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.779+0000 7f252f7fe700 1 -- 172.21.15.130:0/1811932641 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254400cd70 con 0x563299ab95a0 2023-09-12T16:08:33.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.788+0000 7f252f7fe700 1 -- 172.21.15.130:0/1811932641 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2540028a70 con 0x563299ab95a0 2023-09-12T16:08:33.793 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:08:33.794 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:08:33.794 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:08:33.794 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:08:33.795 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:08:33.795 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:08:33.796 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:08:33.796 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:08:33.796 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:08:33.797 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:08:33.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.789+0000 7f256242eb80 10 client.25439 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:08:33.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.789+0000 7f256242eb80 10 client.25439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:08:33.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.789+0000 7f256242eb80 20 client.25439 populate_metadata read hostname 'smithi130' 2023-09-12T16:08:33.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.789+0000 7f256242eb80 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563299b22810 0x563299b24c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:08:33.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.789+0000 7f256242eb80 4 client.25439 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:08:33.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.789+0000 7f256242eb80 1 -- 172.21.15.130:0/1811932641 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563299a02920 con 0x563299b22810 2023-09-12T16:08:33.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.790+0000 7f254ffff700 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563299b22810 0x563299b24c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:08:33.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.791+0000 7f254ffff700 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563299b22810 0x563299b24c30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5ead587c141b76 server_cookie=4ecf455e586ac041 in_seq=0 out_seq=0 2023-09-12T16:08:33.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.791+0000 7f252f7fe700 10 client.25439 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:08:33.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.791+0000 7f252f7fe700 1 -- 172.21.15.130:0/1811932641 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563299a02920 con 0x563299b22810 2023-09-12T16:08:33.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.791+0000 7f252f7fe700 10 client.25439 handle_command_reply: tid=0 2023-09-12T16:08:33.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.791+0000 7f256242eb80 1 client.25439 shutdown 2023-09-12T16:08:33.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.791+0000 7f256242eb80 1 -- 172.21.15.130:0/1811932641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 msgr2=0x563299adb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:33.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.791+0000 7f256242eb80 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299adb3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f25400059d0 tx=0x7f254000a9f0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f256242eb80 1 -- 172.21.15.130:0/1811932641 shutdown_connections 2023-09-12T16:08:33.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f256242eb80 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563299b22810 0x563299b24c30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f256242eb80 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563299ac6de0 0x563299adc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f252f7fe700 0 client.25439 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:08:33.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f256242eb80 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563299ab7e00 0x563299adae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f256242eb80 1 --2- 172.21.15.130:0/1811932641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563299ab95a0 0x563299adb3a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f256242eb80 1 -- 172.21.15.130:0/1811932641 >> 172.21.15.130:0/1811932641 conn(0x563299a2c090 msgr2=0x563299abf650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:33.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f256242eb80 1 -- 172.21.15.130:0/1811932641 shutdown_connections 2023-09-12T16:08:33.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f256242eb80 1 -- 172.21.15.130:0/1811932641 wait complete. 2023-09-12T16:08:33.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.792+0000 7f256242eb80 20 client.25439 trim_cache size 0 max 16384 2023-09-12T16:08:33.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 -- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f25380a0840 msgr2=0x7f25380a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:33.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f25380a0840 0x7f25380a2d00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2540005950 tx=0x7f254000ab90 comp rx=0 tx=0).stop 2023-09-12T16:08:33.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 -- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 msgr2=0x7f25501a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:08:33.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 0x7f25501a3d80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f254800b770 tx=0x7f2548003c30 comp rx=0 tx=0).stop 2023-09-12T16:08:33.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 -- 172.21.15.130:0/3170468888 shutdown_connections 2023-09-12T16:08:33.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f25380a0840 0x7f25380a2d00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f255010c940 0x7f25501a3d80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2550107600 0x7f255019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 --2- 172.21.15.130:0/3170468888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2550109f70 0x7f255019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:08:33.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.793+0000 7f2556268700 1 -- 172.21.15.130:0/3170468888 >> 172.21.15.130:0/3170468888 conn(0x7f255006d3b0 msgr2=0x7f255006f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:08:33.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.803+0000 7f2556268700 1 -- 172.21.15.130:0/3170468888 shutdown_connections 2023-09-12T16:08:33.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:08:33.804+0000 7f2556268700 1 -- 172.21.15.130:0/3170468888 wait complete. 2023-09-12T16:08:33.833 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:08:34.094 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:34 smithi130 ceph-mon[98670]: pgmap v461: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 25 MiB/s wr, 1.05k op/s 2023-09-12T16:08:34.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:34 smithi159 ceph-mon[102522]: pgmap v461: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 25 MiB/s wr, 1.05k op/s 2023-09-12T16:08:34.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:34 smithi203 ceph-mon[103431]: pgmap v461: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 25 MiB/s wr, 1.05k op/s 2023-09-12T16:08:35.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 165.0 s, 714.8 tps, lat 71.010 ms stddev 135.754 2023-09-12T16:08:36.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:36 smithi159 ceph-mon[102522]: pgmap v462: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 24 MiB/s wr, 1.57k op/s 2023-09-12T16:08:36.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:36 smithi130 ceph-mon[98670]: pgmap v462: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 24 MiB/s wr, 1.57k op/s 2023-09-12T16:08:36.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:36 smithi203 ceph-mon[103431]: pgmap v462: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 24 MiB/s wr, 1.57k op/s 2023-09-12T16:08:38.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:37 smithi159 ceph-mon[102522]: pgmap v463: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 32 MiB/s wr, 2.48k op/s 2023-09-12T16:08:38.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:37 smithi130 ceph-mon[98670]: pgmap v463: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 32 MiB/s wr, 2.48k op/s 2023-09-12T16:08:38.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:37 smithi203 ceph-mon[103431]: pgmap v463: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 32 MiB/s wr, 2.48k op/s 2023-09-12T16:08:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 170.0 s, 734.9 tps, lat 203.818 ms stddev 515.367 2023-09-12T16:08:40.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:40 smithi130 ceph-mon[98670]: pgmap v464: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 42 MiB/s wr, 3.73k op/s 2023-09-12T16:08:40.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:40 smithi159 ceph-mon[102522]: pgmap v464: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 42 MiB/s wr, 3.73k op/s 2023-09-12T16:08:40.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:40 smithi203 ceph-mon[103431]: pgmap v464: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 42 MiB/s wr, 3.73k op/s 2023-09-12T16:08:42.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:42 smithi159 ceph-mon[102522]: pgmap v465: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 36 MiB/s wr, 3.51k op/s 2023-09-12T16:08:42.485 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:42 smithi130 ceph-mon[98670]: pgmap v465: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 36 MiB/s wr, 3.51k op/s 2023-09-12T16:08:42.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:42 smithi203 ceph-mon[103431]: pgmap v465: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 36 MiB/s wr, 3.51k op/s 2023-09-12T16:08:44.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:44 smithi130 ceph-mon[98670]: pgmap v466: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 41 MiB/s wr, 3.94k op/s 2023-09-12T16:08:44.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:44 smithi159 ceph-mon[102522]: pgmap v466: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 41 MiB/s wr, 3.94k op/s 2023-09-12T16:08:44.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:44 smithi203 ceph-mon[103431]: pgmap v466: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 41 MiB/s wr, 3.94k op/s 2023-09-12T16:08:45.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 175.0 s, 1716.0 tps, lat 58.381 ms stddev 47.297 2023-09-12T16:08:46.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:46 smithi130 ceph-mon[98670]: pgmap v467: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 41 MiB/s wr, 4.07k op/s 2023-09-12T16:08:46.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:46 smithi159 ceph-mon[102522]: pgmap v467: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 41 MiB/s wr, 4.07k op/s 2023-09-12T16:08:46.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:46 smithi203 ceph-mon[103431]: pgmap v467: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 41 MiB/s wr, 4.07k op/s 2023-09-12T16:08:48.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:47 smithi159 ceph-mon[102522]: pgmap v468: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 39 MiB/s wr, 3.50k op/s 2023-09-12T16:08:48.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:47 smithi130 ceph-mon[98670]: pgmap v468: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 39 MiB/s wr, 3.50k op/s 2023-09-12T16:08:48.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:47 smithi203 ceph-mon[103431]: pgmap v468: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 39 MiB/s wr, 3.50k op/s 2023-09-12T16:08:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 180.0 s, 1257.6 tps, lat 77.446 ms stddev 78.666 2023-09-12T16:08:50.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:50 smithi159 ceph-mon[102522]: pgmap v469: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 35 MiB/s wr, 2.75k op/s 2023-09-12T16:08:50.479 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:50 smithi130 ceph-mon[98670]: pgmap v469: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 35 MiB/s wr, 2.75k op/s 2023-09-12T16:08:50.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:50 smithi203 ceph-mon[103431]: pgmap v469: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 35 MiB/s wr, 2.75k op/s 2023-09-12T16:08:52.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:52 smithi159 ceph-mon[102522]: pgmap v470: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 22 MiB/s wr, 1.35k op/s 2023-09-12T16:08:52.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:52 smithi130 ceph-mon[98670]: pgmap v470: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 22 MiB/s wr, 1.35k op/s 2023-09-12T16:08:52.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:52 smithi203 ceph-mon[103431]: pgmap v470: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 22 MiB/s wr, 1.35k op/s 2023-09-12T16:08:54.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:54 smithi203 ceph-mon[103431]: pgmap v471: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 28 MiB/s wr, 1.93k op/s 2023-09-12T16:08:54.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:54 smithi159 ceph-mon[102522]: pgmap v471: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 28 MiB/s wr, 1.93k op/s 2023-09-12T16:08:54.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:54 smithi130 ceph-mon[98670]: pgmap v471: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 28 MiB/s wr, 1.93k op/s 2023-09-12T16:08:55.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 185.0 s, 446.2 tps, lat 227.792 ms stddev 528.220 2023-09-12T16:08:56.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:56 smithi203 ceph-mon[103431]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 33 MiB/s wr, 2.55k op/s 2023-09-12T16:08:56.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:56 smithi130 ceph-mon[98670]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 33 MiB/s wr, 2.55k op/s 2023-09-12T16:08:56.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:56 smithi159 ceph-mon[102522]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 33 MiB/s wr, 2.55k op/s 2023-09-12T16:08:58.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:08:57 smithi130 ceph-mon[98670]: pgmap v473: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 28 MiB/s wr, 2.40k op/s 2023-09-12T16:08:58.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:08:57 smithi159 ceph-mon[102522]: pgmap v473: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 28 MiB/s wr, 2.40k op/s 2023-09-12T16:08:58.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:08:57 smithi203 ceph-mon[103431]: pgmap v473: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 28 MiB/s wr, 2.40k op/s 2023-09-12T16:09:00.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 190.0 s, 1449.4 tps, lat 69.211 ms stddev 59.644 2023-09-12T16:09:00.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:00 smithi130 ceph-mon[98670]: pgmap v474: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 29 MiB/s wr, 2.53k op/s 2023-09-12T16:09:00.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:00 smithi159 ceph-mon[102522]: pgmap v474: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 29 MiB/s wr, 2.53k op/s 2023-09-12T16:09:00.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:00 smithi203 ceph-mon[103431]: pgmap v474: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 29 MiB/s wr, 2.53k op/s 2023-09-12T16:09:02.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:02 smithi159 ceph-mon[102522]: pgmap v475: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 24 MiB/s wr, 2.26k op/s 2023-09-12T16:09:02.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:02 smithi130 ceph-mon[98670]: pgmap v475: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 24 MiB/s wr, 2.26k op/s 2023-09-12T16:09:02.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:02 smithi203 ceph-mon[103431]: pgmap v475: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 24 MiB/s wr, 2.26k op/s 2023-09-12T16:09:03.834 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:09:04.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.258+0000 7fd12ae4a700 1 -- 172.21.15.130:0/1473402322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd124100ad0 msgr2=0x7fd124100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:04.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.258+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/1473402322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd124100ad0 0x7fd124100ef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00b0a0 tx=0x7fd11c00b3b0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.259+0000 7fd12ae4a700 1 -- 172.21.15.130:0/1473402322 shutdown_connections 2023-09-12T16:09:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.259+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/1473402322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1241000e0 0x7fd1240fe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.259+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/1473402322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd124100ad0 0x7fd124100ef0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.259+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/1473402322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd1240ff720 0x7fd1240ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.259+0000 7fd12ae4a700 1 -- 172.21.15.130:0/1473402322 >> 172.21.15.130:0/1473402322 conn(0x7fd1240fa1f0 msgr2=0x7fd1240fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:04.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.261+0000 7fd12ae4a700 1 -- 172.21.15.130:0/1473402322 shutdown_connections 2023-09-12T16:09:04.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.263+0000 7fd12ae4a700 1 -- 172.21.15.130:0/1473402322 wait complete. 2023-09-12T16:09:04.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.264+0000 7fd12ae4a700 1 Processor -- start 2023-09-12T16:09:04.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.265+0000 7fd12ae4a700 1 -- start start 2023-09-12T16:09:04.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.265+0000 7fd12ae4a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1240ff720 0x7fd124194260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.265+0000 7fd12ae4a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1241000e0 0x7fd1241947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.265+0000 7fd12ae4a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd124100ad0 0x7fd12419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.265+0000 7fd12ae4a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12419be50 con 0x7fd124100ad0 2023-09-12T16:09:04.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.265+0000 7fd12ae4a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12419bfc0 con 0x7fd1240ff720 2023-09-12T16:09:04.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.265+0000 7fd12ae4a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12419c130 con 0x7fd1241000e0 2023-09-12T16:09:04.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.266+0000 7fd123fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1241000e0 0x7fd1241947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.271 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.266+0000 7fd123fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1241000e0 0x7fd1241947a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35624/0 (socket says 172.21.15.130:35624) 2023-09-12T16:09:04.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.266+0000 7fd123fff700 1 -- 172.21.15.130:0/4088036346 learned_addr learned my addr 172.21.15.130:0/4088036346 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:04.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.266+0000 7fd128be6700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1240ff720 0x7fd124194260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.267+0000 7fd123fff700 1 -- 172.21.15.130:0/4088036346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1240ff720 msgr2=0x7fd124194260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:04.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.267+0000 7fd123fff700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1240ff720 0x7fd124194260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.267+0000 7fd123fff700 1 -- 172.21.15.130:0/4088036346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd124100ad0 msgr2=0x7fd12419b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:04.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.267+0000 7fd123fff700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd124100ad0 0x7fd12419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.267+0000 7fd123fff700 1 -- 172.21.15.130:0/4088036346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c009d00 con 0x7fd1241000e0 2023-09-12T16:09:04.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.267+0000 7fd123fff700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1241000e0 0x7fd1241947a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd11800b530 tx=0x7fd11800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:04.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.269+0000 7fd121ffb700 1 -- 172.21.15.130:0/4088036346 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1180118f0 con 0x7fd1241000e0 2023-09-12T16:09:04.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.269+0000 7fd121ffb700 1 -- 172.21.15.130:0/4088036346 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd118010380 con 0x7fd1241000e0 2023-09-12T16:09:04.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.269+0000 7fd121ffb700 1 -- 172.21.15.130:0/4088036346 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800f540 con 0x7fd1241000e0 2023-09-12T16:09:04.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.273+0000 7fd12ae4a700 1 -- 172.21.15.130:0/4088036346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12419c410 con 0x7fd1241000e0 2023-09-12T16:09:04.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.273+0000 7fd12ae4a700 1 -- 172.21.15.130:0/4088036346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12419c920 con 0x7fd1241000e0 2023-09-12T16:09:04.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.278+0000 7fd137010b80 1 Processor -- start 2023-09-12T16:09:04.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.279+0000 7fd121ffb700 1 -- 172.21.15.130:0/4088036346 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fd1180104f0 con 0x7fd1241000e0 2023-09-12T16:09:04.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd137010b80 1 -- start start 2023-09-12T16:09:04.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd137010b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 0x563e507cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd137010b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 0x563e507ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd137010b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cae00 0x563e507cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd137010b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e50723730 con 0x563e507cbc00 2023-09-12T16:09:04.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd137010b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e50721050 con 0x563e507cae00 2023-09-12T16:09:04.282 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd137010b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5071c940 con 0x563e507cc5a0 2023-09-12T16:09:04.284 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd123fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cae00 0x563e507cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.284 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd1293e7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 0x563e507cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.284 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd123fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cae00 0x563e507cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39274/0 (socket says 172.21.15.130:39274) 2023-09-12T16:09:04.284 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd123fff700 1 -- 172.21.15.130:0/2699530274 learned_addr learned my addr 172.21.15.130:0/2699530274 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:04.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.280+0000 7fd121ffb700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd10c0a0790 0x7fd10c0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd1293e7700 1 -- 172.21.15.130:0/2699530274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 msgr2=0x563e507ca800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:09:04.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd1293e7700 1 --2- 172.21.15.130:0/2699530274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 0x563e507ca800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd1293e7700 1 -- 172.21.15.130:0/2699530274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cae00 msgr2=0x563e507cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:04.285 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd1293e7700 1 --2- 172.21.15.130:0/2699530274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cae00 0x563e507cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd121ffb700 1 -- 172.21.15.130:0/4088036346 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd1180cb6f0 con 0x7fd1241000e0 2023-09-12T16:09:04.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd1293e7700 1 -- 172.21.15.130:0/2699530274 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e50548de0 con 0x563e507cbc00 2023-09-12T16:09:04.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd1293e7700 1 --2- 172.21.15.130:0/2699530274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 0x563e507cc020 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd1140098f0 tx=0x7fd114009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8eb425136f358df server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:04.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd1293e7700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd10c0a0790 0x7fd10c0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd10affd700 1 -- 172.21.15.130:0/2699530274 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1140137c0 con 0x563e507cbc00 2023-09-12T16:09:04.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd10affd700 1 -- 172.21.15.130:0/2699530274 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd114013ca0 con 0x563e507cbc00 2023-09-12T16:09:04.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.281+0000 7fd10affd700 1 -- 172.21.15.130:0/2699530274 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11400b750 con 0x563e507cbc00 2023-09-12T16:09:04.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.282+0000 7fd1293e7700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd10c0a0790 0x7fd10c0a2c50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd114018000 tx=0x7fd11400bd00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:04.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.282+0000 7fd137010b80 1 -- 172.21.15.130:0/2699530274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 msgr2=0x563e507cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:04.287 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.282+0000 7fd137010b80 1 --2- 172.21.15.130:0/2699530274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 0x563e507cc020 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd1140098f0 tx=0x7fd114009c00 comp rx=0 tx=0).stop 2023-09-12T16:09:04.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.283+0000 7fd137010b80 1 -- 172.21.15.130:0/2699530274 shutdown_connections 2023-09-12T16:09:04.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.283+0000 7fd137010b80 1 --2- 172.21.15.130:0/2699530274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cae00 0x563e507cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.283+0000 7fd137010b80 1 --2- 172.21.15.130:0/2699530274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 0x563e507cc020 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.283+0000 7fd137010b80 1 --2- 172.21.15.130:0/2699530274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 0x563e507ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.283+0000 7fd137010b80 1 -- 172.21.15.130:0/2699530274 >> 172.21.15.130:0/2699530274 conn(0x563e5073f090 msgr2=0x563e507c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:04.288 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.283+0000 7fd137010b80 1 -- 172.21.15.130:0/2699530274 shutdown_connections 2023-09-12T16:09:04.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.283+0000 7fd137010b80 1 -- 172.21.15.130:0/2699530274 wait complete. 2023-09-12T16:09:04.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.283+0000 7fd137010b80 1 -- start start 2023-09-12T16:09:04.289 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.283+0000 7fd137010b80 1 Processor -- start 2023-09-12T16:09:04.291 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.285+0000 7fd137010b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cae00 0x563e507d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.292 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.285+0000 7fd137010b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 0x563e507d4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.292 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.285+0000 7fd137010b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 0x563e507dbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.292 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.285+0000 7fd137010b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e506bab70 con 0x563e507cbc00 2023-09-12T16:09:04.292 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.285+0000 7fd137010b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e506ba870 con 0x563e507cc5a0 2023-09-12T16:09:04.292 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.285+0000 7fd137010b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5059e920 con 0x563e507cae00 2023-09-12T16:09:04.293 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.285+0000 7fd137010b80 10 client.0 _ll_register_callbacks cb 0x563e50604a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:09:04.293 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.287+0000 7fd128be6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cae00 0x563e507d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.293 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.287+0000 7fd128be6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cae00 0x563e507d4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35634/0 (socket says 172.21.15.130:35634) 2023-09-12T16:09:04.293 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.287+0000 7fd128be6700 1 -- 172.21.15.130:0/3041209988 learned_addr learned my addr 172.21.15.130:0/3041209988 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:04.293 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.287+0000 7fd1293e7700 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 0x563e507dbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.287+0000 7fd128be6700 1 -- 172.21.15.130:0/3041209988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 msgr2=0x563e507dbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:04.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.287+0000 7fd128be6700 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 0x563e507dbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.287+0000 7fd128be6700 1 -- 172.21.15.130:0/3041209988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 msgr2=0x563e507d4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:04.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.287+0000 7fd128be6700 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 0x563e507d4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.287+0000 7fd128be6700 1 -- 172.21.15.130:0/3041209988 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd114009400 con 0x563e507cae00 2023-09-12T16:09:04.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.289+0000 7fd128be6700 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cae00 0x563e507d4550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00b0a0 tx=0x7fd11c004da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5e9c6d7d68a246 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:04.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.289+0000 7fd103fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:09:04.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.289+0000 7fd103fff700 1 -- 172.21.15.130:0/3041209988 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00e040 con 0x563e507cae00 2023-09-12T16:09:04.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.289+0000 7fd103fff700 1 -- 172.21.15.130:0/3041209988 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd11c0078f0 con 0x563e507cae00 2023-09-12T16:09:04.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.289+0000 7fd137010b80 1 -- 172.21.15.130:0/3041209988 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e506744f0 con 0x563e507cae00 2023-09-12T16:09:04.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.289+0000 7fd103fff700 1 -- 172.21.15.130:0/3041209988 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd11c01c440 con 0x563e507cae00 2023-09-12T16:09:04.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.289+0000 7fd137010b80 10 client.25663 fetch_fsmap learned FSMap version 32 2023-09-12T16:09:04.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.289+0000 7fd137010b80 1 -- 172.21.15.130:0/3041209988 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563e50531c60 con 0x563e507cae00 2023-09-12T16:09:04.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.290+0000 7fd103fff700 1 -- 172.21.15.130:0/3041209988 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e506744f0 con 0x563e507cae00 2023-09-12T16:09:04.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.290+0000 7fd103fff700 1 -- 172.21.15.130:0/3041209988 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd11c019050 con 0x563e507cae00 2023-09-12T16:09:04.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.290+0000 7fd137010b80 10 client.25663 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:09:04.297 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.290+0000 7fd137010b80 10 client.25663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:09:04.297 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.290+0000 7fd137010b80 20 client.25663 populate_metadata read hostname 'smithi130' 2023-09-12T16:09:04.297 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.290+0000 7fd137010b80 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563e507e0420 0x563e507e2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.297 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.290+0000 7fd137010b80 4 client.25663 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:09:04.297 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.290+0000 7fd137010b80 1 -- 172.21.15.130:0/3041209988 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e507318a0 con 0x563e507e0420 2023-09-12T16:09:04.297 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.290+0000 7fd123fff700 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563e507e0420 0x563e507e2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.298 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.291+0000 7fd103fff700 10 client.25663 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:09:04.298 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.291+0000 7fd123fff700 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563e507e0420 0x563e507e2840 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c0cb394bb929d80 server_cookie=58f5b3b571fd34d0 in_seq=0 out_seq=0 2023-09-12T16:09:04.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd103fff700 1 -- 172.21.15.130:0/3041209988 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563e507318a0 con 0x563e507e0420 2023-09-12T16:09:04.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd103fff700 10 client.25663 handle_command_reply: tid=0 2023-09-12T16:09:04.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 client.25663 shutdown 2023-09-12T16:09:04.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 -- 172.21.15.130:0/3041209988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cae00 msgr2=0x563e507d4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:04.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cae00 0x563e507d4550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00b0a0 tx=0x7fd11c004da0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 -- 172.21.15.130:0/3041209988 shutdown_connections 2023-09-12T16:09:04.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563e507e0420 0x563e507e2840 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 0x563e507dbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cae00 0x563e507d4550 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 --2- 172.21.15.130:0/3041209988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cbc00 0x563e507d4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 -- 172.21.15.130:0/3041209988 >> 172.21.15.130:0/3041209988 conn(0x563e5073f090 msgr2=0x563e507c4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:04.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd103fff700 0 client.25663 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:09:04.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 -- 172.21.15.130:0/3041209988 shutdown_connections 2023-09-12T16:09:04.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 1 -- 172.21.15.130:0/3041209988 wait complete. 2023-09-12T16:09:04.302 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.292+0000 7fd137010b80 20 client.25663 trim_cache size 0 max 16384 2023-09-12T16:09:04.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.328+0000 7fd137010b80 1 Processor -- start 2023-09-12T16:09:04.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.329+0000 7fd137010b80 1 -- start start 2023-09-12T16:09:04.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.329+0000 7fd137010b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 0x563e507d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.329+0000 7fd137010b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cbc00 0x563e507d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.329+0000 7fd137010b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 0x563e507dba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.329+0000 7fd137010b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e507d3a30 con 0x563e507cae00 2023-09-12T16:09:04.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.329+0000 7fd137010b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e507d5450 con 0x563e507cbc00 2023-09-12T16:09:04.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.329+0000 7fd137010b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e507caca0 con 0x563e507cc5a0 2023-09-12T16:09:04.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.330+0000 7fd1293e7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 0x563e507dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.330+0000 7fd1293e7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 0x563e507dba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35642/0 (socket says 172.21.15.130:35642) 2023-09-12T16:09:04.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.330+0000 7fd1293e7700 1 -- 172.21.15.130:0/1098251704 learned_addr learned my addr 172.21.15.130:0/1098251704 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:04.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.331+0000 7fd123fff700 1 --2- 172.21.15.130:0/1098251704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cbc00 0x563e507d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.331+0000 7fd123fff700 1 -- 172.21.15.130:0/1098251704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 msgr2=0x563e507dba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:04.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.331+0000 7fd123fff700 1 --2- 172.21.15.130:0/1098251704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 0x563e507dba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.331+0000 7fd123fff700 1 -- 172.21.15.130:0/1098251704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 msgr2=0x563e507d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:04.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.331+0000 7fd123fff700 1 --2- 172.21.15.130:0/1098251704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 0x563e507d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.331+0000 7fd123fff700 1 -- 172.21.15.130:0/1098251704 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e5056f250 con 0x563e507cbc00 2023-09-12T16:09:04.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.332+0000 7fd123fff700 1 --2- 172.21.15.130:0/1098251704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cbc00 0x563e507d2830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd118095070 tx=0x7fd118004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=555aa3470dc1e5ac server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:04.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.341+0000 7fd10b7fe700 1 -- 172.21.15.130:0/1098251704 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1180984d0 con 0x563e507cbc00 2023-09-12T16:09:04.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.341+0000 7fd10b7fe700 1 -- 172.21.15.130:0/1098251704 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd118099270 con 0x563e507cbc00 2023-09-12T16:09:04.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.341+0000 7fd10b7fe700 1 -- 172.21.15.130:0/1098251704 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1180cb440 con 0x563e507cbc00 2023-09-12T16:09:04.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.342+0000 7fd137010b80 1 -- 172.21.15.130:0/1098251704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cbc00 msgr2=0x563e507d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:04.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.342+0000 7fd137010b80 1 --2- 172.21.15.130:0/1098251704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cbc00 0x563e507d2830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd118095070 tx=0x7fd118004750 comp rx=0 tx=0).stop 2023-09-12T16:09:04.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.352+0000 7fd137010b80 1 -- 172.21.15.130:0/1098251704 shutdown_connections 2023-09-12T16:09:04.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.352+0000 7fd137010b80 1 --2- 172.21.15.130:0/1098251704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cc5a0 0x563e507dba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.352+0000 7fd137010b80 1 --2- 172.21.15.130:0/1098251704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 0x563e507d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.352+0000 7fd137010b80 1 --2- 172.21.15.130:0/1098251704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cbc00 0x563e507d2830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.352+0000 7fd137010b80 1 -- 172.21.15.130:0/1098251704 >> 172.21.15.130:0/1098251704 conn(0x563e5073f090 msgr2=0x563e507cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:04.372 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.364+0000 7fd137010b80 1 -- 172.21.15.130:0/1098251704 shutdown_connections 2023-09-12T16:09:04.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.373+0000 7fd137010b80 1 -- 172.21.15.130:0/1098251704 wait complete. 2023-09-12T16:09:04.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.373+0000 7fd137010b80 1 -- start start 2023-09-12T16:09:04.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.373+0000 7fd137010b80 1 Processor -- start 2023-09-12T16:09:04.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.374+0000 7fd137010b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 0x563e507d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.374+0000 7fd137010b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cbc00 0x563e507d8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.374+0000 7fd137010b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 0x563e50834220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.374+0000 7fd137010b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e507dbc10 con 0x563e507cae00 2023-09-12T16:09:04.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.374+0000 7fd137010b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e507dbd80 con 0x563e507cc5a0 2023-09-12T16:09:04.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.374+0000 7fd137010b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e507dbf40 con 0x563e507cbc00 2023-09-12T16:09:04.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.374+0000 7fd137010b80 10 client.0 _ll_register_callbacks cb 0x7fd11c009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:09:04.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.377+0000 7fd123fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cbc00 0x563e507d8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.377+0000 7fd123fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cbc00 0x563e507d8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35652/0 (socket says 172.21.15.130:35652) 2023-09-12T16:09:04.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.377+0000 7fd123fff700 1 -- 172.21.15.130:0/2450179238 learned_addr learned my addr 172.21.15.130:0/2450179238 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:04.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.384+0000 7fd128be6700 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 0x563e507d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.385+0000 7fd1293e7700 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 0x563e50834220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.386+0000 7fd128be6700 1 -- 172.21.15.130:0/2450179238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cbc00 msgr2=0x563e507d8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:04.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.386+0000 7fd128be6700 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cbc00 0x563e507d8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.386+0000 7fd128be6700 1 -- 172.21.15.130:0/2450179238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 msgr2=0x563e50834220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:04.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.386+0000 7fd128be6700 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 0x563e50834220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.386+0000 7fd128be6700 1 -- 172.21.15.130:0/2450179238 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12419c920 con 0x563e507cae00 2023-09-12T16:09:04.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.386+0000 7fd128be6700 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 0x563e507d86b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd11c007b60 tx=0x7fd11c012620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8925e6680a12bc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:04.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.386+0000 7fd103fff700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:09:04.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.387+0000 7fd103fff700 1 -- 172.21.15.130:0/2450179238 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c012ea0 con 0x563e507cae00 2023-09-12T16:09:04.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.387+0000 7fd103fff700 1 -- 172.21.15.130:0/2450179238 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd11c0078f0 con 0x563e507cae00 2023-09-12T16:09:04.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.387+0000 7fd137010b80 1 -- 172.21.15.130:0/2450179238 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd114013ca0 con 0x563e507cae00 2023-09-12T16:09:04.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd103fff700 1 -- 172.21.15.130:0/2450179238 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd11c00e040 con 0x563e507cae00 2023-09-12T16:09:04.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd137010b80 10 client.15987 fetch_fsmap learned FSMap version 32 2023-09-12T16:09:04.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd137010b80 1 -- 172.21.15.130:0/2450179238 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563e505472b0 con 0x563e507cae00 2023-09-12T16:09:04.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd103fff700 1 -- 172.21.15.130:0/2450179238 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd114013ca0 con 0x563e507cae00 2023-09-12T16:09:04.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd103fff700 1 -- 172.21.15.130:0/2450179238 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd11c019070 con 0x563e507cae00 2023-09-12T16:09:04.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd137010b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:09:04.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd137010b80 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:09:04.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd137010b80 20 client.15987 populate_metadata read hostname 'smithi130' 2023-09-12T16:09:04.395 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd137010b80 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563e50835ac0 0x563e50837eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:04.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd137010b80 4 client.15987 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:09:04.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.392+0000 7fd137010b80 1 -- 172.21.15.130:0/2450179238 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563e50715920 con 0x563e50835ac0 2023-09-12T16:09:04.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.406+0000 7fd123fff700 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563e50835ac0 0x563e50837eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:04.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.459+0000 7fd123fff700 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563e50835ac0 0x563e50837eb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0ee77312c17ce5 server_cookie=c8690eb730e943f5 in_seq=0 out_seq=0 2023-09-12T16:09:04.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.460+0000 7fd103fff700 10 client.15987 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:09:04.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.465+0000 7fd103fff700 1 -- 172.21.15.130:0/2450179238 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563e50715920 con 0x563e50835ac0 2023-09-12T16:09:04.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.465+0000 7fd103fff700 10 client.15987 handle_command_reply: tid=0 2023-09-12T16:09:04.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.465+0000 7fd137010b80 1 client.15987 shutdown 2023-09-12T16:09:04.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.468+0000 7fd137010b80 1 -- 172.21.15.130:0/2450179238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 msgr2=0x563e507d86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:04.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.468+0000 7fd137010b80 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 0x563e507d86b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd11c007b60 tx=0x7fd11c012620 comp rx=0 tx=0).stop 2023-09-12T16:09:04.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.471+0000 7fd137010b80 1 -- 172.21.15.130:0/2450179238 shutdown_connections 2023-09-12T16:09:04.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.471+0000 7fd137010b80 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x563e50835ac0 0x563e50837eb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.471+0000 7fd137010b80 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e507cc5a0 0x563e50834220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.471+0000 7fd137010b80 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563e507cae00 0x563e507d86b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.471+0000 7fd137010b80 1 --2- 172.21.15.130:0/2450179238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e507cbc00 0x563e507d8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.471+0000 7fd137010b80 1 -- 172.21.15.130:0/2450179238 >> 172.21.15.130:0/2450179238 conn(0x563e5073f090 msgr2=0x563e507cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:04.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.471+0000 7fd103fff700 0 client.15987 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:09:04.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:04 smithi159 ceph-mon[102522]: pgmap v476: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 28 MiB/s wr, 2.43k op/s 2023-09-12T16:09:04.480 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:04 smithi130 ceph-mon[98670]: pgmap v476: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 28 MiB/s wr, 2.43k op/s 2023-09-12T16:09:04.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.479+0000 7fd137010b80 1 -- 172.21.15.130:0/2450179238 shutdown_connections 2023-09-12T16:09:04.494 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:09:04.494 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:09:04.494 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:09:04.495 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:09:04.495 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:09:04.495 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:09:04.496 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:09:04.496 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:09:04.496 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:09:04.496 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:09:04.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.489+0000 7fd137010b80 1 -- 172.21.15.130:0/2450179238 wait complete. 2023-09-12T16:09:04.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.493+0000 7fd137010b80 20 client.15987 trim_cache size 0 max 16384 2023-09-12T16:09:04.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:04 smithi203 ceph-mon[103431]: pgmap v476: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 28 MiB/s wr, 2.43k op/s 2023-09-12T16:09:04.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.500+0000 7fd12ae4a700 1 -- 172.21.15.130:0/4088036346 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd10c0a0790 msgr2=0x7fd10c0a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:04.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.500+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd10c0a0790 0x7fd10c0a2c50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd114018000 tx=0x7fd11400bd00 comp rx=0 tx=0).stop 2023-09-12T16:09:04.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.500+0000 7fd12ae4a700 1 -- 172.21.15.130:0/4088036346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1241000e0 msgr2=0x7fd1241947a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:04.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.500+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1241000e0 0x7fd1241947a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd11800b530 tx=0x7fd11800b840 comp rx=0 tx=0).stop 2023-09-12T16:09:04.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.501+0000 7fd12ae4a700 1 -- 172.21.15.130:0/4088036346 shutdown_connections 2023-09-12T16:09:04.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.501+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd10c0a0790 0x7fd10c0a2c50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.501+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd124100ad0 0x7fd12419b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.501+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1240ff720 0x7fd124194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:04.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.501+0000 7fd12ae4a700 1 --2- 172.21.15.130:0/4088036346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1241000e0 0x7fd1241947a0 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd11800b530 tx=0x7fd11800b840 comp rx=0 tx=0).stop 2023-09-12T16:09:04.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.501+0000 7fd12ae4a700 1 -- 172.21.15.130:0/4088036346 >> 172.21.15.130:0/4088036346 conn(0x7fd1240fa1f0 msgr2=0x7fd1240fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:04.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.507+0000 7fd12ae4a700 1 -- 172.21.15.130:0/4088036346 shutdown_connections 2023-09-12T16:09:04.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:04.508+0000 7fd12ae4a700 1 -- 172.21.15.130:0/4088036346 wait complete. 2023-09-12T16:09:04.538 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:09:05.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 195.0 s, 781.7 tps, lat 125.678 ms stddev 306.268 2023-09-12T16:09:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:06 smithi159 ceph-mon[102522]: pgmap v477: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 33 MiB/s wr, 2.75k op/s 2023-09-12T16:09:06.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:06 smithi130 ceph-mon[98670]: pgmap v477: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 33 MiB/s wr, 2.75k op/s 2023-09-12T16:09:06.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:06 smithi203 ceph-mon[103431]: pgmap v477: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 33 MiB/s wr, 2.75k op/s 2023-09-12T16:09:08.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:07 smithi130 ceph-mon[98670]: pgmap v478: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 29 MiB/s wr, 2.29k op/s 2023-09-12T16:09:08.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:07 smithi159 ceph-mon[102522]: pgmap v478: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 29 MiB/s wr, 2.29k op/s 2023-09-12T16:09:08.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:07 smithi203 ceph-mon[103431]: pgmap v478: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 29 MiB/s wr, 2.29k op/s 2023-09-12T16:09:10.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 200.0 s, 1074.8 tps, lat 95.237 ms stddev 167.255 2023-09-12T16:09:10.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:10 smithi130 ceph-mon[98670]: pgmap v479: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 35 MiB/s wr, 2.77k op/s 2023-09-12T16:09:10.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:10 smithi159 ceph-mon[102522]: pgmap v479: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 35 MiB/s wr, 2.77k op/s 2023-09-12T16:09:10.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:10 smithi203 ceph-mon[103431]: pgmap v479: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 35 MiB/s wr, 2.77k op/s 2023-09-12T16:09:12.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:12 smithi159 ceph-mon[102522]: pgmap v480: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 28 MiB/s wr, 2.44k op/s 2023-09-12T16:09:12.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:12 smithi130 ceph-mon[98670]: pgmap v480: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 28 MiB/s wr, 2.44k op/s 2023-09-12T16:09:12.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:12 smithi203 ceph-mon[103431]: pgmap v480: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 28 MiB/s wr, 2.44k op/s 2023-09-12T16:09:14.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:14 smithi159 ceph-mon[102522]: pgmap v481: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 32 MiB/s wr, 2.55k op/s 2023-09-12T16:09:14.479 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:14 smithi130 ceph-mon[98670]: pgmap v481: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 32 MiB/s wr, 2.55k op/s 2023-09-12T16:09:14.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:14 smithi203 ceph-mon[103431]: pgmap v481: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 32 MiB/s wr, 2.55k op/s 2023-09-12T16:09:15.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 205.0 s, 1774.5 tps, lat 56.253 ms stddev 43.814 2023-09-12T16:09:16.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:16 smithi130 ceph-mon[98670]: pgmap v482: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 35 MiB/s wr, 2.70k op/s 2023-09-12T16:09:16.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:16 smithi159 ceph-mon[102522]: pgmap v482: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 35 MiB/s wr, 2.70k op/s 2023-09-12T16:09:16.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:16 smithi203 ceph-mon[103431]: pgmap v482: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 35 MiB/s wr, 2.70k op/s 2023-09-12T16:09:18.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:17 smithi159 ceph-mon[102522]: pgmap v483: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 28 MiB/s wr, 1.90k op/s 2023-09-12T16:09:18.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:17 smithi130 ceph-mon[98670]: pgmap v483: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 28 MiB/s wr, 1.90k op/s 2023-09-12T16:09:18.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:17 smithi203 ceph-mon[103431]: pgmap v483: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 28 MiB/s wr, 1.90k op/s 2023-09-12T16:09:20.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 210.0 s, 870.6 tps, lat 112.227 ms stddev 100.401 2023-09-12T16:09:20.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:20 smithi130 ceph-mon[98670]: pgmap v484: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 27 MiB/s wr, 1.58k op/s 2023-09-12T16:09:20.482 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:20 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:09:20.482 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:20 smithi159 ceph-mon[102522]: pgmap v484: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 27 MiB/s wr, 1.58k op/s 2023-09-12T16:09:20.482 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:20 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:09:20.500 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:20 smithi203 ceph-mon[103431]: pgmap v484: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 27 MiB/s wr, 1.58k op/s 2023-09-12T16:09:20.500 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:20 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:09:22.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:22 smithi130 ceph-mon[98670]: pgmap v485: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 20 MiB/s wr, 956 op/s 2023-09-12T16:09:22.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:22 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:09:22.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:22 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:09:22.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:22 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:09:22.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:22 smithi159 ceph-mon[102522]: pgmap v485: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 20 MiB/s wr, 956 op/s 2023-09-12T16:09:22.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:22 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:09:22.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:22 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:09:22.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:22 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:09:22.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:22 smithi203 ceph-mon[103431]: pgmap v485: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 20 MiB/s wr, 956 op/s 2023-09-12T16:09:22.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:22 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:09:22.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:22 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:09:22.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:22 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:09:24.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:24 smithi130 ceph-mon[98670]: pgmap v486: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2023-09-12T16:09:24.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:24 smithi159 ceph-mon[102522]: pgmap v486: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2023-09-12T16:09:24.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:24 smithi203 ceph-mon[103431]: pgmap v486: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2023-09-12T16:09:25.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 215.0 s, 305.9 tps, lat 327.935 ms stddev 637.456 2023-09-12T16:09:26.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:26 smithi130 ceph-mon[98670]: pgmap v487: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 30 MiB/s wr, 2.20k op/s 2023-09-12T16:09:26.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:26 smithi159 ceph-mon[102522]: pgmap v487: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 30 MiB/s wr, 2.20k op/s 2023-09-12T16:09:26.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:26 smithi203 ceph-mon[103431]: pgmap v487: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 30 MiB/s wr, 2.20k op/s 2023-09-12T16:09:28.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:27 smithi130 ceph-mon[98670]: pgmap v488: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 27 MiB/s wr, 2.24k op/s 2023-09-12T16:09:28.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:27 smithi159 ceph-mon[102522]: pgmap v488: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 27 MiB/s wr, 2.24k op/s 2023-09-12T16:09:28.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:27 smithi203 ceph-mon[103431]: pgmap v488: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 27 MiB/s wr, 2.24k op/s 2023-09-12T16:09:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 220.0 s, 1508.4 tps, lat 66.922 ms stddev 52.906 2023-09-12T16:09:30.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:30 smithi130 ceph-mon[98670]: pgmap v489: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 28 MiB/s wr, 2.44k op/s 2023-09-12T16:09:30.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:30 smithi159 ceph-mon[102522]: pgmap v489: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 28 MiB/s wr, 2.44k op/s 2023-09-12T16:09:30.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:30 smithi203 ceph-mon[103431]: pgmap v489: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 28 MiB/s wr, 2.44k op/s 2023-09-12T16:09:32.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:32 smithi130 ceph-mon[98670]: pgmap v490: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.28k op/s 2023-09-12T16:09:32.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:32 smithi159 ceph-mon[102522]: pgmap v490: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.28k op/s 2023-09-12T16:09:32.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:32 smithi203 ceph-mon[103431]: pgmap v490: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.28k op/s 2023-09-12T16:09:34.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:34 smithi159 ceph-mon[102522]: pgmap v491: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 29 MiB/s wr, 2.44k op/s 2023-09-12T16:09:34.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:34 smithi130 ceph-mon[98670]: pgmap v491: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 29 MiB/s wr, 2.44k op/s 2023-09-12T16:09:34.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:34 smithi203 ceph-mon[103431]: pgmap v491: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 29 MiB/s wr, 2.44k op/s 2023-09-12T16:09:34.539 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:09:34.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.941+0000 7fcf7b491700 1 -- 172.21.15.130:0/3372203125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf7410c940 msgr2=0x7fcf7410cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:34.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.941+0000 7fcf7b491700 1 --2- 172.21.15.130:0/3372203125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf7410c940 0x7fcf7410cdc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcf7001a9a0 tx=0x7fcf7001acb0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.946+0000 7fcf7b491700 1 -- 172.21.15.130:0/3372203125 shutdown_connections 2023-09-12T16:09:34.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.946+0000 7fcf7b491700 1 --2- 172.21.15.130:0/3372203125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf7410c940 0x7fcf7410cdc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.946+0000 7fcf7b491700 1 --2- 172.21.15.130:0/3372203125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf74107600 0x7fcf74109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.946+0000 7fcf7b491700 1 --2- 172.21.15.130:0/3372203125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf74109f70 0x7fcf7410c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.946+0000 7fcf7b491700 1 -- 172.21.15.130:0/3372203125 >> 172.21.15.130:0/3372203125 conn(0x7fcf7406d3b0 msgr2=0x7fcf7406f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:34.952 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.951+0000 7fcf7b491700 1 -- 172.21.15.130:0/3372203125 shutdown_connections 2023-09-12T16:09:34.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.953+0000 7fcf7b491700 1 -- 172.21.15.130:0/3372203125 wait complete. 2023-09-12T16:09:34.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.954+0000 7fcf7b491700 1 Processor -- start 2023-09-12T16:09:34.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.954+0000 7fcf7b491700 1 -- start start 2023-09-12T16:09:34.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.954+0000 7fcf7b491700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf74107600 0x7fcf74117a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.954+0000 7fcf7b491700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf74109f70 0x7fcf74117f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.954+0000 7fcf7b491700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf7410c940 0x7fcf741184c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.954+0000 7fcf7b491700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf741c64b0 con 0x7fcf74107600 2023-09-12T16:09:34.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.954+0000 7fcf7b491700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf741c6620 con 0x7fcf7410c940 2023-09-12T16:09:34.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.954+0000 7fcf7b491700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf741c67e0 con 0x7fcf74109f70 2023-09-12T16:09:34.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf79a2e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf7410c940 0x7fcf741184c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:34.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf7922d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf74107600 0x7fcf74117a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:34.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf7922d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf74107600 0x7fcf74117a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56048/0 (socket says 172.21.15.130:56048) 2023-09-12T16:09:34.968 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf7922d700 1 -- 172.21.15.130:0/465294611 learned_addr learned my addr 172.21.15.130:0/465294611 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:34.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf7922d700 1 -- 172.21.15.130:0/465294611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf74109f70 msgr2=0x7fcf74117f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-12T16:09:34.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf7922d700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf74109f70 0x7fcf74117f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf7922d700 1 -- 172.21.15.130:0/465294611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf7410c940 msgr2=0x7fcf741184c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:34.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf7922d700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf7410c940 0x7fcf741184c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.969 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf7922d700 1 -- 172.21.15.130:0/465294611 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7001a600 con 0x7fcf74107600 2023-09-12T16:09:34.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.955+0000 7fcf7922d700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf74107600 0x7fcf74117a40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400b530 tx=0x7fcf6400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:34.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.956+0000 7fcf6a7fc700 1 -- 172.21.15.130:0/465294611 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64011830 con 0x7fcf74107600 2023-09-12T16:09:34.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.956+0000 7fcf6a7fc700 1 -- 172.21.15.130:0/465294611 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf64011e50 con 0x7fcf74107600 2023-09-12T16:09:34.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.956+0000 7fcf6a7fc700 1 -- 172.21.15.130:0/465294611 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400f2e0 con 0x7fcf74107600 2023-09-12T16:09:34.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.956+0000 7fcf7b491700 1 -- 172.21.15.130:0/465294611 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf741c6ac0 con 0x7fcf74107600 2023-09-12T16:09:34.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.956+0000 7fcf7b491700 1 -- 172.21.15.130:0/465294611 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf741c7000 con 0x7fcf74107600 2023-09-12T16:09:34.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf87657b80 1 Processor -- start 2023-09-12T16:09:34.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf87657b80 1 -- start start 2023-09-12T16:09:34.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x561796828020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617968285a0 0x561796826800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796826e00 0x5617968272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf87657b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x56179677f730 con 0x561796826e00 2023-09-12T16:09:34.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf87657b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56179677d050 con 0x561796827c00 2023-09-12T16:09:34.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf87657b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561796778940 con 0x5617968285a0 2023-09-12T16:09:34.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf6a7fc700 1 -- 172.21.15.130:0/465294611 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fcf64011990 con 0x7fcf74107600 2023-09-12T16:09:34.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf79a2e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x561796828020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:34.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.962+0000 7fcf79a2e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x561796828020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39310/0 (socket says 172.21.15.130:39310) 2023-09-12T16:09:34.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.963+0000 7fcf79a2e700 1 -- 172.21.15.130:0/2699750082 learned_addr learned my addr 172.21.15.130:0/2699750082 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:34.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.963+0000 7fcf7922d700 1 --2- 172.21.15.130:0/2699750082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617968285a0 0x561796826800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:34.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.965+0000 7fcf6a7fc700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf600a0840 0x7fcf600a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.967+0000 7fcf7922d700 1 -- 172.21.15.130:0/2699750082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 msgr2=0x561796828020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:34.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.967+0000 7fcf7922d700 1 --2- 172.21.15.130:0/2699750082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x561796828020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.967+0000 7fcf7922d700 1 -- 172.21.15.130:0/2699750082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796826e00 msgr2=0x5617968272b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:34.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.967+0000 7fcf7922d700 1 --2- 172.21.15.130:0/2699750082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796826e00 0x5617968272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.967+0000 7fcf7922d700 1 -- 172.21.15.130:0/2699750082 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617965a4de0 con 0x5617968285a0 2023-09-12T16:09:34.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.969+0000 7fcf79a2e700 1 --2- 172.21.15.130:0/2699750082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x561796828020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:09:34.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.969+0000 7fcf6a7fc700 1 -- 172.21.15.130:0/465294611 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fcf640cb890 con 0x7fcf74107600 2023-09-12T16:09:34.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.976+0000 7fcf7922d700 1 --2- 172.21.15.130:0/2699750082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617968285a0 0x561796826800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf640c8f90 tx=0x7fcf640d1d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82938bce3f332507 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:34.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.978+0000 7fcf5f7fe700 1 -- 172.21.15.130:0/2699750082 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64009380 con 0x5617968285a0 2023-09-12T16:09:34.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.978+0000 7fcf5f7fe700 1 -- 172.21.15.130:0/2699750082 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf640d1e90 con 0x5617968285a0 2023-09-12T16:09:34.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.978+0000 7fcf5f7fe700 1 -- 172.21.15.130:0/2699750082 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640987f0 con 0x5617968285a0 2023-09-12T16:09:34.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.981+0000 7fcf87657b80 1 -- 172.21.15.130:0/2699750082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617968285a0 msgr2=0x561796826800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:34.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.981+0000 7fcf87657b80 1 --2- 172.21.15.130:0/2699750082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617968285a0 0x561796826800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf640c8f90 tx=0x7fcf640d1d00 comp rx=0 tx=0).stop 2023-09-12T16:09:34.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.984+0000 7fcf79a2e700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf600a0840 0x7fcf600a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:34.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.984+0000 7fcf87657b80 1 -- 172.21.15.130:0/2699750082 shutdown_connections 2023-09-12T16:09:34.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.984+0000 7fcf87657b80 1 --2- 172.21.15.130:0/2699750082 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796826e00 0x5617968272b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.984+0000 7fcf87657b80 1 --2- 172.21.15.130:0/2699750082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x561796828020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.984+0000 7fcf87657b80 1 --2- 172.21.15.130:0/2699750082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617968285a0 0x561796826800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:34.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.984+0000 7fcf87657b80 1 -- 172.21.15.130:0/2699750082 >> 172.21.15.130:0/2699750082 conn(0x56179679b090 msgr2=0x561796821720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:34.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.995+0000 7fcf87657b80 1 -- 172.21.15.130:0/2699750082 shutdown_connections 2023-09-12T16:09:34.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.996+0000 7fcf87657b80 1 -- 172.21.15.130:0/2699750082 wait complete. 2023-09-12T16:09:34.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.997+0000 7fcf87657b80 1 -- start start 2023-09-12T16:09:34.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.997+0000 7fcf87657b80 1 Processor -- start 2023-09-12T16:09:34.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x561796874650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.998 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x561796874b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5617968285a0 0x56179687bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:34.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf87657b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561796716b70 con 0x5617968285a0 2023-09-12T16:09:34.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf87657b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561796716870 con 0x561796827c00 2023-09-12T16:09:34.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf87657b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617965fa920 con 0x561796826e00 2023-09-12T16:09:34.999 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf87657b80 10 client.0 _ll_register_callbacks cb 0x561796660a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:09:35.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf79a2e700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf600a0840 0x7fcf600a2d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcf7001c5d0 tx=0x7fcf7001a320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:35.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf7922d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x561796874650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:35.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf7922d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x561796874650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35672/0 (socket says 172.21.15.130:35672) 2023-09-12T16:09:35.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf7922d700 1 -- 172.21.15.130:0/572175802 learned_addr learned my addr 172.21.15.130:0/572175802 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:35.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf7922d700 1 -- 172.21.15.130:0/572175802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 msgr2=0x561796874b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:09:35.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf7922d700 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x561796874b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf7922d700 1 -- 172.21.15.130:0/572175802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5617968285a0 msgr2=0x56179687bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:35.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf7922d700 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5617968285a0 0x56179687bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.998+0000 7fcf7922d700 1 -- 172.21.15.130:0/572175802 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7410d940 con 0x561796826e00 2023-09-12T16:09:35.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:34.999+0000 7fcf7922d700 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x561796874650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf640c8f60 tx=0x7fcf64098b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca5b5f71c0440b5d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:35.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.003+0000 7fcf5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:09:35.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.003+0000 7fcf5cff9700 1 -- 172.21.15.130:0/572175802 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64098610 con 0x561796826e00 2023-09-12T16:09:35.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.003+0000 7fcf5cff9700 1 -- 172.21.15.130:0/572175802 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fcf640993b0 con 0x561796826e00 2023-09-12T16:09:35.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.003+0000 7fcf87657b80 1 -- 172.21.15.130:0/572175802 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617966d04f0 con 0x561796826e00 2023-09-12T16:09:35.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.003+0000 7fcf5cff9700 1 -- 172.21.15.130:0/572175802 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf640987f0 con 0x561796826e00 2023-09-12T16:09:35.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.003+0000 7fcf87657b80 10 client.25675 fetch_fsmap learned FSMap version 32 2023-09-12T16:09:35.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.003+0000 7fcf87657b80 1 -- 172.21.15.130:0/572175802 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fcf741c7000 con 0x561796826e00 2023-09-12T16:09:35.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.003+0000 7fcf5cff9700 1 -- 172.21.15.130:0/572175802 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf640a4020 con 0x561796826e00 2023-09-12T16:09:35.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.003+0000 7fcf5cff9700 1 -- 172.21.15.130:0/572175802 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf640b2060 con 0x561796826e00 2023-09-12T16:09:35.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.004+0000 7fcf87657b80 10 client.25675 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:09:35.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.004+0000 7fcf87657b80 10 client.25675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:09:35.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.004+0000 7fcf87657b80 20 client.25675 populate_metadata read hostname 'smithi130' 2023-09-12T16:09:35.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.004+0000 7fcf87657b80 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5617968a20e0 0x5617968a4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:35.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.004+0000 7fcf87657b80 4 client.25675 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:09:35.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.004+0000 7fcf87657b80 1 -- 172.21.15.130:0/572175802 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56179678d8a0 con 0x5617968a20e0 2023-09-12T16:09:35.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.004+0000 7fcf78a2c700 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5617968a20e0 0x5617968a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:35.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.004+0000 7fcf5cff9700 10 client.25675 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:09:35.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.005+0000 7fcf78a2c700 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5617968a20e0 0x5617968a4520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2653dbc77db548c8 server_cookie=96c62d397e6fe552 in_seq=0 out_seq=0 2023-09-12T16:09:35.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.009+0000 7fcf5cff9700 1 -- 172.21.15.130:0/572175802 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56179678d8a0 con 0x5617968a20e0 2023-09-12T16:09:35.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.009+0000 7fcf5cff9700 10 client.25675 handle_command_reply: tid=0 2023-09-12T16:09:35.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.009+0000 7fcf87657b80 1 client.25675 shutdown 2023-09-12T16:09:35.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.019+0000 7fcf87657b80 1 -- 172.21.15.130:0/572175802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 msgr2=0x561796874650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:35.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.019+0000 7fcf87657b80 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x561796874650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf640c8f60 tx=0x7fcf64098b80 comp rx=0 tx=0).stop 2023-09-12T16:09:35.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.019+0000 7fcf87657b80 1 -- 172.21.15.130:0/572175802 shutdown_connections 2023-09-12T16:09:35.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.019+0000 7fcf87657b80 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5617968a20e0 0x5617968a4520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.019+0000 7fcf87657b80 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5617968285a0 0x56179687bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.019+0000 7fcf87657b80 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x561796874650 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.019+0000 7fcf87657b80 1 --2- 172.21.15.130:0/572175802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x561796874b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.019+0000 7fcf87657b80 1 -- 172.21.15.130:0/572175802 >> 172.21.15.130:0/572175802 conn(0x56179679b090 msgr2=0x561796820dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:35.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.023+0000 7fcf5cff9700 0 client.25675 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:09:35.026 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.027+0000 7fcf87657b80 1 -- 172.21.15.130:0/572175802 shutdown_connections 2023-09-12T16:09:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.027+0000 7fcf87657b80 1 -- 172.21.15.130:0/572175802 wait complete. 2023-09-12T16:09:35.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.037+0000 7fcf87657b80 20 client.25675 trim_cache size 0 max 16384 2023-09-12T16:09:35.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.066+0000 7fcf87657b80 1 Processor -- start 2023-09-12T16:09:35.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.066+0000 7fcf87657b80 1 -- start start 2023-09-12T16:09:35.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.066+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x5617968721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:35.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.066+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x56179687bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:35.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.066+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5617968285a0 0x5617968726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:35.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.066+0000 7fcf87657b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561796822790 con 0x5617968285a0 2023-09-12T16:09:35.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.066+0000 7fcf87657b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561796872c30 con 0x561796827c00 2023-09-12T16:09:35.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.066+0000 7fcf87657b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617968741a0 con 0x561796826e00 2023-09-12T16:09:35.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.067+0000 7fcf78a2c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x56179687bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:35.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.067+0000 7fcf78a2c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x56179687bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39326/0 (socket says 172.21.15.130:39326) 2023-09-12T16:09:35.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.067+0000 7fcf7922d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x5617968721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:35.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.067+0000 7fcf78a2c700 1 -- 172.21.15.130:0/3674775950 learned_addr learned my addr 172.21.15.130:0/3674775950 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:35.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.067+0000 7fcf7922d700 1 -- 172.21.15.130:0/3674775950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 msgr2=0x56179687bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:35.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.067+0000 7fcf7922d700 1 --2- 172.21.15.130:0/3674775950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x56179687bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.067+0000 7fcf7922d700 1 -- 172.21.15.130:0/3674775950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5617968285a0 msgr2=0x5617968726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:09:35.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.067+0000 7fcf7922d700 1 --2- 172.21.15.130:0/3674775950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5617968285a0 0x5617968726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.067+0000 7fcf7922d700 1 -- 172.21.15.130:0/3674775950 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56179658dc60 con 0x561796826e00 2023-09-12T16:09:35.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.068+0000 7fcf7922d700 1 --2- 172.21.15.130:0/3674775950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x5617968721b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcf640989b0 tx=0x7fcf640d1d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba63dd76fb34b5c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:35.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.070+0000 7fcf5ffff700 1 -- 172.21.15.130:0/3674775950 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640955a0 con 0x561796826e00 2023-09-12T16:09:35.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.070+0000 7fcf5ffff700 1 -- 172.21.15.130:0/3674775950 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf640a4070 con 0x561796826e00 2023-09-12T16:09:35.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.071+0000 7fcf87657b80 1 -- 172.21.15.130:0/3674775950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 msgr2=0x5617968721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:35.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.071+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3674775950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x5617968721b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcf640989b0 tx=0x7fcf640d1d00 comp rx=0 tx=0).stop 2023-09-12T16:09:35.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.072+0000 7fcf87657b80 1 -- 172.21.15.130:0/3674775950 shutdown_connections 2023-09-12T16:09:35.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.072+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3674775950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5617968285a0 0x5617968726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.072+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3674775950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796826e00 0x5617968721b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.072+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3674775950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561796827c00 0x56179687bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.072+0000 7fcf87657b80 1 -- 172.21.15.130:0/3674775950 >> 172.21.15.130:0/3674775950 conn(0x56179679b090 msgr2=0x56179686ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:35.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.070+0000 7fcf5ffff700 1 -- 172.21.15.130:0/3674775950 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640a0af0 con 0x561796826e00 2023-09-12T16:09:35.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.083+0000 7fcf87657b80 1 -- 172.21.15.130:0/3674775950 shutdown_connections 2023-09-12T16:09:35.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 225.0 s, 862.6 tps, lat 109.029 ms stddev 256.094 2023-09-12T16:09:35.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.089+0000 7fcf87657b80 1 -- 172.21.15.130:0/3674775950 wait complete. 2023-09-12T16:09:35.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.090+0000 7fcf87657b80 1 -- start start 2023-09-12T16:09:35.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.090+0000 7fcf87657b80 1 Processor -- start 2023-09-12T16:09:35.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796827c00 0x56179687b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:35.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617968285a0 0x56179687bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:35.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf87657b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796879d00 0x56179687a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:35.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf87657b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x56179687a860 con 0x561796827c00 2023-09-12T16:09:35.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf87657b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56179687a9d0 con 0x5617968285a0 2023-09-12T16:09:35.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf87657b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617968f5f20 con 0x561796879d00 2023-09-12T16:09:35.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf87657b80 10 client.0 _ll_register_callbacks cb 0x7fcf7001ad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:09:35.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf7922d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796827c00 0x56179687b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:35.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf7922d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796827c00 0x56179687b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56072/0 (socket says 172.21.15.130:56072) 2023-09-12T16:09:35.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.091+0000 7fcf7922d700 1 -- 172.21.15.130:0/3445904146 learned_addr learned my addr 172.21.15.130:0/3445904146 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:09:35.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.092+0000 7fcf7922d700 1 -- 172.21.15.130:0/3445904146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796879d00 msgr2=0x56179687a180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:09:35.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.092+0000 7fcf7922d700 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796879d00 0x56179687a180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.092+0000 7fcf7922d700 1 -- 172.21.15.130:0/3445904146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617968285a0 msgr2=0x56179687bb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:09:35.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.092+0000 7fcf7922d700 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617968285a0 0x56179687bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.092+0000 7fcf7922d700 1 -- 172.21.15.130:0/3445904146 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf741c7000 con 0x561796827c00 2023-09-12T16:09:35.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.092+0000 7fcf7922d700 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796827c00 0x56179687b650 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcf64098c40 tx=0x7fcf64097260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16eb109ce9ac29e server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:09:35.100 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.101+0000 7fcf5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:09:35.101 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.101+0000 7fcf5cff9700 1 -- 172.21.15.130:0/3445904146 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640955a0 con 0x561796827c00 2023-09-12T16:09:35.101 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.101+0000 7fcf5cff9700 1 -- 172.21.15.130:0/3445904146 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fcf640a5080 con 0x561796827c00 2023-09-12T16:09:35.101 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.101+0000 7fcf87657b80 1 -- 172.21.15.130:0/3445904146 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf640d1e90 con 0x561796827c00 2023-09-12T16:09:35.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.101+0000 7fcf5cff9700 1 -- 172.21.15.130:0/3445904146 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf64009380 con 0x561796827c00 2023-09-12T16:09:35.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf87657b80 10 client.15996 fetch_fsmap learned FSMap version 32 2023-09-12T16:09:35.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf87657b80 1 -- 172.21.15.130:0/3445904146 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5617965cb250 con 0x561796827c00 2023-09-12T16:09:35.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf5cff9700 1 -- 172.21.15.130:0/3445904146 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf640d1e90 con 0x561796827c00 2023-09-12T16:09:35.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf5cff9700 1 -- 172.21.15.130:0/3445904146 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf640b2070 con 0x561796827c00 2023-09-12T16:09:35.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf87657b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:09:35.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf87657b80 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:09:35.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf87657b80 20 client.15996 populate_metadata read hostname 'smithi130' 2023-09-12T16:09:35.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5617968f71f0 0x5617968f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:09:35.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf87657b80 4 client.15996 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:09:35.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf87657b80 1 -- 172.21.15.130:0/3445904146 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561796771920 con 0x5617968f71f0 2023-09-12T16:09:35.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.102+0000 7fcf78a2c700 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5617968f71f0 0x5617968f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:09:35.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.103+0000 7fcf78a2c700 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5617968f71f0 0x5617968f95e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77c3d72555b6246f server_cookie=1ce95191402f67de in_seq=0 out_seq=0 2023-09-12T16:09:35.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.103+0000 7fcf5cff9700 10 client.15996 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:09:35.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.103+0000 7fcf5cff9700 1 -- 172.21.15.130:0/3445904146 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561796771920 con 0x5617968f71f0 2023-09-12T16:09:35.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.103+0000 7fcf5cff9700 10 client.15996 handle_command_reply: tid=0 2023-09-12T16:09:35.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.103+0000 7fcf87657b80 1 client.15996 shutdown 2023-09-12T16:09:35.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.104+0000 7fcf87657b80 1 -- 172.21.15.130:0/3445904146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796827c00 msgr2=0x56179687b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:35.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.104+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796827c00 0x56179687b650 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcf64098c40 tx=0x7fcf64097260 comp rx=0 tx=0).stop 2023-09-12T16:09:35.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.105+0000 7fcf87657b80 1 -- 172.21.15.130:0/3445904146 shutdown_connections 2023-09-12T16:09:35.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.105+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5617968f71f0 0x5617968f95e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.105+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561796879d00 0x56179687a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.105+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561796827c00 0x56179687b650 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.105+0000 7fcf87657b80 1 --2- 172.21.15.130:0/3445904146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617968285a0 0x56179687bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.105+0000 7fcf87657b80 1 -- 172.21.15.130:0/3445904146 >> 172.21.15.130:0/3445904146 conn(0x56179679b090 msgr2=0x5617968721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:35.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.109+0000 7fcf5cff9700 0 client.15996 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:09:35.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.114+0000 7fcf87657b80 1 -- 172.21.15.130:0/3445904146 shutdown_connections 2023-09-12T16:09:35.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.115+0000 7fcf87657b80 1 -- 172.21.15.130:0/3445904146 wait complete. 2023-09-12T16:09:35.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.120+0000 7fcf87657b80 20 client.15996 trim_cache size 0 max 16384 2023-09-12T16:09:35.120 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:09:35.120 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:09:35.120 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:09:35.121 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:09:35.121 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:09:35.121 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:09:35.121 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:09:35.121 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:09:35.122 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:09:35.122 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:09:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.125+0000 7fcf7b491700 1 -- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf600a0840 msgr2=0x7fcf600a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.125+0000 7fcf7b491700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf600a0840 0x7fcf600a2d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcf7001c5d0 tx=0x7fcf7001a320 comp rx=0 tx=0).stop 2023-09-12T16:09:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.125+0000 7fcf7b491700 1 -- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf74107600 msgr2=0x7fcf74117a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:09:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.125+0000 7fcf7b491700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf74107600 0x7fcf74117a40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400b530 tx=0x7fcf6400b8f0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.127+0000 7fcf7b491700 1 -- 172.21.15.130:0/465294611 shutdown_connections 2023-09-12T16:09:35.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.127+0000 7fcf7b491700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fcf600a0840 0x7fcf600a2d00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.127+0000 7fcf7b491700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf7410c940 0x7fcf741184c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.127+0000 7fcf7b491700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf74107600 0x7fcf74117a40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.127+0000 7fcf7b491700 1 --2- 172.21.15.130:0/465294611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf74109f70 0x7fcf74117f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:09:35.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.127+0000 7fcf7b491700 1 -- 172.21.15.130:0/465294611 >> 172.21.15.130:0/465294611 conn(0x7fcf7406d3b0 msgr2=0x7fcf7406f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:09:35.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.132+0000 7fcf7b491700 1 -- 172.21.15.130:0/465294611 shutdown_connections 2023-09-12T16:09:35.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:09:35.136+0000 7fcf7b491700 1 -- 172.21.15.130:0/465294611 wait complete. 2023-09-12T16:09:35.167 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:09:36.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:36 smithi130 ceph-mon[98670]: pgmap v492: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 36 MiB/s wr, 3.02k op/s 2023-09-12T16:09:36.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:36 smithi159 ceph-mon[102522]: pgmap v492: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 36 MiB/s wr, 3.02k op/s 2023-09-12T16:09:36.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:36 smithi203 ceph-mon[103431]: pgmap v492: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 36 MiB/s wr, 3.02k op/s 2023-09-12T16:09:38.232 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:37 smithi130 ceph-mon[98670]: pgmap v493: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 28 MiB/s wr, 2.21k op/s 2023-09-12T16:09:38.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:37 smithi159 ceph-mon[102522]: pgmap v493: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 28 MiB/s wr, 2.21k op/s 2023-09-12T16:09:38.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:37 smithi203 ceph-mon[103431]: pgmap v493: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 28 MiB/s wr, 2.21k op/s 2023-09-12T16:09:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 230.0 s, 1344.3 tps, lat 79.418 ms stddev 131.955 2023-09-12T16:09:40.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:40 smithi159 ceph-mon[102522]: pgmap v494: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 29 MiB/s wr, 2.28k op/s 2023-09-12T16:09:40.486 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:40 smithi130 ceph-mon[98670]: pgmap v494: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 29 MiB/s wr, 2.28k op/s 2023-09-12T16:09:40.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:40 smithi203 ceph-mon[103431]: pgmap v494: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 29 MiB/s wr, 2.28k op/s 2023-09-12T16:09:42.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:42 smithi130 ceph-mon[98670]: pgmap v495: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 28 MiB/s wr, 2.24k op/s 2023-09-12T16:09:42.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:42 smithi159 ceph-mon[102522]: pgmap v495: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 28 MiB/s wr, 2.24k op/s 2023-09-12T16:09:42.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:42 smithi203 ceph-mon[103431]: pgmap v495: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 28 MiB/s wr, 2.24k op/s 2023-09-12T16:09:44.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:44 smithi130 ceph-mon[98670]: pgmap v496: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 30 MiB/s wr, 2.28k op/s 2023-09-12T16:09:44.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:44 smithi159 ceph-mon[102522]: pgmap v496: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 30 MiB/s wr, 2.28k op/s 2023-09-12T16:09:44.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:44 smithi203 ceph-mon[103431]: pgmap v496: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 30 MiB/s wr, 2.28k op/s 2023-09-12T16:09:45.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 235.0 s, 1852.7 tps, lat 53.524 ms stddev 43.926 2023-09-12T16:09:46.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:46 smithi159 ceph-mon[102522]: pgmap v497: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 32 MiB/s wr, 2.37k op/s 2023-09-12T16:09:46.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:46 smithi130 ceph-mon[98670]: pgmap v497: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 32 MiB/s wr, 2.37k op/s 2023-09-12T16:09:46.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:46 smithi203 ceph-mon[103431]: pgmap v497: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 32 MiB/s wr, 2.37k op/s 2023-09-12T16:09:48.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:47 smithi130 ceph-mon[98670]: pgmap v498: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 26 MiB/s wr, 1.70k op/s 2023-09-12T16:09:48.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:47 smithi159 ceph-mon[102522]: pgmap v498: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 26 MiB/s wr, 1.70k op/s 2023-09-12T16:09:48.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:47 smithi203 ceph-mon[103431]: pgmap v498: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 26 MiB/s wr, 1.70k op/s 2023-09-12T16:09:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 240.0 s, 436.6 tps, lat 140.366 ms stddev 238.289 2023-09-12T16:09:50.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:50 smithi159 ceph-mon[102522]: pgmap v499: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 24 MiB/s wr, 1.35k op/s 2023-09-12T16:09:50.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:50 smithi130 ceph-mon[98670]: pgmap v499: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 24 MiB/s wr, 1.35k op/s 2023-09-12T16:09:50.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:50 smithi203 ceph-mon[103431]: pgmap v499: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 24 MiB/s wr, 1.35k op/s 2023-09-12T16:09:52.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:52 smithi130 ceph-mon[98670]: pgmap v500: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 29 MiB/s wr, 1.91k op/s 2023-09-12T16:09:52.485 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:52 smithi159 ceph-mon[102522]: pgmap v500: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 29 MiB/s wr, 1.91k op/s 2023-09-12T16:09:52.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:52 smithi203 ceph-mon[103431]: pgmap v500: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 29 MiB/s wr, 1.91k op/s 2023-09-12T16:09:54.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:54 smithi130 ceph-mon[98670]: pgmap v501: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 29 MiB/s wr, 2.10k op/s 2023-09-12T16:09:54.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:54 smithi159 ceph-mon[102522]: pgmap v501: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 29 MiB/s wr, 2.10k op/s 2023-09-12T16:09:54.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:54 smithi203 ceph-mon[103431]: pgmap v501: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 29 MiB/s wr, 2.10k op/s 2023-09-12T16:09:55.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 245.0 s, 753.7 tps, lat 184.604 ms stddev 402.927 2023-09-12T16:09:56.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:56 smithi130 ceph-mon[98670]: pgmap v502: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 31 MiB/s wr, 2.33k op/s 2023-09-12T16:09:56.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:56 smithi159 ceph-mon[102522]: pgmap v502: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 31 MiB/s wr, 2.33k op/s 2023-09-12T16:09:56.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:56 smithi203 ceph-mon[103431]: pgmap v502: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 31 MiB/s wr, 2.33k op/s 2023-09-12T16:09:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:09:57 smithi159 ceph-mon[102522]: pgmap v503: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.24k op/s 2023-09-12T16:09:58.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:09:57 smithi130 ceph-mon[98670]: pgmap v503: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.24k op/s 2023-09-12T16:09:58.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:09:57 smithi203 ceph-mon[103431]: pgmap v503: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.24k op/s 2023-09-12T16:10:00.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 250.0 s, 1711.1 tps, lat 58.689 ms stddev 55.335 2023-09-12T16:10:00.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:00 smithi159 ceph-mon[102522]: pgmap v504: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 24 MiB/s wr, 2.17k op/s 2023-09-12T16:10:00.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:00 smithi159 ceph-mon[102522]: overall HEALTH_OK 2023-09-12T16:10:00.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:00 smithi130 ceph-mon[98670]: pgmap v504: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 24 MiB/s wr, 2.17k op/s 2023-09-12T16:10:00.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:00 smithi130 ceph-mon[98670]: overall HEALTH_OK 2023-09-12T16:10:00.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:00 smithi203 ceph-mon[103431]: pgmap v504: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 24 MiB/s wr, 2.17k op/s 2023-09-12T16:10:00.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:00 smithi203 ceph-mon[103431]: overall HEALTH_OK 2023-09-12T16:10:02.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:02 smithi130 ceph-mon[98670]: pgmap v505: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 30 MiB/s wr, 2.35k op/s 2023-09-12T16:10:02.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:02 smithi159 ceph-mon[102522]: pgmap v505: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 30 MiB/s wr, 2.35k op/s 2023-09-12T16:10:02.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:02 smithi203 ceph-mon[103431]: pgmap v505: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 30 MiB/s wr, 2.35k op/s 2023-09-12T16:10:04.480 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:04 smithi130 ceph-mon[98670]: pgmap v506: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 25 MiB/s wr, 1.64k op/s 2023-09-12T16:10:04.484 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:04 smithi159 ceph-mon[102522]: pgmap v506: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 25 MiB/s wr, 1.64k op/s 2023-09-12T16:10:04.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:04 smithi203 ceph-mon[103431]: pgmap v506: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 25 MiB/s wr, 1.64k op/s 2023-09-12T16:10:05.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 255.0 s, 478.0 tps, lat 198.277 ms stddev 336.153 2023-09-12T16:10:05.168 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:10:05.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.647+0000 7fc933552700 1 -- 172.21.15.130:0/1894167511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc92c1086b0 msgr2=0x7fc92c108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:05.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.647+0000 7fc933552700 1 --2- 172.21.15.130:0/1894167511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc92c1086b0 0x7fc92c108b10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc92400c210 tx=0x7fc92400c520 comp rx=0 tx=0).stop 2023-09-12T16:10:05.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.659+0000 7fc933552700 1 -- 172.21.15.130:0/1894167511 shutdown_connections 2023-09-12T16:10:05.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.659+0000 7fc933552700 1 --2- 172.21.15.130:0/1894167511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c109050 0x7fc92c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.659+0000 7fc933552700 1 --2- 172.21.15.130:0/1894167511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc92c1079a0 0x7fc92c107dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.659+0000 7fc933552700 1 --2- 172.21.15.130:0/1894167511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc92c1086b0 0x7fc92c108b10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.659+0000 7fc933552700 1 -- 172.21.15.130:0/1894167511 >> 172.21.15.130:0/1894167511 conn(0x7fc92c06d3b0 msgr2=0x7fc92c06f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:05.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.665+0000 7fc933552700 1 -- 172.21.15.130:0/1894167511 shutdown_connections 2023-09-12T16:10:05.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.665+0000 7fc933552700 1 -- 172.21.15.130:0/1894167511 wait complete. 2023-09-12T16:10:05.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.666+0000 7fc933552700 1 Processor -- start 2023-09-12T16:10:05.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.666+0000 7fc933552700 1 -- start start 2023-09-12T16:10:05.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.666+0000 7fc933552700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc92c1079a0 0x7fc92c19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.666+0000 7fc933552700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc92c1086b0 0x7fc92c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.666+0000 7fc933552700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c109050 0x7fc92c1a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.666+0000 7fc933552700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1a4550 con 0x7fc92c1086b0 2023-09-12T16:10:05.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.666+0000 7fc933552700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1b5750 con 0x7fc92c1079a0 2023-09-12T16:10:05.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.667+0000 7fc933552700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1b5890 con 0x7fc92c109050 2023-09-12T16:10:05.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.667+0000 7fc931aef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c109050 0x7fc92c1a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.667+0000 7fc931aef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c109050 0x7fc92c1a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35700/0 (socket says 172.21.15.130:35700) 2023-09-12T16:10:05.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.667+0000 7fc931aef700 1 -- 172.21.15.130:0/1995674813 learned_addr learned my addr 172.21.15.130:0/1995674813 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:05.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.667+0000 7fc9312ee700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc92c1079a0 0x7fc92c19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.668+0000 7fc930aed700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc92c1086b0 0x7fc92c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.668+0000 7fc930aed700 1 -- 172.21.15.130:0/1995674813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c109050 msgr2=0x7fc92c1a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:05.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.668+0000 7fc930aed700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c109050 0x7fc92c1a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.668+0000 7fc930aed700 1 -- 172.21.15.130:0/1995674813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc92c1079a0 msgr2=0x7fc92c19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:05.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.668+0000 7fc930aed700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc92c1079a0 0x7fc92c19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.668+0000 7fc930aed700 1 -- 172.21.15.130:0/1995674813 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92400ae10 con 0x7fc92c1086b0 2023-09-12T16:10:05.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.668+0000 7fc930aed700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc92c1086b0 0x7fc92c19cf10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc92400a720 tx=0x7fc92400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:05.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.677+0000 7fc91e7fc700 1 -- 172.21.15.130:0/1995674813 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924022070 con 0x7fc92c1086b0 2023-09-12T16:10:05.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.677+0000 7fc91e7fc700 1 -- 172.21.15.130:0/1995674813 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc924008350 con 0x7fc92c1086b0 2023-09-12T16:10:05.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.677+0000 7fc933552700 1 -- 172.21.15.130:0/1995674813 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c1b5ac0 con 0x7fc92c1086b0 2023-09-12T16:10:05.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.677+0000 7fc91e7fc700 1 -- 172.21.15.130:0/1995674813 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92400f070 con 0x7fc92c1086b0 2023-09-12T16:10:05.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.677+0000 7fc933552700 1 -- 172.21.15.130:0/1995674813 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c1b5f60 con 0x7fc92c1086b0 2023-09-12T16:10:05.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.681+0000 7fc93f718b80 1 Processor -- start 2023-09-12T16:10:05.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc93f718b80 1 -- start start 2023-09-12T16:10:05.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 0x558cd69f1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 0x558cd69ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 0x558cd69f02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc93f718b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd6948730 con 0x558cd69f15a0 2023-09-12T16:10:05.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc93f718b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd6946050 con 0x558cd69efe00 2023-09-12T16:10:05.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc93f718b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd6941940 con 0x558cd69f0c00 2023-09-12T16:10:05.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc931aef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 0x558cd69f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc931aef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 0x558cd69f1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35704/0 (socket says 172.21.15.130:35704) 2023-09-12T16:10:05.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.682+0000 7fc931aef700 1 -- 172.21.15.130:0/1568533624 learned_addr learned my addr 172.21.15.130:0/1568533624 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:05.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.684+0000 7fc9312ee700 1 --2- 172.21.15.130:0/1568533624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 0x558cd69ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.685+0000 7fc931aef700 1 -- 172.21.15.130:0/1568533624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 msgr2=0x558cd69f02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:05.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.685+0000 7fc931aef700 1 --2- 172.21.15.130:0/1568533624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 0x558cd69f02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.685+0000 7fc931aef700 1 -- 172.21.15.130:0/1568533624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 msgr2=0x558cd69ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:05.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.685+0000 7fc931aef700 1 --2- 172.21.15.130:0/1568533624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 0x558cd69ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.692 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.685+0000 7fc931aef700 1 -- 172.21.15.130:0/1568533624 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd676dde0 con 0x558cd69f0c00 2023-09-12T16:10:05.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.689+0000 7fc91e7fc700 1 -- 172.21.15.130:0/1995674813 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fc924005020 con 0x7fc92c1086b0 2023-09-12T16:10:05.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.690+0000 7fc91e7fc700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc9180a0810 0x7fc9180a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.693 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.690+0000 7fc91e7fc700 1 -- 172.21.15.130:0/1995674813 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc9240cf340 con 0x7fc92c1086b0 2023-09-12T16:10:05.699 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.699+0000 7fc9312ee700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc9180a0810 0x7fc9180a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.699 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.699+0000 7fc931aef700 1 --2- 172.21.15.130:0/1568533624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 0x558cd69f1020 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc9200079d0 tx=0x7fc920007ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3b21540125db09b server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:05.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.699+0000 7fc9312ee700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc9180a0810 0x7fc9180a2cd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0719c0 tx=0x7fc928009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:05.701 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.700+0000 7fc9177fe700 1 -- 172.21.15.130:0/1568533624 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92002f860 con 0x558cd69f0c00 2023-09-12T16:10:05.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.700+0000 7fc9177fe700 1 -- 172.21.15.130:0/1568533624 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92002fe80 con 0x558cd69f0c00 2023-09-12T16:10:05.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.700+0000 7fc9177fe700 1 -- 172.21.15.130:0/1568533624 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920034e60 con 0x558cd69f0c00 2023-09-12T16:10:05.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 -- 172.21.15.130:0/1568533624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 msgr2=0x558cd69f1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:05.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 --2- 172.21.15.130:0/1568533624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 0x558cd69f1020 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc9200079d0 tx=0x7fc920007ce0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.702 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 -- 172.21.15.130:0/1568533624 shutdown_connections 2023-09-12T16:10:05.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 --2- 172.21.15.130:0/1568533624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 0x558cd69f02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 --2- 172.21.15.130:0/1568533624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 0x558cd69f1020 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 --2- 172.21.15.130:0/1568533624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 0x558cd69ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.703 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 -- 172.21.15.130:0/1568533624 >> 172.21.15.130:0/1568533624 conn(0x558cd6964090 msgr2=0x558cd69ea720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:05.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 -- 172.21.15.130:0/1568533624 shutdown_connections 2023-09-12T16:10:05.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 -- 172.21.15.130:0/1568533624 wait complete. 2023-09-12T16:10:05.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 -- start start 2023-09-12T16:10:05.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.701+0000 7fc93f718b80 1 Processor -- start 2023-09-12T16:10:05.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.702+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 0x558cd69f9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.702+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 0x558cd69f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.702+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 0x558cd6a00bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.702+0000 7fc93f718b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd68dfb70 con 0x558cd69f15a0 2023-09-12T16:10:05.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.702+0000 7fc93f718b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd68df870 con 0x558cd69efe00 2023-09-12T16:10:05.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.702+0000 7fc93f718b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd67c3920 con 0x558cd69f0c00 2023-09-12T16:10:05.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.702+0000 7fc93f718b80 10 client.0 _ll_register_callbacks cb 0x558cd6829a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:10:05.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.703+0000 7fc9312ee700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 0x558cd69f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.703+0000 7fc9312ee700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 0x558cd69f9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39360/0 (socket says 172.21.15.130:39360) 2023-09-12T16:10:05.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.703+0000 7fc9312ee700 1 -- 172.21.15.130:0/966313420 learned_addr learned my addr 172.21.15.130:0/966313420 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:05.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.703+0000 7fc931aef700 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 0x558cd6a00bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.703+0000 7fc9312ee700 1 -- 172.21.15.130:0/966313420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 msgr2=0x558cd69f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:05.714 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.703+0000 7fc9312ee700 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 0x558cd69f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.703+0000 7fc9312ee700 1 -- 172.21.15.130:0/966313420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 msgr2=0x558cd6a00bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:05.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.703+0000 7fc9312ee700 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 0x558cd6a00bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.703+0000 7fc9312ee700 1 -- 172.21.15.130:0/966313420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc920007600 con 0x558cd69efe00 2023-09-12T16:10:05.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.704+0000 7fc9312ee700 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 0x558cd69f9670 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc928009fd0 tx=0x7fc928004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb6ad4225788580 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:05.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.705+0000 7fc914ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:10:05.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.705+0000 7fc914ff9700 1 -- 172.21.15.130:0/966313420 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928011880 con 0x558cd69efe00 2023-09-12T16:10:05.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.705+0000 7fc914ff9700 1 -- 172.21.15.130:0/966313420 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc9280119e0 con 0x558cd69efe00 2023-09-12T16:10:05.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.706+0000 7fc93f718b80 1 -- 172.21.15.130:0/966313420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cd69568a0 con 0x558cd69efe00 2023-09-12T16:10:05.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.709+0000 7fc914ff9700 1 -- 172.21.15.130:0/966313420 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc928011ba0 con 0x558cd69efe00 2023-09-12T16:10:05.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.709+0000 7fc93f718b80 10 client.25472 fetch_fsmap learned FSMap version 32 2023-09-12T16:10:05.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.709+0000 7fc93f718b80 1 -- 172.21.15.130:0/966313420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558cd6756c60 con 0x558cd69efe00 2023-09-12T16:10:05.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.709+0000 7fc914ff9700 1 -- 172.21.15.130:0/966313420 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558cd69568a0 con 0x558cd69efe00 2023-09-12T16:10:05.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.709+0000 7fc914ff9700 1 -- 172.21.15.130:0/966313420 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc928028420 con 0x558cd69efe00 2023-09-12T16:10:05.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.710+0000 7fc93f718b80 10 client.25472 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:10:05.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.710+0000 7fc93f718b80 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:10:05.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.710+0000 7fc93f718b80 20 client.25472 populate_metadata read hostname 'smithi130' 2023-09-12T16:10:05.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.710+0000 7fc93f718b80 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558cd6a05590 0x558cd6a079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.710+0000 7fc93f718b80 4 client.25472 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:10:05.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.710+0000 7fc93f718b80 1 -- 172.21.15.130:0/966313420 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cd693a920 con 0x558cd6a05590 2023-09-12T16:10:05.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.711+0000 7fc930aed700 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558cd6a05590 0x558cd6a079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.712+0000 7fc930aed700 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558cd6a05590 0x558cd6a079b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15bd9a304eeb95b server_cookie=e9cab685d002d509 in_seq=0 out_seq=0 2023-09-12T16:10:05.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.717+0000 7fc914ff9700 10 client.25472 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:10:05.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.717+0000 7fc914ff9700 1 -- 172.21.15.130:0/966313420 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558cd693a920 con 0x558cd6a05590 2023-09-12T16:10:05.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.717+0000 7fc914ff9700 10 client.25472 handle_command_reply: tid=0 2023-09-12T16:10:05.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.717+0000 7fc93f718b80 1 client.25472 shutdown 2023-09-12T16:10:05.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.719+0000 7fc93f718b80 1 -- 172.21.15.130:0/966313420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 msgr2=0x558cd69f9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:05.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.719+0000 7fc93f718b80 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 0x558cd69f9670 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc928009fd0 tx=0x7fc928004750 comp rx=0 tx=0).stop 2023-09-12T16:10:05.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.719+0000 7fc93f718b80 1 -- 172.21.15.130:0/966313420 shutdown_connections 2023-09-12T16:10:05.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.719+0000 7fc93f718b80 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558cd6a05590 0x558cd6a079b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.719+0000 7fc93f718b80 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69f15a0 0x558cd6a00bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.719+0000 7fc93f718b80 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69efe00 0x558cd69f9670 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.719+0000 7fc93f718b80 1 --2- 172.21.15.130:0/966313420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f0c00 0x558cd69f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.719+0000 7fc93f718b80 1 -- 172.21.15.130:0/966313420 >> 172.21.15.130:0/966313420 conn(0x558cd6964090 msgr2=0x558cd69e9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:05.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.720+0000 7fc914ff9700 0 client.25472 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:10:05.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.720+0000 7fc93f718b80 1 -- 172.21.15.130:0/966313420 shutdown_connections 2023-09-12T16:10:05.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.720+0000 7fc93f718b80 1 -- 172.21.15.130:0/966313420 wait complete. 2023-09-12T16:10:05.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.723+0000 7fc93f718b80 20 client.25472 trim_cache size 0 max 16384 2023-09-12T16:10:05.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.760+0000 7fc93f718b80 1 Processor -- start 2023-09-12T16:10:05.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.761+0000 7fc93f718b80 1 -- start start 2023-09-12T16:10:05.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 0x558cd69f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69f0c00 0x558cd69f4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 0x558cd6a00bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc93f718b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd69f91c0 con 0x558cd69efe00 2023-09-12T16:10:05.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc93f718b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd69b4850 con 0x558cd69f0c00 2023-09-12T16:10:05.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc93f718b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd69eb9b0 con 0x558cd69f15a0 2023-09-12T16:10:05.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc9312ee700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 0x558cd69f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc9312ee700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 0x558cd69f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56108/0 (socket says 172.21.15.130:56108) 2023-09-12T16:10:05.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc9312ee700 1 -- 172.21.15.130:0/3841770656 learned_addr learned my addr 172.21.15.130:0/3841770656 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:05.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc9312ee700 1 -- 172.21.15.130:0/3841770656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 msgr2=0x558cd6a00bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:10:05.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc9312ee700 1 --2- 172.21.15.130:0/3841770656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 0x558cd6a00bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc9312ee700 1 -- 172.21.15.130:0/3841770656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69f0c00 msgr2=0x558cd69f4480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:10:05.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc9312ee700 1 --2- 172.21.15.130:0/3841770656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69f0c00 0x558cd69f4480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.762+0000 7fc9312ee700 1 -- 172.21.15.130:0/3841770656 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd69f04b0 con 0x558cd69efe00 2023-09-12T16:10:05.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.763+0000 7fc9312ee700 1 --2- 172.21.15.130:0/3841770656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 0x558cd69f3f40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc928028780 tx=0x7fc928009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6597d3290ef2b54e server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:05.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.768+0000 7fc917fff700 1 -- 172.21.15.130:0/3841770656 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928028e10 con 0x558cd69efe00 2023-09-12T16:10:05.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.768+0000 7fc917fff700 1 -- 172.21.15.130:0/3841770656 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9280289f0 con 0x558cd69efe00 2023-09-12T16:10:05.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.768+0000 7fc917fff700 1 -- 172.21.15.130:0/3841770656 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92801f680 con 0x558cd69efe00 2023-09-12T16:10:05.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.773+0000 7fc93f718b80 1 -- 172.21.15.130:0/3841770656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 msgr2=0x558cd69f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:05.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.773+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3841770656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 0x558cd69f3f40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc928028780 tx=0x7fc928009a40 comp rx=0 tx=0).stop 2023-09-12T16:10:05.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.780+0000 7fc93f718b80 1 -- 172.21.15.130:0/3841770656 shutdown_connections 2023-09-12T16:10:05.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.780+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3841770656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 0x558cd6a00bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.780+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3841770656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 0x558cd69f3f40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.780+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3841770656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69f0c00 0x558cd69f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.780+0000 7fc93f718b80 1 -- 172.21.15.130:0/3841770656 >> 172.21.15.130:0/3841770656 conn(0x558cd6964090 msgr2=0x558cd69f7430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:05.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.788+0000 7fc93f718b80 1 -- 172.21.15.130:0/3841770656 shutdown_connections 2023-09-12T16:10:05.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.788+0000 7fc93f718b80 1 -- 172.21.15.130:0/3841770656 wait complete. 2023-09-12T16:10:05.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc93f718b80 1 -- start start 2023-09-12T16:10:05.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc93f718b80 1 Processor -- start 2023-09-12T16:10:05.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 0x558cd6a004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69f0c00 0x558cd6a00a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc93f718b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 0x558cd69feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc93f718b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd69ff180 con 0x558cd69efe00 2023-09-12T16:10:05.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc93f718b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd69ff2f0 con 0x558cd69f0c00 2023-09-12T16:10:05.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc93f718b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd69ff4b0 con 0x558cd69f15a0 2023-09-12T16:10:05.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc93f718b80 10 client.0 _ll_register_callbacks cb 0x7fc9280043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:10:05.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc931aef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 0x558cd69feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc931aef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 0x558cd69feaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35726/0 (socket says 172.21.15.130:35726) 2023-09-12T16:10:05.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc931aef700 1 -- 172.21.15.130:0/3801584946 learned_addr learned my addr 172.21.15.130:0/3801584946 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:05.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.789+0000 7fc930aed700 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69f0c00 0x558cd6a00a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.790+0000 7fc931aef700 1 -- 172.21.15.130:0/3801584946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69f0c00 msgr2=0x558cd6a00a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:05.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.790+0000 7fc931aef700 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69f0c00 0x558cd6a00a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.790+0000 7fc931aef700 1 -- 172.21.15.130:0/3801584946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 msgr2=0x558cd6a004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:05.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.790+0000 7fc931aef700 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 0x558cd6a004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.790+0000 7fc931aef700 1 -- 172.21.15.130:0/3801584946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cd6756c60 con 0x558cd69f15a0 2023-09-12T16:10:05.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.790+0000 7fc931aef700 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 0x558cd69feaf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc9200079a0 tx=0x7fc920007090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69067808ab4cbfee server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:05.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.795+0000 7fc914ff9700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:10:05.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.795+0000 7fc914ff9700 1 -- 172.21.15.130:0/3801584946 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920004450 con 0x558cd69f15a0 2023-09-12T16:10:05.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.795+0000 7fc914ff9700 1 -- 172.21.15.130:0/3801584946 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc9200045b0 con 0x558cd69f15a0 2023-09-12T16:10:05.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.795+0000 7fc93f718b80 1 -- 172.21.15.130:0/3801584946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cd68994f0 con 0x558cd69f15a0 2023-09-12T16:10:05.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.796+0000 7fc914ff9700 1 -- 172.21.15.130:0/3801584946 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc92002f660 con 0x558cd69f15a0 2023-09-12T16:10:05.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.805+0000 7fc93f718b80 10 client.25687 fetch_fsmap learned FSMap version 32 2023-09-12T16:10:05.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.805+0000 7fc93f718b80 1 -- 172.21.15.130:0/3801584946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558cd6794250 con 0x558cd69f15a0 2023-09-12T16:10:05.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.810+0000 7fc914ff9700 1 -- 172.21.15.130:0/3801584946 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558cd68994f0 con 0x558cd69f15a0 2023-09-12T16:10:05.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.810+0000 7fc914ff9700 1 -- 172.21.15.130:0/3801584946 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc92004f070 con 0x558cd69f15a0 2023-09-12T16:10:05.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.811+0000 7fc93f718b80 10 client.25687 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:10:05.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.811+0000 7fc93f718b80 10 client.25687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:10:05.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.811+0000 7fc93f718b80 20 client.25687 populate_metadata read hostname 'smithi130' 2023-09-12T16:10:05.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.811+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558cd6a5a6f0 0x558cd6a5cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:05.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.811+0000 7fc93f718b80 4 client.25687 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:10:05.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.811+0000 7fc93f718b80 1 -- 172.21.15.130:0/3801584946 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558cd68fb0b0 con 0x558cd6a5a6f0 2023-09-12T16:10:05.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.819+0000 7fc9312ee700 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558cd6a5a6f0 0x558cd6a5cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:05.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.819+0000 7fc9312ee700 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558cd6a5a6f0 0x558cd6a5cb10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706f607d7e93d85a server_cookie=ed7ea43af4a96fcd in_seq=0 out_seq=0 2023-09-12T16:10:05.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.823+0000 7fc914ff9700 10 client.25687 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:10:05.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.823+0000 7fc914ff9700 1 -- 172.21.15.130:0/3801584946 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558cd68fb0b0 con 0x558cd6a5a6f0 2023-09-12T16:10:05.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.823+0000 7fc914ff9700 10 client.25687 handle_command_reply: tid=0 2023-09-12T16:10:05.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.823+0000 7fc93f718b80 1 client.25687 shutdown 2023-09-12T16:10:05.863 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.857+0000 7fc93f718b80 1 -- 172.21.15.130:0/3801584946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 msgr2=0x558cd69feaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:05.864 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.857+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 0x558cd69feaf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc9200079a0 tx=0x7fc920007090 comp rx=0 tx=0).stop 2023-09-12T16:10:05.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.865+0000 7fc93f718b80 1 -- 172.21.15.130:0/3801584946 shutdown_connections 2023-09-12T16:10:05.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.865+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558cd6a5a6f0 0x558cd6a5cb10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.865+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd69f15a0 0x558cd69feaf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.865+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558cd69efe00 0x558cd6a004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.865+0000 7fc93f718b80 1 --2- 172.21.15.130:0/3801584946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd69f0c00 0x558cd6a00a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.865+0000 7fc93f718b80 1 -- 172.21.15.130:0/3801584946 >> 172.21.15.130:0/3801584946 conn(0x558cd6964090 msgr2=0x558cd69f3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:05.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.866+0000 7fc914ff9700 0 client.25687 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:10:05.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.876+0000 7fc93f718b80 1 -- 172.21.15.130:0/3801584946 shutdown_connections 2023-09-12T16:10:05.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.878+0000 7fc93f718b80 1 -- 172.21.15.130:0/3801584946 wait complete. 2023-09-12T16:10:05.885 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:10:05.885 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:10:05.886 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:10:05.886 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:10:05.886 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:10:05.886 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:10:05.886 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:10:05.887 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:10:05.887 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:10:05.887 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:10:05.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.883+0000 7fc93f718b80 20 client.25687 trim_cache size 0 max 16384 2023-09-12T16:10:05.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.885+0000 7fc933552700 1 -- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc9180a0810 msgr2=0x7fc9180a2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:05.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.885+0000 7fc933552700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc9180a0810 0x7fc9180a2cd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0719c0 tx=0x7fc928009260 comp rx=0 tx=0).stop 2023-09-12T16:10:05.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.885+0000 7fc933552700 1 -- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc92c1086b0 msgr2=0x7fc92c19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:05.888 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.885+0000 7fc933552700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc92c1086b0 0x7fc92c19cf10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc92400a720 tx=0x7fc92400a430 comp rx=0 tx=0).stop 2023-09-12T16:10:05.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.891+0000 7fc933552700 1 -- 172.21.15.130:0/1995674813 shutdown_connections 2023-09-12T16:10:05.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.891+0000 7fc933552700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fc9180a0810 0x7fc9180a2cd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.891+0000 7fc933552700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c109050 0x7fc92c1a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.891+0000 7fc933552700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc92c1079a0 0x7fc92c19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.891+0000 7fc933552700 1 --2- 172.21.15.130:0/1995674813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc92c1086b0 0x7fc92c19cf10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:05.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.891+0000 7fc933552700 1 -- 172.21.15.130:0/1995674813 >> 172.21.15.130:0/1995674813 conn(0x7fc92c06d3b0 msgr2=0x7fc92c10bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:05.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.902+0000 7fc933552700 1 -- 172.21.15.130:0/1995674813 shutdown_connections 2023-09-12T16:10:05.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:05.903+0000 7fc933552700 1 -- 172.21.15.130:0/1995674813 wait complete. 2023-09-12T16:10:05.953 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:10:06.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:06 smithi130 ceph-mon[98670]: pgmap v507: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 28 MiB/s wr, 1.85k op/s 2023-09-12T16:10:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:06 smithi159 ceph-mon[102522]: pgmap v507: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 28 MiB/s wr, 1.85k op/s 2023-09-12T16:10:06.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:06 smithi203 ceph-mon[103431]: pgmap v507: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 28 MiB/s wr, 1.85k op/s 2023-09-12T16:10:08.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:07 smithi203 ceph-mon[103431]: pgmap v508: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 28 MiB/s wr, 2.06k op/s 2023-09-12T16:10:08.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:07 smithi130 ceph-mon[98670]: pgmap v508: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 28 MiB/s wr, 2.06k op/s 2023-09-12T16:10:08.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:07 smithi159 ceph-mon[102522]: pgmap v508: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 28 MiB/s wr, 2.06k op/s 2023-09-12T16:10:10.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 260.0 s, 1486.3 tps, lat 71.086 ms stddev 115.648 2023-09-12T16:10:10.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:10 smithi130 ceph-mon[98670]: pgmap v509: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 27 MiB/s wr, 1.94k op/s 2023-09-12T16:10:10.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:10 smithi159 ceph-mon[102522]: pgmap v509: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 27 MiB/s wr, 1.94k op/s 2023-09-12T16:10:10.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:10 smithi203 ceph-mon[103431]: pgmap v509: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 27 MiB/s wr, 1.94k op/s 2023-09-12T16:10:12.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:12 smithi159 ceph-mon[102522]: pgmap v510: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 34 MiB/s wr, 2.31k op/s 2023-09-12T16:10:12.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:12 smithi130 ceph-mon[98670]: pgmap v510: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 34 MiB/s wr, 2.31k op/s 2023-09-12T16:10:12.503 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:12 smithi203 ceph-mon[103431]: pgmap v510: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 34 MiB/s wr, 2.31k op/s 2023-09-12T16:10:14.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:14 smithi159 ceph-mon[102522]: pgmap v511: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 31 MiB/s wr, 2.33k op/s 2023-09-12T16:10:14.482 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:14 smithi130 ceph-mon[98670]: pgmap v511: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 31 MiB/s wr, 2.33k op/s 2023-09-12T16:10:14.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:14 smithi203 ceph-mon[103431]: pgmap v511: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 31 MiB/s wr, 2.33k op/s 2023-09-12T16:10:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 265.0 s, 719.0 tps, lat 139.030 ms stddev 451.537 2023-09-12T16:10:16.663 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:16 smithi159 ceph-mon[102522]: pgmap v512: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 32 MiB/s wr, 2.67k op/s 2023-09-12T16:10:16.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:16 smithi130 ceph-mon[98670]: pgmap v512: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 32 MiB/s wr, 2.67k op/s 2023-09-12T16:10:16.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:16 smithi203 ceph-mon[103431]: pgmap v512: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 32 MiB/s wr, 2.67k op/s 2023-09-12T16:10:18.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:18 smithi130 ceph-mon[98670]: pgmap v513: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 30 MiB/s wr, 2.31k op/s 2023-09-12T16:10:18.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:18 smithi159 ceph-mon[102522]: pgmap v513: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 30 MiB/s wr, 2.31k op/s 2023-09-12T16:10:18.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:18 smithi203 ceph-mon[103431]: pgmap v513: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 30 MiB/s wr, 2.31k op/s 2023-09-12T16:10:20.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 270.0 s, 989.6 tps, lat 100.340 ms stddev 237.832 2023-09-12T16:10:20.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:20 smithi159 ceph-mon[102522]: pgmap v514: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 26 MiB/s wr, 1.90k op/s 2023-09-12T16:10:20.486 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:20 smithi130 ceph-mon[98670]: pgmap v514: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 26 MiB/s wr, 1.90k op/s 2023-09-12T16:10:20.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:20 smithi203 ceph-mon[103431]: pgmap v514: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 26 MiB/s wr, 1.90k op/s 2023-09-12T16:10:22.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:22 smithi130 ceph-mon[98670]: pgmap v515: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 29 MiB/s wr, 2.09k op/s 2023-09-12T16:10:22.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:22 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:10:22.483 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:22 smithi159 ceph-mon[102522]: pgmap v515: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 29 MiB/s wr, 2.09k op/s 2023-09-12T16:10:22.483 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:22 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:10:22.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:22 smithi203 ceph-mon[103431]: pgmap v515: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 29 MiB/s wr, 2.09k op/s 2023-09-12T16:10:22.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:22 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:10:24.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:24 smithi130 ceph-mon[98670]: pgmap v516: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 26 MiB/s wr, 1.98k op/s 2023-09-12T16:10:24.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:24 smithi159 ceph-mon[102522]: pgmap v516: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 26 MiB/s wr, 1.98k op/s 2023-09-12T16:10:24.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:24 smithi203 ceph-mon[103431]: pgmap v516: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 26 MiB/s wr, 1.98k op/s 2023-09-12T16:10:25.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 275.0 s, 345.4 tps, lat 284.373 ms stddev 354.144 2023-09-12T16:10:26.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:26 smithi159 ceph-mon[102522]: pgmap v517: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:10:26.732 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:26 smithi130 ceph-mon[98670]: pgmap v517: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:10:26.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:26 smithi203 ceph-mon[103431]: pgmap v517: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:10:27.736 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:10:27.736 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:10:27.737 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:10:27.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:10:27.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:10:27.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:10:27.926 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:10:27.926 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:10:27.927 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:10:28.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:28 smithi130 ceph-mon[98670]: pgmap v518: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-09-12T16:10:28.752 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:28 smithi159 ceph-mon[102522]: pgmap v518: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-09-12T16:10:28.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:28 smithi203 ceph-mon[103431]: pgmap v518: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-09-12T16:10:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 280.0 s, 693.7 tps, lat 144.898 ms stddev 112.982 2023-09-12T16:10:30.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:30 smithi130 ceph-mon[98670]: pgmap v519: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-09-12T16:10:30.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:30 smithi203 ceph-mon[103431]: pgmap v519: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-09-12T16:10:30.749 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:30 smithi159 ceph-mon[102522]: pgmap v519: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-09-12T16:10:32.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:32 smithi159 ceph-mon[102522]: pgmap v520: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-09-12T16:10:32.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:32 smithi130 ceph-mon[98670]: pgmap v520: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-09-12T16:10:32.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:32 smithi203 ceph-mon[103431]: pgmap v520: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-09-12T16:10:34.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:34 smithi203 ceph-mon[103431]: pgmap v521: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-09-12T16:10:34.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:34 smithi159 ceph-mon[102522]: pgmap v521: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-09-12T16:10:34.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:34 smithi130 ceph-mon[98670]: pgmap v521: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-09-12T16:10:35.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 285.0 s, 1270.2 tps, lat 79.695 ms stddev 77.393 2023-09-12T16:10:35.955 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:10:36.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.510+0000 7fb93e405700 1 -- 172.21.15.130:0/3333484358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93810b040 msgr2=0x7fb93810d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:36.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.510+0000 7fb93e405700 1 --2- 172.21.15.130:0/3333484358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93810b040 0x7fb93810d430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb928009990 tx=0x7fb928009ca0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.511+0000 7fb93e405700 1 -- 172.21.15.130:0/3333484358 shutdown_connections 2023-09-12T16:10:36.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.511+0000 7fb93e405700 1 --2- 172.21.15.130:0/3333484358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb93810d9c0 0x7fb93810fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.511+0000 7fb93e405700 1 --2- 172.21.15.130:0/3333484358 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb938108710 0x7fb93810ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.511+0000 7fb93e405700 1 --2- 172.21.15.130:0/3333484358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93810b040 0x7fb93810d430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.511+0000 7fb93e405700 1 -- 172.21.15.130:0/3333484358 >> 172.21.15.130:0/3333484358 conn(0x7fb93806d3b0 msgr2=0x7fb93806f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:36.520 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.521+0000 7fb93e405700 1 -- 172.21.15.130:0/3333484358 shutdown_connections 2023-09-12T16:10:36.521 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.521+0000 7fb93e405700 1 -- 172.21.15.130:0/3333484358 wait complete. 2023-09-12T16:10:36.522 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.522+0000 7fb93e405700 1 Processor -- start 2023-09-12T16:10:36.522 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.522+0000 7fb93e405700 1 -- start start 2023-09-12T16:10:36.522 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb93e405700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb938108710 0x7fb93819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb93e405700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb93810b040 0x7fb93819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb93e405700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb93810d9c0 0x7fb9381a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb93e405700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381a6800 con 0x7fb93810d9c0 2023-09-12T16:10:36.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb93e405700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381a6970 con 0x7fb938108710 2023-09-12T16:10:36.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb93e405700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381a6b30 con 0x7fb93810b040 2023-09-12T16:10:36.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb92ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb93810b040 0x7fb93819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb937fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb938108710 0x7fb93819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb92ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb93810b040 0x7fb93819f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35740/0 (socket says 172.21.15.130:35740) 2023-09-12T16:10:36.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb92ffff700 1 -- 172.21.15.130:0/3919289855 learned_addr learned my addr 172.21.15.130:0/3919289855 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:36.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb93c9a2700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb93810d9c0 0x7fb9381a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb92ffff700 1 -- 172.21.15.130:0/3919289855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb938108710 msgr2=0x7fb93819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:36.526 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb92ffff700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb938108710 0x7fb93819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb92ffff700 1 -- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb93810d9c0 msgr2=0x7fb9381a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb92ffff700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb93810d9c0 0x7fb9381a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb92ffff700 1 -- 172.21.15.130:0/3919289855 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280095f0 con 0x7fb93810b040 2023-09-12T16:10:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb93c9a2700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb93810d9c0 0x7fb9381a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:10:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.523+0000 7fb92ffff700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb93810b040 0x7fb93819f1d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb9280036f0 tx=0x7fb928003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:36.527 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.524+0000 7fb935ffb700 1 -- 172.21.15.130:0/3919289855 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92801d070 con 0x7fb93810b040 2023-09-12T16:10:36.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.524+0000 7fb935ffb700 1 -- 172.21.15.130:0/3919289855 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb92800f460 con 0x7fb93810b040 2023-09-12T16:10:36.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.524+0000 7fb935ffb700 1 -- 172.21.15.130:0/3919289855 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928017650 con 0x7fb93810b040 2023-09-12T16:10:36.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.524+0000 7fb93e405700 1 -- 172.21.15.130:0/3919289855 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9381a6db0 con 0x7fb93810b040 2023-09-12T16:10:36.528 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.524+0000 7fb93e405700 1 -- 172.21.15.130:0/3919289855 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9381a71d0 con 0x7fb93810b040 2023-09-12T16:10:36.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.526+0000 7fb935ffb700 1 -- 172.21.15.130:0/3919289855 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fb92800f5d0 con 0x7fb93810b040 2023-09-12T16:10:36.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.527+0000 7fb935ffb700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb9240a06c0 0x7fb9240a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.527+0000 7fb935ffb700 1 -- 172.21.15.130:0/3919289855 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb9280cd0f0 con 0x7fb93810b040 2023-09-12T16:10:36.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.527+0000 7fb937fff700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb9240a06c0 0x7fb9240a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb94a5cbb80 1 Processor -- start 2023-09-12T16:10:36.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb94a5cbb80 1 -- start start 2023-09-12T16:10:36.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb937fff700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb9240a06c0 0x7fb9240a2b80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb93819fb60 tx=0x7fb920009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:36.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb2c00 0x5586aeeb3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeeb1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb1e00 0x5586aeeb22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aee0a730 con 0x5586aeeb35a0 2023-09-12T16:10:36.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aee08050 con 0x5586aeeb2c00 2023-09-12T16:10:36.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aee03940 con 0x5586aeeb1e00 2023-09-12T16:10:36.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb92ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeeb1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb937fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb2c00 0x5586aeeb3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb92ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeeb1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56136/0 (socket says 172.21.15.130:56136) 2023-09-12T16:10:36.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb92ffff700 1 -- 172.21.15.130:0/622522142 learned_addr learned my addr 172.21.15.130:0/622522142 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:36.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb937fff700 1 --2- 172.21.15.130:0/622522142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb2c00 0x5586aeeb3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39392/0 (socket says 172.21.15.130:39392) 2023-09-12T16:10:36.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.528+0000 7fb93c9a2700 1 --2- 172.21.15.130:0/622522142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb1e00 0x5586aeeb22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.529+0000 7fb93c9a2700 1 -- 172.21.15.130:0/622522142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb2c00 msgr2=0x5586aeeb3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:36.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.529+0000 7fb93c9a2700 1 --2- 172.21.15.130:0/622522142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb2c00 0x5586aeeb3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.529+0000 7fb93c9a2700 1 -- 172.21.15.130:0/622522142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 msgr2=0x5586aeeb1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:36.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.529+0000 7fb93c9a2700 1 --2- 172.21.15.130:0/622522142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeeb1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.529+0000 7fb93c9a2700 1 -- 172.21.15.130:0/622522142 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586aec2fde0 con 0x5586aeeb1e00 2023-09-12T16:10:36.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.529+0000 7fb93c9a2700 1 --2- 172.21.15.130:0/622522142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb1e00 0x5586aeeb22b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb930007850 tx=0x7fb930007b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e3c0773753e1fe server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:36.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.529+0000 7fb92e7fc700 1 -- 172.21.15.130:0/622522142 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930017070 con 0x5586aeeb1e00 2023-09-12T16:10:36.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.529+0000 7fb92e7fc700 1 -- 172.21.15.130:0/622522142 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb930010c40 con 0x5586aeeb1e00 2023-09-12T16:10:36.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.529+0000 7fb92e7fc700 1 -- 172.21.15.130:0/622522142 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93001b750 con 0x5586aeeb1e00 2023-09-12T16:10:36.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.530+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/622522142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb1e00 msgr2=0x5586aeeb22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:36.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.530+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/622522142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb1e00 0x5586aeeb22b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb930007850 tx=0x7fb930007b60 comp rx=0 tx=0).stop 2023-09-12T16:10:36.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.530+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/622522142 shutdown_connections 2023-09-12T16:10:36.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.530+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/622522142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb1e00 0x5586aeeb22b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.530+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/622522142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb2c00 0x5586aeeb3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.530+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/622522142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeeb1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.530+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/622522142 >> 172.21.15.130:0/622522142 conn(0x5586aee26090 msgr2=0x5586aeeac720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:36.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.530+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/622522142 shutdown_connections 2023-09-12T16:10:36.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.530+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/622522142 wait complete. 2023-09-12T16:10:36.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb94a5cbb80 1 -- start start 2023-09-12T16:10:36.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb94a5cbb80 1 Processor -- start 2023-09-12T16:10:36.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeebbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeec2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aeda1b70 con 0x5586aeeb35a0 2023-09-12T16:10:36.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aeda1870 con 0x5586aeeb1e00 2023-09-12T16:10:36.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aec85920 con 0x5586aeeb2c00 2023-09-12T16:10:36.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb94a5cbb80 10 client.0 _ll_register_callbacks cb 0x5586aeceba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:10:36.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb937fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb93c9a2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeec2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb937fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39398/0 (socket says 172.21.15.130:39398) 2023-09-12T16:10:36.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb937fff700 1 -- 172.21.15.130:0/2937314575 learned_addr learned my addr 172.21.15.130:0/2937314575 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:36.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.531+0000 7fb92ffff700 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeebbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb937fff700 1 -- 172.21.15.130:0/2937314575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 msgr2=0x5586aeebbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:36.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb937fff700 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeebbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb937fff700 1 -- 172.21.15.130:0/2937314575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 msgr2=0x5586aeec2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:36.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb937fff700 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeec2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb937fff700 1 -- 172.21.15.130:0/2937314575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb930007500 con 0x5586aeeb1e00 2023-09-12T16:10:36.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb93c9a2700 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeec2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:10:36.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb937fff700 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebb560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb920015730 tx=0x7fb920004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6617232559adc8a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:36.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb9177fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:10:36.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb9177fe700 1 -- 172.21.15.130:0/2937314575 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92000eea0 con 0x5586aeeb1e00 2023-09-12T16:10:36.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2937314575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586aed5b4f0 con 0x5586aeeb1e00 2023-09-12T16:10:36.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb9177fe700 1 -- 172.21.15.130:0/2937314575 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb920015ca0 con 0x5586aeeb1e00 2023-09-12T16:10:36.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb9177fe700 1 -- 172.21.15.130:0/2937314575 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb920015e60 con 0x5586aeeb1e00 2023-09-12T16:10:36.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb94a5cbb80 10 client.25487 fetch_fsmap learned FSMap version 32 2023-09-12T16:10:36.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.532+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2937314575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5586aec18c60 con 0x5586aeeb1e00 2023-09-12T16:10:36.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.533+0000 7fb9177fe700 1 -- 172.21.15.130:0/2937314575 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586aed5b4f0 con 0x5586aeeb1e00 2023-09-12T16:10:36.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.533+0000 7fb9177fe700 1 -- 172.21.15.130:0/2937314575 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb9200136b0 con 0x5586aeeb1e00 2023-09-12T16:10:36.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.533+0000 7fb94a5cbb80 10 client.25487 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:10:36.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.533+0000 7fb94a5cbb80 10 client.25487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:10:36.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.533+0000 7fb94a5cbb80 20 client.25487 populate_metadata read hostname 'smithi130' 2023-09-12T16:10:36.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.533+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5586aeec7380 0x5586aeec97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.533+0000 7fb94a5cbb80 4 client.25487 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:10:36.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.533+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2937314575 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586aee188a0 con 0x5586aeec7380 2023-09-12T16:10:36.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.533+0000 7fb92ffff700 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5586aeec7380 0x5586aeec97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.534+0000 7fb9177fe700 10 client.25487 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:10:36.548 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.534+0000 7fb92ffff700 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5586aeec7380 0x5586aeec97a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e429304d07a3196e server_cookie=72998a78323f2c7b in_seq=0 out_seq=0 2023-09-12T16:10:36.549 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.534+0000 7fb9177fe700 1 -- 172.21.15.130:0/2937314575 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5586aee188a0 con 0x5586aeec7380 2023-09-12T16:10:36.549 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.534+0000 7fb9177fe700 10 client.25487 handle_command_reply: tid=0 2023-09-12T16:10:36.549 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.534+0000 7fb94a5cbb80 1 client.25487 shutdown 2023-09-12T16:10:36.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2937314575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 msgr2=0x5586aeebb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:36.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebb560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb920015730 tx=0x7fb920004860 comp rx=0 tx=0).stop 2023-09-12T16:10:36.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2937314575 shutdown_connections 2023-09-12T16:10:36.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5586aeec7380 0x5586aeec97a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeec2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebb560 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb9177fe700 0 client.25487 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:10:36.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2937314575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeebbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2937314575 >> 172.21.15.130:0/2937314575 conn(0x5586aee26090 msgr2=0x5586aeeabdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:36.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2937314575 shutdown_connections 2023-09-12T16:10:36.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2937314575 wait complete. 2023-09-12T16:10:36.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.535+0000 7fb94a5cbb80 20 client.25487 trim_cache size 0 max 16384 2023-09-12T16:10:36.566 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.567+0000 7fb94a5cbb80 1 Processor -- start 2023-09-12T16:10:36.568 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.567+0000 7fb94a5cbb80 1 -- start start 2023-09-12T16:10:36.568 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.567+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb1e00 0x5586aeeb1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.569 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.567+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeec2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.569 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.567+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb35a0 0x5586aeeb5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.569 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.567+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aeebb130 con 0x5586aeeb1e00 2023-09-12T16:10:36.570 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.567+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aeebaa40 con 0x5586aeeb35a0 2023-09-12T16:10:36.570 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.567+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aeead670 con 0x5586aeeb2c00 2023-09-12T16:10:36.570 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.567+0000 7fb92ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeec2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.571 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb93c9a2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb35a0 0x5586aeeb5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.571 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb93c9a2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb35a0 0x5586aeeb5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39410/0 (socket says 172.21.15.130:39410) 2023-09-12T16:10:36.572 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeec2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35762/0 (socket says 172.21.15.130:35762) 2023-09-12T16:10:36.573 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb93c9a2700 1 -- 172.21.15.130:0/2588958106 learned_addr learned my addr 172.21.15.130:0/2588958106 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:36.573 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92ffff700 1 -- 172.21.15.130:0/2588958106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb35a0 msgr2=0x5586aeeb5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:36.573 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92ffff700 1 --2- 172.21.15.130:0/2588958106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb35a0 0x5586aeeb5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.573 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92ffff700 1 -- 172.21.15.130:0/2588958106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb1e00 msgr2=0x5586aeeb1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:36.573 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92ffff700 1 --2- 172.21.15.130:0/2588958106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb1e00 0x5586aeeb1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92ffff700 1 -- 172.21.15.130:0/2588958106 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586aec56250 con 0x5586aeeb2c00 2023-09-12T16:10:36.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb93c9a2700 1 --2- 172.21.15.130:0/2588958106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb35a0 0x5586aeeb5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:10:36.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92ffff700 1 --2- 172.21.15.130:0/2588958106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeec2aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb928096b00 tx=0x7fb9280ca750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed383be7d8ecce87 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:36.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92effd700 1 -- 172.21.15.130:0/2588958106 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92809a600 con 0x5586aeeb2c00 2023-09-12T16:10:36.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92effd700 1 -- 172.21.15.130:0/2588958106 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9280ade90 con 0x5586aeeb2c00 2023-09-12T16:10:36.575 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.568+0000 7fb92effd700 1 -- 172.21.15.130:0/2588958106 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928098ac0 con 0x5586aeeb2c00 2023-09-12T16:10:36.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.569+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2588958106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 msgr2=0x5586aeec2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:36.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.569+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2588958106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeec2aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb928096b00 tx=0x7fb9280ca750 comp rx=0 tx=0).stop 2023-09-12T16:10:36.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.569+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2588958106 shutdown_connections 2023-09-12T16:10:36.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.569+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2588958106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb35a0 0x5586aeeb5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.569+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2588958106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb1e00 0x5586aeeb1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.569+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/2588958106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeec2aa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.569+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2588958106 >> 172.21.15.130:0/2588958106 conn(0x5586aee26090 msgr2=0x5586aeeabdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:36.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.569+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2588958106 shutdown_connections 2023-09-12T16:10:36.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.569+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/2588958106 wait complete. 2023-09-12T16:10:36.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.570+0000 7fb94a5cbb80 1 -- start start 2023-09-12T16:10:36.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.570+0000 7fb94a5cbb80 1 Processor -- start 2023-09-12T16:10:36.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.570+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.570+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeebd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.570+0000 7fb94a5cbb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeec2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.570+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aeec2790 con 0x5586aeeb35a0 2023-09-12T16:10:36.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.570+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aeec2900 con 0x5586aeeb1e00 2023-09-12T16:10:36.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.570+0000 7fb94a5cbb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586aeec2ac0 con 0x5586aeeb2c00 2023-09-12T16:10:36.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.570+0000 7fb94a5cbb80 10 client.0 _ll_register_callbacks cb 0x7fb92000eaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:10:36.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb937fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb92ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeebd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb937fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39412/0 (socket says 172.21.15.130:39412) 2023-09-12T16:10:36.582 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb937fff700 1 -- 172.21.15.130:0/3268286388 learned_addr learned my addr 172.21.15.130:0/3268286388 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:10:36.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb93c9a2700 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeec2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb92ffff700 1 -- 172.21.15.130:0/3268286388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 msgr2=0x5586aeebcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:36.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb92ffff700 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb92ffff700 1 -- 172.21.15.130:0/3268286388 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 msgr2=0x5586aeec2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:10:36.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb92ffff700 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeec2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb92ffff700 1 -- 172.21.15.130:0/3268286388 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586aec2fde0 con 0x5586aeeb2c00 2023-09-12T16:10:36.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb937fff700 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:10:36.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb92ffff700 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeebd220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb9280a27a0 tx=0x7fb92800fca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d649084f508426f server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:10:36.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb9177fe700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:10:36.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb9177fe700 1 -- 172.21.15.130:0/3268286388 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92809e230 con 0x5586aeeb2c00 2023-09-12T16:10:36.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/3268286388 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb930010c40 con 0x5586aeeb2c00 2023-09-12T16:10:36.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.571+0000 7fb9177fe700 1 -- 172.21.15.130:0/3268286388 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb9280a25e0 con 0x5586aeeb2c00 2023-09-12T16:10:36.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.572+0000 7fb9177fe700 1 -- 172.21.15.130:0/3268286388 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92809a600 con 0x5586aeeb2c00 2023-09-12T16:10:36.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.572+0000 7fb94a5cbb80 10 client.25702 fetch_fsmap learned FSMap version 32 2023-09-12T16:10:36.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.572+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/3268286388 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fb9381a71d0 con 0x5586aeeb2c00 2023-09-12T16:10:36.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.572+0000 7fb9177fe700 1 -- 172.21.15.130:0/3268286388 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb930010c40 con 0x5586aeeb2c00 2023-09-12T16:10:36.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.572+0000 7fb9177fe700 1 -- 172.21.15.130:0/3268286388 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb928099af0 con 0x5586aeeb2c00 2023-09-12T16:10:36.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.577+0000 7fb94a5cbb80 10 client.25702 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:10:36.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.577+0000 7fb94a5cbb80 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:10:36.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.577+0000 7fb94a5cbb80 20 client.25702 populate_metadata read hostname 'smithi130' 2023-09-12T16:10:36.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.577+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5586aef2d230 0x5586aef2f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:10:36.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.577+0000 7fb94a5cbb80 4 client.25702 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:10:36.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.577+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/3268286388 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5586aedfc920 con 0x5586aef2d230 2023-09-12T16:10:36.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.577+0000 7fb937fff700 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5586aef2d230 0x5586aef2f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:10:36.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.577+0000 7fb937fff700 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5586aef2d230 0x5586aef2f650 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714aa11df006a920 server_cookie=9159c9f2f64e19e1 in_seq=0 out_seq=0 2023-09-12T16:10:36.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.577+0000 7fb9177fe700 10 client.25702 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:10:36.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb9177fe700 1 -- 172.21.15.130:0/3268286388 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5586aedfc920 con 0x5586aef2d230 2023-09-12T16:10:36.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb9177fe700 10 client.25702 handle_command_reply: tid=0 2023-09-12T16:10:36.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 client.25702 shutdown 2023-09-12T16:10:36.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/3268286388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 msgr2=0x5586aeebd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:36.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeebd220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb9280a27a0 tx=0x7fb92800fca0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/3268286388 shutdown_connections 2023-09-12T16:10:36.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5586aef2d230 0x5586aef2f650 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586aeeb35a0 0x5586aeec2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586aeeb1e00 0x5586aeebcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb9177fe700 0 client.25702 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:10:36.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 --2- 172.21.15.130:0/3268286388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586aeeb2c00 0x5586aeebd220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/3268286388 >> 172.21.15.130:0/3268286388 conn(0x5586aee26090 msgr2=0x5586aeeb5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:36.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/3268286388 shutdown_connections 2023-09-12T16:10:36.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 1 -- 172.21.15.130:0/3268286388 wait complete. 2023-09-12T16:10:36.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.578+0000 7fb94a5cbb80 20 client.25702 trim_cache size 0 max 16384 2023-09-12T16:10:36.591 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:10:36.591 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:10:36.591 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:10:36.591 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:10:36.591 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:10:36.592 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:10:36.592 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:10:36.592 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:10:36.592 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:10:36.592 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:10:36.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 -- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb9240a06c0 msgr2=0x7fb9240a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:36.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb9240a06c0 0x7fb9240a2b80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb93819fb60 tx=0x7fb920009190 comp rx=0 tx=0).stop 2023-09-12T16:10:36.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 -- 172.21.15.130:0/3919289855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb93810b040 msgr2=0x7fb93819f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:10:36.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb93810b040 0x7fb93819f1d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb9280036f0 tx=0x7fb928003cd0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 -- 172.21.15.130:0/3919289855 shutdown_connections 2023-09-12T16:10:36.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fb9240a06c0 0x7fb9240a2b80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb93810d9c0 0x7fb9381a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb938108710 0x7fb93819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 --2- 172.21.15.130:0/3919289855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb93810b040 0x7fb93819f1d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:10:36.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 -- 172.21.15.130:0/3919289855 >> 172.21.15.130:0/3919289855 conn(0x7fb93806d3b0 msgr2=0x7fb93810c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:10:36.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 -- 172.21.15.130:0/3919289855 shutdown_connections 2023-09-12T16:10:36.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:10:36.579+0000 7fb93e405700 1 -- 172.21.15.130:0/3919289855 wait complete. 2023-09-12T16:10:36.598 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:10:36.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:36 smithi159 ceph-mon[102522]: pgmap v522: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-12T16:10:36.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:36 smithi130 ceph-mon[98670]: pgmap v522: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-12T16:10:36.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:36 smithi203 ceph-mon[103431]: pgmap v522: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-12T16:10:38.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:37 smithi130 ceph-mon[98670]: pgmap v523: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 14 MiB/s wr, 709 op/s 2023-09-12T16:10:38.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:37 smithi159 ceph-mon[102522]: pgmap v523: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 14 MiB/s wr, 709 op/s 2023-09-12T16:10:38.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:37 smithi203 ceph-mon[103431]: pgmap v523: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 14 MiB/s wr, 709 op/s 2023-09-12T16:10:40.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 290.0 s, 1005.2 tps, lat 91.297 ms stddev 142.790 2023-09-12T16:10:40.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:40 smithi159 ceph-mon[102522]: pgmap v524: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 16 MiB/s wr, 720 op/s 2023-09-12T16:10:40.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:40 smithi130 ceph-mon[98670]: pgmap v524: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 16 MiB/s wr, 720 op/s 2023-09-12T16:10:40.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:40 smithi203 ceph-mon[103431]: pgmap v524: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 16 MiB/s wr, 720 op/s 2023-09-12T16:10:42.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:42 smithi159 ceph-mon[102522]: pgmap v525: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 21 MiB/s wr, 1.12k op/s 2023-09-12T16:10:42.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:42 smithi130 ceph-mon[98670]: pgmap v525: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 21 MiB/s wr, 1.12k op/s 2023-09-12T16:10:42.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:42 smithi203 ceph-mon[103431]: pgmap v525: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 21 MiB/s wr, 1.12k op/s 2023-09-12T16:10:44.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:44 smithi159 ceph-mon[102522]: pgmap v526: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 25 MiB/s wr, 1.52k op/s 2023-09-12T16:10:44.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:44 smithi130 ceph-mon[98670]: pgmap v526: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 25 MiB/s wr, 1.52k op/s 2023-09-12T16:10:44.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:44 smithi203 ceph-mon[103431]: pgmap v526: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 25 MiB/s wr, 1.52k op/s 2023-09-12T16:10:45.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 295.0 s, 1150.4 tps, lat 94.660 ms stddev 149.025 2023-09-12T16:10:46.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:46 smithi130 ceph-mon[98670]: pgmap v527: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 27 MiB/s wr, 1.90k op/s 2023-09-12T16:10:46.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:46 smithi159 ceph-mon[102522]: pgmap v527: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 27 MiB/s wr, 1.90k op/s 2023-09-12T16:10:46.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:46 smithi203 ceph-mon[103431]: pgmap v527: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 27 MiB/s wr, 1.90k op/s 2023-09-12T16:10:48.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:47 smithi203 ceph-mon[103431]: pgmap v528: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 26 MiB/s wr, 1.80k op/s 2023-09-12T16:10:48.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:47 smithi130 ceph-mon[98670]: pgmap v528: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 26 MiB/s wr, 1.80k op/s 2023-09-12T16:10:48.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:47 smithi159 ceph-mon[102522]: pgmap v528: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 26 MiB/s wr, 1.80k op/s 2023-09-12T16:10:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 300.0 s, 1718.6 tps, lat 58.107 ms stddev 53.875 2023-09-12T16:10:50.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:50 smithi159 ceph-mon[102522]: pgmap v529: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 27 MiB/s wr, 1.84k op/s 2023-09-12T16:10:50.485 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:50 smithi130 ceph-mon[98670]: pgmap v529: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 27 MiB/s wr, 1.84k op/s 2023-09-12T16:10:50.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:50 smithi203 ceph-mon[103431]: pgmap v529: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 27 MiB/s wr, 1.84k op/s 2023-09-12T16:10:52.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:52 smithi159 ceph-mon[102522]: pgmap v530: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 28 MiB/s wr, 1.88k op/s 2023-09-12T16:10:52.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:52 smithi130 ceph-mon[98670]: pgmap v530: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 28 MiB/s wr, 1.88k op/s 2023-09-12T16:10:52.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:52 smithi203 ceph-mon[103431]: pgmap v530: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 28 MiB/s wr, 1.88k op/s 2023-09-12T16:10:54.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:54 smithi203 ceph-mon[103431]: pgmap v531: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 28 MiB/s wr, 1.81k op/s 2023-09-12T16:10:54.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:54 smithi159 ceph-mon[102522]: pgmap v531: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 28 MiB/s wr, 1.81k op/s 2023-09-12T16:10:54.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:54 smithi130 ceph-mon[98670]: pgmap v531: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 28 MiB/s wr, 1.81k op/s 2023-09-12T16:10:55.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 305.0 s, 999.8 tps, lat 98.125 ms stddev 266.616 2023-09-12T16:10:56.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:56 smithi159 ceph-mon[102522]: pgmap v532: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 30 MiB/s wr, 1.83k op/s 2023-09-12T16:10:56.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:56 smithi130 ceph-mon[98670]: pgmap v532: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 30 MiB/s wr, 1.83k op/s 2023-09-12T16:10:56.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:56 smithi203 ceph-mon[103431]: pgmap v532: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 30 MiB/s wr, 1.83k op/s 2023-09-12T16:10:58.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:10:58 smithi130 ceph-mon[98670]: pgmap v533: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 24 MiB/s wr, 1.39k op/s 2023-09-12T16:10:58.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:10:58 smithi159 ceph-mon[102522]: pgmap v533: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 24 MiB/s wr, 1.39k op/s 2023-09-12T16:10:58.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:10:58 smithi203 ceph-mon[103431]: pgmap v533: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 24 MiB/s wr, 1.39k op/s 2023-09-12T16:11:00.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 310.0 s, 234.5 tps, lat 429.750 ms stddev 783.960 2023-09-12T16:11:00.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:00 smithi159 ceph-mon[102522]: pgmap v534: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 26 MiB/s wr, 1.60k op/s 2023-09-12T16:11:00.480 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:00 smithi130 ceph-mon[98670]: pgmap v534: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 26 MiB/s wr, 1.60k op/s 2023-09-12T16:11:00.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:00 smithi203 ceph-mon[103431]: pgmap v534: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 26 MiB/s wr, 1.60k op/s 2023-09-12T16:11:02.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:02 smithi159 ceph-mon[102522]: pgmap v535: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 26 MiB/s wr, 1.87k op/s 2023-09-12T16:11:02.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:02 smithi130 ceph-mon[98670]: pgmap v535: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 26 MiB/s wr, 1.87k op/s 2023-09-12T16:11:02.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:02 smithi203 ceph-mon[103431]: pgmap v535: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 26 MiB/s wr, 1.87k op/s 2023-09-12T16:11:04.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:04 smithi130 ceph-mon[98670]: pgmap v536: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 23 MiB/s wr, 1.84k op/s 2023-09-12T16:11:04.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:04 smithi159 ceph-mon[102522]: pgmap v536: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 23 MiB/s wr, 1.84k op/s 2023-09-12T16:11:04.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:04 smithi203 ceph-mon[103431]: pgmap v536: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 23 MiB/s wr, 1.84k op/s 2023-09-12T16:11:05.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 315.0 s, 1262.1 tps, lat 79.644 ms stddev 84.989 2023-09-12T16:11:06.599 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:11:06.702 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:06 smithi130 ceph-mon[98670]: pgmap v537: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 22 MiB/s wr, 1.62k op/s 2023-09-12T16:11:06.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:06 smithi159 ceph-mon[102522]: pgmap v537: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 22 MiB/s wr, 1.62k op/s 2023-09-12T16:11:06.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:06 smithi203 ceph-mon[103431]: pgmap v537: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 22 MiB/s wr, 1.62k op/s 2023-09-12T16:11:06.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.884+0000 7f7b36f89700 1 -- 172.21.15.130:0/2414388710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b300fee60 msgr2=0x7f7b30101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:06.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.884+0000 7f7b36f89700 1 --2- 172.21.15.130:0/2414388710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b300fee60 0x7f7b30101290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009990 tx=0x7f7b20009ca0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.884+0000 7f7b36f89700 1 -- 172.21.15.130:0/2414388710 shutdown_connections 2023-09-12T16:11:06.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.884+0000 7f7b36f89700 1 --2- 172.21.15.130:0/2414388710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b301041a0 0x7f7b30104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.884+0000 7f7b36f89700 1 --2- 172.21.15.130:0/2414388710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b300fee60 0x7f7b30101290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.884+0000 7f7b36f89700 1 --2- 172.21.15.130:0/2414388710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b301017d0 0x7f7b30103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.886 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.884+0000 7f7b36f89700 1 -- 172.21.15.130:0/2414388710 >> 172.21.15.130:0/2414388710 conn(0x7f7b300faa60 msgr2=0x7f7b300fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:06.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.884+0000 7f7b36f89700 1 -- 172.21.15.130:0/2414388710 shutdown_connections 2023-09-12T16:11:06.887 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.884+0000 7f7b36f89700 1 -- 172.21.15.130:0/2414388710 wait complete. 2023-09-12T16:11:06.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b36f89700 1 Processor -- start 2023-09-12T16:11:06.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b36f89700 1 -- start start 2023-09-12T16:11:06.889 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b36f89700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b300fee60 0x7f7b301940d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b36f89700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b301017d0 0x7f7b30194610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b36f89700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b301041a0 0x7f7b3019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b36f89700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3019bc50 con 0x7f7b301017d0 2023-09-12T16:11:06.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b36f89700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3019bd90 con 0x7f7b301041a0 2023-09-12T16:11:06.890 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b36f89700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3019bf20 con 0x7f7b300fee60 2023-09-12T16:11:06.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b2ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b301017d0 0x7f7b30194610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b35526700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b301041a0 0x7f7b3019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b2ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b301017d0 0x7f7b30194610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56176/0 (socket says 172.21.15.130:56176) 2023-09-12T16:11:06.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b35526700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b301041a0 0x7f7b3019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39436/0 (socket says 172.21.15.130:39436) 2023-09-12T16:11:06.891 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b2ffff700 1 -- 172.21.15.130:0/3873401405 learned_addr learned my addr 172.21.15.130:0/3873401405 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:06.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.885+0000 7f7b34d25700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b300fee60 0x7f7b301940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b35526700 1 -- 172.21.15.130:0/3873401405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b300fee60 msgr2=0x7f7b301940d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:06.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b35526700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b300fee60 0x7f7b301940d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b35526700 1 -- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b301017d0 msgr2=0x7f7b30194610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:06.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b35526700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b301017d0 0x7f7b30194610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b35526700 1 -- 172.21.15.130:0/3873401405 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200095f0 con 0x7f7b301041a0 2023-09-12T16:11:06.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b2ffff700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b301017d0 0x7f7b30194610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:11:06.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b35526700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b301041a0 0x7f7b3019b680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800efc0 tx=0x7f7b2800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:06.893 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b2dffb700 1 -- 172.21.15.130:0/3873401405 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28018070 con 0x7f7b301041a0 2023-09-12T16:11:06.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b2dffb700 1 -- 172.21.15.130:0/3873401405 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2800f040 con 0x7f7b301041a0 2023-09-12T16:11:06.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b2dffb700 1 -- 172.21.15.130:0/3873401405 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2800cd40 con 0x7f7b301041a0 2023-09-12T16:11:06.894 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b36f89700 1 -- 172.21.15.130:0/3873401405 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3019c1a0 con 0x7f7b301041a0 2023-09-12T16:11:06.897 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.886+0000 7f7b36f89700 1 -- 172.21.15.130:0/3873401405 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3019c6b0 con 0x7f7b301041a0 2023-09-12T16:11:06.897 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.889+0000 7f7b2dffb700 1 -- 172.21.15.130:0/3873401405 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f7b28003a00 con 0x7f7b301041a0 2023-09-12T16:11:06.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.889+0000 7f7b2dffb700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7b180b16d0 0x7f7b180b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.890+0000 7f7b2dffb700 1 -- 172.21.15.130:0/3873401405 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7b280cba60 con 0x7f7b301041a0 2023-09-12T16:11:06.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.891+0000 7f7b4314fb80 1 Processor -- start 2023-09-12T16:11:06.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.891+0000 7f7b34d25700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7b180b16d0 0x7f7b180b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.892+0000 7f7b34d25700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7b180b16d0 0x7f7b180b3b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000b2e0 tx=0x7f7b20005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:06.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b4314fb80 1 -- start start 2023-09-12T16:11:06.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87f4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f87f2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.903 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f2e00 0x5643f87f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.904 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f874b730 con 0x5643f87f2e00 2023-09-12T16:11:06.904 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f8749050 con 0x5643f87f3c00 2023-09-12T16:11:06.904 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f8744940 con 0x5643f87f45a0 2023-09-12T16:11:06.904 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b34d25700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b34d25700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87f4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39442/0 (socket says 172.21.15.130:39442) 2023-09-12T16:11:06.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b34d25700 1 -- 172.21.15.130:0/545725606 learned_addr learned my addr 172.21.15.130:0/545725606 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:06.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.893+0000 7f7b2ffff700 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f87f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.894+0000 7f7b35526700 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f2e00 0x5643f87f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.894+0000 7f7b34d25700 1 -- 172.21.15.130:0/545725606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 msgr2=0x5643f87f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:06.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.894+0000 7f7b34d25700 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f87f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.894+0000 7f7b34d25700 1 -- 172.21.15.130:0/545725606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f2e00 msgr2=0x5643f87f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:06.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.894+0000 7f7b34d25700 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f2e00 0x5643f87f32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.894+0000 7f7b34d25700 1 -- 172.21.15.130:0/545725606 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643f8570de0 con 0x5643f87f3c00 2023-09-12T16:11:06.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.894+0000 7f7b35526700 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f2e00 0x5643f87f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:11:06.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.894+0000 7f7b34d25700 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87f4020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000bf10 tx=0x7f7b20020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f634d20432c0d17c server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:06.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.895+0000 7f7b16ffd700 1 -- 172.21.15.130:0/545725606 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2002b070 con 0x5643f87f3c00 2023-09-12T16:11:06.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.895+0000 7f7b16ffd700 1 -- 172.21.15.130:0/545725606 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2001e040 con 0x5643f87f3c00 2023-09-12T16:11:06.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.895+0000 7f7b16ffd700 1 -- 172.21.15.130:0/545725606 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000f7d0 con 0x5643f87f3c00 2023-09-12T16:11:06.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 -- 172.21.15.130:0/545725606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 msgr2=0x5643f87f4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:06.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87f4020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000bf10 tx=0x7f7b20020000 comp rx=0 tx=0).stop 2023-09-12T16:11:06.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 -- 172.21.15.130:0/545725606 shutdown_connections 2023-09-12T16:11:06.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f2e00 0x5643f87f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87f4020 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/545725606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f87f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 -- 172.21.15.130:0/545725606 >> 172.21.15.130:0/545725606 conn(0x5643f8767090 msgr2=0x5643f87ed720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:06.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 -- 172.21.15.130:0/545725606 shutdown_connections 2023-09-12T16:11:06.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 -- 172.21.15.130:0/545725606 wait complete. 2023-09-12T16:11:06.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 -- start start 2023-09-12T16:11:06.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.896+0000 7f7b4314fb80 1 Processor -- start 2023-09-12T16:11:06.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 0x5643f8803b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f86e2b70 con 0x5643f87f45a0 2023-09-12T16:11:06.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f86e2870 con 0x5643f87f3c00 2023-09-12T16:11:06.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f85c6920 con 0x5643f87f2e00 2023-09-12T16:11:06.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b4314fb80 10 client.0 _ll_register_callbacks cb 0x5643f862ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:11:06.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b2ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b34d25700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b35526700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 0x5643f8803b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b2ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39448/0 (socket says 172.21.15.130:39448) 2023-09-12T16:11:06.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b2ffff700 1 -- 172.21.15.130:0/638014162 learned_addr learned my addr 172.21.15.130:0/638014162 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:06.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b2ffff700 1 -- 172.21.15.130:0/638014162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 msgr2=0x5643f87fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:06.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b2ffff700 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b2ffff700 1 -- 172.21.15.130:0/638014162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 msgr2=0x5643f8803b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:06.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b2ffff700 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 0x5643f8803b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b2ffff700 1 -- 172.21.15.130:0/638014162 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b30105150 con 0x5643f87f3c00 2023-09-12T16:11:06.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b35526700 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 0x5643f8803b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:11:06.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b34d25700 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:11:06.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b2ffff700 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87fcae0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b24012d20 tx=0x7f7b2400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbb34255fcc5fc95 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:06.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.897+0000 7f7b0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:11:06.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b0ffff700 1 -- 172.21.15.130:0/638014162 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24017070 con 0x5643f87f3c00 2023-09-12T16:11:06.913 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b4314fb80 1 -- 172.21.15.130:0/638014162 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643f869c4f0 con 0x5643f87f3c00 2023-09-12T16:11:06.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b0ffff700 1 -- 172.21.15.130:0/638014162 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7b2400fb80 con 0x5643f87f3c00 2023-09-12T16:11:06.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b0ffff700 1 -- 172.21.15.130:0/638014162 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b2400fd40 con 0x5643f87f3c00 2023-09-12T16:11:06.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b4314fb80 10 client.25505 fetch_fsmap learned FSMap version 32 2023-09-12T16:11:06.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b4314fb80 1 -- 172.21.15.130:0/638014162 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5643f8559c60 con 0x5643f87f3c00 2023-09-12T16:11:06.914 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b0ffff700 1 -- 172.21.15.130:0/638014162 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643f869c4f0 con 0x5643f87f3c00 2023-09-12T16:11:06.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b0ffff700 1 -- 172.21.15.130:0/638014162 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7b24003c80 con 0x5643f87f3c00 2023-09-12T16:11:06.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b4314fb80 10 client.25505 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:11:06.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b4314fb80 10 client.25505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:11:06.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b4314fb80 20 client.25505 populate_metadata read hostname 'smithi130' 2023-09-12T16:11:06.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5643f88083f0 0x5643f880a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b4314fb80 4 client.25505 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:11:06.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.898+0000 7f7b4314fb80 1 -- 172.21.15.130:0/638014162 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643f87598a0 con 0x5643f88083f0 2023-09-12T16:11:06.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.899+0000 7f7b34d25700 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5643f88083f0 0x5643f880a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.899+0000 7f7b0ffff700 10 client.25505 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:11:06.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.899+0000 7f7b34d25700 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5643f88083f0 0x5643f880a810 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4559089c9e7a07 server_cookie=717028272ef94ce0 in_seq=0 out_seq=0 2023-09-12T16:11:06.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.901+0000 7f7b0ffff700 1 -- 172.21.15.130:0/638014162 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5643f87598a0 con 0x5643f88083f0 2023-09-12T16:11:06.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.901+0000 7f7b0ffff700 10 client.25505 handle_command_reply: tid=0 2023-09-12T16:11:06.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.901+0000 7f7b4314fb80 1 client.25505 shutdown 2023-09-12T16:11:06.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.902+0000 7f7b4314fb80 1 -- 172.21.15.130:0/638014162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 msgr2=0x5643f87fcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:06.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.902+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87fcae0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b24012d20 tx=0x7f7b2400f6a0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.903+0000 7f7b4314fb80 1 -- 172.21.15.130:0/638014162 shutdown_connections 2023-09-12T16:11:06.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.903+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5643f88083f0 0x5643f880a810 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.903+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 0x5643f8803b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.903+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.903+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/638014162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87fcae0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.903+0000 7f7b4314fb80 1 -- 172.21.15.130:0/638014162 >> 172.21.15.130:0/638014162 conn(0x5643f8767090 msgr2=0x5643f87ecdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:06.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.903+0000 7f7b0ffff700 0 client.25505 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:11:06.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.904+0000 7f7b4314fb80 1 -- 172.21.15.130:0/638014162 shutdown_connections 2023-09-12T16:11:06.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.904+0000 7f7b4314fb80 1 -- 172.21.15.130:0/638014162 wait complete. 2023-09-12T16:11:06.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.904+0000 7f7b4314fb80 20 client.25505 trim_cache size 0 max 16384 2023-09-12T16:11:06.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.931+0000 7f7b4314fb80 1 Processor -- start 2023-09-12T16:11:06.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.931+0000 7f7b4314fb80 1 -- start start 2023-09-12T16:11:06.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.931+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.931+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.931+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 0x5643f8803a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.931+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f87ef230 con 0x5643f87f45a0 2023-09-12T16:11:06.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.931+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f87f2c50 con 0x5643f87f3c00 2023-09-12T16:11:06.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.931+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f87b7850 con 0x5643f87f2e00 2023-09-12T16:11:06.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.932+0000 7f7b34d25700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.932+0000 7f7b34d25700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35810/0 (socket says 172.21.15.130:35810) 2023-09-12T16:11:06.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.932+0000 7f7b34d25700 1 -- 172.21.15.130:0/3120391927 learned_addr learned my addr 172.21.15.130:0/3120391927 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:06.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.933+0000 7f7b34d25700 1 -- 172.21.15.130:0/3120391927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 msgr2=0x5643f87faa20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:11:06.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.933+0000 7f7b34d25700 1 --2- 172.21.15.130:0/3120391927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87faa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.933+0000 7f7b34d25700 1 -- 172.21.15.130:0/3120391927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 msgr2=0x5643f8803a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:06.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.933+0000 7f7b34d25700 1 --2- 172.21.15.130:0/3120391927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 0x5643f8803a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.933+0000 7f7b34d25700 1 -- 172.21.15.130:0/3120391927 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643f8597250 con 0x5643f87f2e00 2023-09-12T16:11:06.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.933+0000 7f7b34d25700 1 --2- 172.21.15.130:0/3120391927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fa4e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009350 tx=0x7f7b20026040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17e48d26dc7879a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:06.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.934+0000 7f7b177fe700 1 -- 172.21.15.130:0/3120391927 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2002b040 con 0x5643f87f2e00 2023-09-12T16:11:06.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.934+0000 7f7b177fe700 1 -- 172.21.15.130:0/3120391927 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b20010e80 con 0x5643f87f2e00 2023-09-12T16:11:06.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.934+0000 7f7b177fe700 1 -- 172.21.15.130:0/3120391927 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20004760 con 0x5643f87f2e00 2023-09-12T16:11:06.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.934+0000 7f7b4314fb80 1 -- 172.21.15.130:0/3120391927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 msgr2=0x5643f87fa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:06.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.934+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/3120391927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fa4e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009350 tx=0x7f7b20026040 comp rx=0 tx=0).stop 2023-09-12T16:11:06.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.935+0000 7f7b4314fb80 1 -- 172.21.15.130:0/3120391927 shutdown_connections 2023-09-12T16:11:06.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.935+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/3120391927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f45a0 0x5643f8803a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.935+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/3120391927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f2e00 0x5643f87fa4e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.935+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/3120391927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f3c00 0x5643f87faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.935+0000 7f7b4314fb80 1 -- 172.21.15.130:0/3120391927 >> 172.21.15.130:0/3120391927 conn(0x5643f8767090 msgr2=0x5643f8804290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:06.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.935+0000 7f7b4314fb80 1 -- 172.21.15.130:0/3120391927 shutdown_connections 2023-09-12T16:11:06.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.935+0000 7f7b4314fb80 1 -- 172.21.15.130:0/3120391927 wait complete. 2023-09-12T16:11:06.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.935+0000 7f7b4314fb80 1 -- start start 2023-09-12T16:11:06.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b4314fb80 1 Processor -- start 2023-09-12T16:11:06.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f2e00 0x5643f87ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f3c00 0x5643f87ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b4314fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f8802e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f8803370 con 0x5643f87f3c00 2023-09-12T16:11:06.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f88034e0 con 0x5643f87f2e00 2023-09-12T16:11:06.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b4314fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f88036a0 con 0x5643f87f45a0 2023-09-12T16:11:06.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b4314fb80 10 client.0 _ll_register_callbacks cb 0x7f7b240127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:11:06.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b35526700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f8802e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b35526700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f8802e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35816/0 (socket says 172.21.15.130:35816) 2023-09-12T16:11:06.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b34d25700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f2e00 0x5643f87ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.936+0000 7f7b35526700 1 -- 172.21.15.130:0/2332979743 learned_addr learned my addr 172.21.15.130:0/2332979743 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:06.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b35526700 1 -- 172.21.15.130:0/2332979743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f2e00 msgr2=0x5643f87ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:06.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b35526700 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f2e00 0x5643f87ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b35526700 1 -- 172.21.15.130:0/2332979743 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f3c00 msgr2=0x5643f87ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:06.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b35526700 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f3c00 0x5643f87ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b35526700 1 -- 172.21.15.130:0/2332979743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643f8570de0 con 0x5643f87f45a0 2023-09-12T16:11:06.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b35526700 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f8802e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800ef90 tx=0x7f7b28098870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4e986672b4b4bb7 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:06.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:11:06.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b0ffff700 1 -- 172.21.15.130:0/2332979743 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280a0c60 con 0x5643f87f45a0 2023-09-12T16:11:06.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b0ffff700 1 -- 172.21.15.130:0/2332979743 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7b28099610 con 0x5643f87f45a0 2023-09-12T16:11:06.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b4314fb80 1 -- 172.21.15.130:0/2332979743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b2001e040 con 0x5643f87f45a0 2023-09-12T16:11:06.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b0ffff700 1 -- 172.21.15.130:0/2332979743 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b280b1260 con 0x5643f87f45a0 2023-09-12T16:11:06.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b4314fb80 10 client.25720 fetch_fsmap learned FSMap version 32 2023-09-12T16:11:06.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b4314fb80 1 -- 172.21.15.130:0/2332979743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5643f856f2b0 con 0x5643f87f45a0 2023-09-12T16:11:06.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.937+0000 7f7b0ffff700 1 -- 172.21.15.130:0/2332979743 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2001e040 con 0x5643f87f45a0 2023-09-12T16:11:06.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.938+0000 7f7b0ffff700 1 -- 172.21.15.130:0/2332979743 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7b28094d90 con 0x5643f87f45a0 2023-09-12T16:11:06.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.938+0000 7f7b4314fb80 10 client.25720 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:11:06.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.938+0000 7f7b4314fb80 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:11:06.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.938+0000 7f7b4314fb80 20 client.25720 populate_metadata read hostname 'smithi130' 2023-09-12T16:11:06.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.938+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5643f886e2e0 0x5643f8870700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:06.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.938+0000 7f7b4314fb80 4 client.25720 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:11:06.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.938+0000 7f7b4314fb80 1 -- 172.21.15.130:0/2332979743 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5643f873d920 con 0x5643f886e2e0 2023-09-12T16:11:06.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.938+0000 7f7b34d25700 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5643f886e2e0 0x5643f8870700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:06.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b34d25700 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5643f886e2e0 0x5643f8870700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c693b1c87aae59a3 server_cookie=de6f52cec391517d in_seq=0 out_seq=0 2023-09-12T16:11:06.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b0ffff700 10 client.25720 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:11:06.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b0ffff700 1 -- 172.21.15.130:0/2332979743 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5643f873d920 con 0x5643f886e2e0 2023-09-12T16:11:06.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b0ffff700 10 client.25720 handle_command_reply: tid=0 2023-09-12T16:11:06.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 client.25720 shutdown 2023-09-12T16:11:06.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 -- 172.21.15.130:0/2332979743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 msgr2=0x5643f8802e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:06.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f8802e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800ef90 tx=0x7f7b28098870 comp rx=0 tx=0).stop 2023-09-12T16:11:06.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 -- 172.21.15.130:0/2332979743 shutdown_connections 2023-09-12T16:11:06.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5643f886e2e0 0x5643f8870700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5643f87f45a0 0x5643f8802e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5643f87f2e00 0x5643f87ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 --2- 172.21.15.130:0/2332979743 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5643f87f3c00 0x5643f87ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 -- 172.21.15.130:0/2332979743 >> 172.21.15.130:0/2332979743 conn(0x5643f8767090 msgr2=0x5643f87fa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:06.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b0ffff700 0 client.25720 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:11:06.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.939+0000 7f7b4314fb80 1 -- 172.21.15.130:0/2332979743 shutdown_connections 2023-09-12T16:11:06.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.940+0000 7f7b4314fb80 1 -- 172.21.15.130:0/2332979743 wait complete. 2023-09-12T16:11:06.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.940+0000 7f7b4314fb80 20 client.25720 trim_cache size 0 max 16384 2023-09-12T16:11:06.952 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:11:06.952 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:11:06.952 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:11:06.952 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:11:06.953 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:11:06.953 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:11:06.953 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:11:06.953 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:11:06.953 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:11:06.954 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:11:06.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.940+0000 7f7b36f89700 1 -- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7b180b16d0 msgr2=0x7f7b180b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:06.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7b180b16d0 0x7f7b180b3b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000b2e0 tx=0x7f7b20005fb0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 -- 172.21.15.130:0/3873401405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b301041a0 msgr2=0x7f7b3019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:06.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b301041a0 0x7f7b3019b680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800efc0 tx=0x7f7b2800c3a0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 -- 172.21.15.130:0/3873401405 shutdown_connections 2023-09-12T16:11:06.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7b180b16d0 0x7f7b180b3b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b301041a0 0x7f7b3019b680 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b300fee60 0x7f7b301940d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 --2- 172.21.15.130:0/3873401405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b301017d0 0x7f7b30194610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:06.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 -- 172.21.15.130:0/3873401405 >> 172.21.15.130:0/3873401405 conn(0x7f7b300faa60 msgr2=0x7f7b300fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:06.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 -- 172.21.15.130:0/3873401405 shutdown_connections 2023-09-12T16:11:06.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:06.941+0000 7f7b36f89700 1 -- 172.21.15.130:0/3873401405 wait complete. 2023-09-12T16:11:06.963 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:11:08.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:08 smithi159 ceph-mon[102522]: pgmap v538: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-12T16:11:08.480 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:08 smithi130 ceph-mon[98670]: pgmap v538: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-12T16:11:08.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:08 smithi203 ceph-mon[103431]: pgmap v538: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-12T16:11:10.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 320.0 s, 1267.1 tps, lat 78.441 ms stddev 170.548 2023-09-12T16:11:10.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:10 smithi130 ceph-mon[98670]: pgmap v539: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 20 MiB/s wr, 1.18k op/s 2023-09-12T16:11:10.484 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:10 smithi159 ceph-mon[102522]: pgmap v539: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 20 MiB/s wr, 1.18k op/s 2023-09-12T16:11:10.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:10 smithi203 ceph-mon[103431]: pgmap v539: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 20 MiB/s wr, 1.18k op/s 2023-09-12T16:11:12.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:12 smithi130 ceph-mon[98670]: pgmap v540: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 24 MiB/s wr, 1.45k op/s 2023-09-12T16:11:12.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:12 smithi159 ceph-mon[102522]: pgmap v540: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 24 MiB/s wr, 1.45k op/s 2023-09-12T16:11:12.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:12 smithi203 ceph-mon[103431]: pgmap v540: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 24 MiB/s wr, 1.45k op/s 2023-09-12T16:11:14.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:14 smithi130 ceph-mon[98670]: pgmap v541: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 24 MiB/s wr, 1.49k op/s 2023-09-12T16:11:14.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:14 smithi159 ceph-mon[102522]: pgmap v541: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 24 MiB/s wr, 1.49k op/s 2023-09-12T16:11:14.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:14 smithi203 ceph-mon[103431]: pgmap v541: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 24 MiB/s wr, 1.49k op/s 2023-09-12T16:11:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 325.0 s, 764.2 tps, lat 132.807 ms stddev 245.956 2023-09-12T16:11:16.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:16 smithi130 ceph-mon[98670]: pgmap v542: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 32 MiB/s wr, 2.40k op/s 2023-09-12T16:11:16.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:16 smithi159 ceph-mon[102522]: pgmap v542: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 32 MiB/s wr, 2.40k op/s 2023-09-12T16:11:16.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:16 smithi203 ceph-mon[103431]: pgmap v542: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 32 MiB/s wr, 2.40k op/s 2023-09-12T16:11:18.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:17 smithi203 ceph-mon[103431]: pgmap v543: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 30 MiB/s wr, 2.37k op/s 2023-09-12T16:11:18.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:17 smithi130 ceph-mon[98670]: pgmap v543: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 30 MiB/s wr, 2.37k op/s 2023-09-12T16:11:18.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:17 smithi159 ceph-mon[102522]: pgmap v543: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 30 MiB/s wr, 2.37k op/s 2023-09-12T16:11:20.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 330.0 s, 1578.0 tps, lat 63.268 ms stddev 55.844 2023-09-12T16:11:20.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:20 smithi130 ceph-mon[98670]: pgmap v544: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 32 MiB/s wr, 2.47k op/s 2023-09-12T16:11:20.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:20 smithi159 ceph-mon[102522]: pgmap v544: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 32 MiB/s wr, 2.47k op/s 2023-09-12T16:11:20.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:20 smithi203 ceph-mon[103431]: pgmap v544: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 32 MiB/s wr, 2.47k op/s 2023-09-12T16:11:22.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:22 smithi130 ceph-mon[98670]: pgmap v545: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 32 MiB/s wr, 2.44k op/s 2023-09-12T16:11:22.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:22 smithi159 ceph-mon[102522]: pgmap v545: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 32 MiB/s wr, 2.44k op/s 2023-09-12T16:11:22.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:22 smithi203 ceph-mon[103431]: pgmap v545: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 32 MiB/s wr, 2.44k op/s 2023-09-12T16:11:24.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:24 smithi159 ceph-mon[102522]: pgmap v546: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 27 MiB/s wr, 2.16k op/s 2023-09-12T16:11:24.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:24 smithi130 ceph-mon[98670]: pgmap v546: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 27 MiB/s wr, 2.16k op/s 2023-09-12T16:11:24.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:24 smithi203 ceph-mon[103431]: pgmap v546: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 27 MiB/s wr, 2.16k op/s 2023-09-12T16:11:25.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 335.0 s, 607.0 tps, lat 162.857 ms stddev 533.783 2023-09-12T16:11:26.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:26 smithi130 ceph-mon[98670]: pgmap v547: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 32 MiB/s wr, 2.69k op/s 2023-09-12T16:11:26.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:26 smithi159 ceph-mon[102522]: pgmap v547: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 32 MiB/s wr, 2.69k op/s 2023-09-12T16:11:26.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:26 smithi203 ceph-mon[103431]: pgmap v547: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 32 MiB/s wr, 2.69k op/s 2023-09-12T16:11:27.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:27 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:11:27.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:27 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:11:27.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:27 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:11:28.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: pgmap v548: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 24 MiB/s wr, 1.76k op/s 2023-09-12T16:11:28.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:11:28.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:11:28.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:28 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:11:28.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: pgmap v548: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 24 MiB/s wr, 1.76k op/s 2023-09-12T16:11:28.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.733 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.733 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.733 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.733 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.734 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.734 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.734 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.734 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.734 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:11:28.735 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:11:28.735 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:28 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:11:28.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: pgmap v548: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 24 MiB/s wr, 1.76k op/s 2023-09-12T16:11:28.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:11:28.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:11:28.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:11:28.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:28 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:11:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:29 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi203 to 196.2M 2023-09-12T16:11:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:29 smithi203 ceph-mon[103431]: Unable to set osd_memory_target on smithi203 to 205740953: error parsing value: Value '205740953' is below minimum 939524096 2023-09-12T16:11:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:29 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-09-12T16:11:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:29 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi130 to 196.2M 2023-09-12T16:11:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:29 smithi203 ceph-mon[103431]: Unable to set osd_memory_target on smithi130 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-09-12T16:11:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:29 smithi203 ceph-mon[103431]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-12T16:11:29.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:29 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi203 to 196.2M 2023-09-12T16:11:29.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:29 smithi130 ceph-mon[98670]: Unable to set osd_memory_target on smithi203 to 205740953: error parsing value: Value '205740953' is below minimum 939524096 2023-09-12T16:11:29.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:29 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-09-12T16:11:29.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:29 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi130 to 196.2M 2023-09-12T16:11:29.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:29 smithi130 ceph-mon[98670]: Unable to set osd_memory_target on smithi130 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-09-12T16:11:29.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:29 smithi130 ceph-mon[98670]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-12T16:11:29.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:29 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi203 to 196.2M 2023-09-12T16:11:29.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:29 smithi159 ceph-mon[102522]: Unable to set osd_memory_target on smithi203 to 205740953: error parsing value: Value '205740953' is below minimum 939524096 2023-09-12T16:11:29.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:29 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-09-12T16:11:29.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:29 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi130 to 196.2M 2023-09-12T16:11:29.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:29 smithi159 ceph-mon[102522]: Unable to set osd_memory_target on smithi130 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-09-12T16:11:29.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:29 smithi159 ceph-mon[102522]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-12T16:11:30.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 340.0 s, 619.9 tps, lat 92.555 ms stddev 200.019 2023-09-12T16:11:30.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:30 smithi203 ceph-mon[103431]: pgmap v549: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 24 MiB/s wr, 1.78k op/s 2023-09-12T16:11:30.982 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:30 smithi130 ceph-mon[98670]: pgmap v549: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 24 MiB/s wr, 1.78k op/s 2023-09-12T16:11:30.983 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:30 smithi159 ceph-mon[102522]: pgmap v549: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 24 MiB/s wr, 1.78k op/s 2023-09-12T16:11:32.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:32 smithi159 ceph-mon[102522]: pgmap v550: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 28 MiB/s wr, 2.23k op/s 2023-09-12T16:11:32.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:32 smithi130 ceph-mon[98670]: pgmap v550: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 28 MiB/s wr, 2.23k op/s 2023-09-12T16:11:32.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:32 smithi203 ceph-mon[103431]: pgmap v550: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 28 MiB/s wr, 2.23k op/s 2023-09-12T16:11:34.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:34 smithi130 ceph-mon[98670]: pgmap v551: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 25 MiB/s wr, 2.16k op/s 2023-09-12T16:11:34.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:34 smithi159 ceph-mon[102522]: pgmap v551: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 25 MiB/s wr, 2.16k op/s 2023-09-12T16:11:34.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:34 smithi203 ceph-mon[103431]: pgmap v551: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 25 MiB/s wr, 2.16k op/s 2023-09-12T16:11:35.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 345.0 s, 369.0 tps, lat 384.041 ms stddev 658.114 2023-09-12T16:11:36.964 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:11:36.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:36 smithi159 ceph-mon[102522]: pgmap v552: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 26 MiB/s wr, 2.17k op/s 2023-09-12T16:11:36.993 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:36 smithi130 ceph-mon[98670]: pgmap v552: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 26 MiB/s wr, 2.17k op/s 2023-09-12T16:11:36.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:36 smithi203 ceph-mon[103431]: pgmap v552: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 26 MiB/s wr, 2.17k op/s 2023-09-12T16:11:37.377 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.376+0000 7f32e28f7700 1 -- 172.21.15.130:0/707463389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32dc107600 msgr2=0x7f32dc109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:37.377 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.376+0000 7f32e28f7700 1 --2- 172.21.15.130:0/707463389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32dc107600 0x7f32dc109a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b0a0 tx=0x7f32d400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.390+0000 7f32e28f7700 1 -- 172.21.15.130:0/707463389 shutdown_connections 2023-09-12T16:11:37.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.390+0000 7f32e28f7700 1 --2- 172.21.15.130:0/707463389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32dc10c940 0x7f32dc10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.390+0000 7f32e28f7700 1 --2- 172.21.15.130:0/707463389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32dc107600 0x7f32dc109a30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.390+0000 7f32e28f7700 1 --2- 172.21.15.130:0/707463389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32dc109f70 0x7f32dc10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.390+0000 7f32e28f7700 1 -- 172.21.15.130:0/707463389 >> 172.21.15.130:0/707463389 conn(0x7f32dc06d3b0 msgr2=0x7f32dc06f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:37.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.401+0000 7f32e28f7700 1 -- 172.21.15.130:0/707463389 shutdown_connections 2023-09-12T16:11:37.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.412+0000 7f32e28f7700 1 -- 172.21.15.130:0/707463389 wait complete. 2023-09-12T16:11:37.418 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.412+0000 7f32e28f7700 1 Processor -- start 2023-09-12T16:11:37.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.419+0000 7f32e28f7700 1 -- start start 2023-09-12T16:11:37.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.419+0000 7f32e28f7700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32dc107600 0x7f32dc113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.419 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.419+0000 7f32e28f7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32dc109f70 0x7f32dc1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.419+0000 7f32e28f7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32dc10c940 0x7f32dc1b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.419+0000 7f32e28f7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1171c0 con 0x7f32dc10c940 2023-09-12T16:11:37.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.419+0000 7f32e28f7700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc117330 con 0x7f32dc107600 2023-09-12T16:11:37.420 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.419+0000 7f32e28f7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1b5970 con 0x7f32dc109f70 2023-09-12T16:11:37.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.420+0000 7f32db7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32dc109f70 0x7f32dc1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.420+0000 7f32db7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32dc109f70 0x7f32dc1140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35826/0 (socket says 172.21.15.130:35826) 2023-09-12T16:11:37.421 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.420+0000 7f32db7fe700 1 -- 172.21.15.130:0/2429184975 learned_addr learned my addr 172.21.15.130:0/2429184975 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:37.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.424+0000 7f32db7fe700 1 -- 172.21.15.130:0/2429184975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32dc107600 msgr2=0x7f32dc113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:37.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.424+0000 7f32db7fe700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32dc107600 0x7f32dc113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.424+0000 7f32db7fe700 1 -- 172.21.15.130:0/2429184975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32dc10c940 msgr2=0x7f32dc1b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:37.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.424+0000 7f32db7fe700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32dc10c940 0x7f32dc1b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.424+0000 7f32db7fe700 1 -- 172.21.15.130:0/2429184975 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d4009d00 con 0x7f32dc109f70 2023-09-12T16:11:37.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.424+0000 7f32db7fe700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32dc109f70 0x7f32dc1140e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00b530 tx=0x7f32cc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:37.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.428+0000 7f32d97fa700 1 -- 172.21.15.130:0/2429184975 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc011830 con 0x7f32dc109f70 2023-09-12T16:11:37.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.429+0000 7f32d97fa700 1 -- 172.21.15.130:0/2429184975 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32cc011e50 con 0x7f32dc109f70 2023-09-12T16:11:37.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.429+0000 7f32d97fa700 1 -- 172.21.15.130:0/2429184975 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc00f2e0 con 0x7f32dc109f70 2023-09-12T16:11:37.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.429+0000 7f32e28f7700 1 -- 172.21.15.130:0/2429184975 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32dc1b5bd0 con 0x7f32dc109f70 2023-09-12T16:11:37.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.429+0000 7f32e28f7700 1 -- 172.21.15.130:0/2429184975 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32dc1b60e0 con 0x7f32dc109f70 2023-09-12T16:11:37.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.436+0000 7f32eeabdb80 1 Processor -- start 2023-09-12T16:11:37.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.436+0000 7f32eeabdb80 1 -- start start 2023-09-12T16:11:37.436 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.436+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7bc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.436+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.436+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7bb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd713730 con 0x55bbdd7bc5a0 2023-09-12T16:11:37.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd711050 con 0x55bbdd7bbc00 2023-09-12T16:11:37.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd70c940 con 0x55bbdd7bae00 2023-09-12T16:11:37.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32e0e94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7bc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32e0e94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7bc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39478/0 (socket says 172.21.15.130:39478) 2023-09-12T16:11:37.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32e0e94700 1 -- 172.21.15.130:0/194462921 learned_addr learned my addr 172.21.15.130:0/194462921 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:37.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32e0e94700 1 -- 172.21.15.130:0/194462921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 msgr2=0x55bbdd7bb2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:11:37.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32e0e94700 1 --2- 172.21.15.130:0/194462921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7bb2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32e0e94700 1 -- 172.21.15.130:0/194462921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bc5a0 msgr2=0x55bbdd7ba800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:11:37.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32e0e94700 1 --2- 172.21.15.130:0/194462921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7ba800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32e0e94700 1 -- 172.21.15.130:0/194462921 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbdd538de0 con 0x55bbdd7bbc00 2023-09-12T16:11:37.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.437+0000 7f32e0e94700 1 --2- 172.21.15.130:0/194462921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7bc020 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009820 tx=0x7f32d0009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141daf47014655cf server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:37.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.438+0000 7f32d97fa700 1 -- 172.21.15.130:0/2429184975 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f32cc011990 con 0x7f32dc109f70 2023-09-12T16:11:37.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.439+0000 7f32d97fa700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32c40a0710 0x7f32c40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.439+0000 7f32d97fa700 1 -- 172.21.15.130:0/2429184975 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f32cc0cb960 con 0x7f32dc109f70 2023-09-12T16:11:37.441 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.441+0000 7f32dbfff700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32c40a0710 0x7f32c40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.444+0000 7f32c27fc700 1 -- 172.21.15.130:0/194462921 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0011830 con 0x55bbdd7bbc00 2023-09-12T16:11:37.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.444+0000 7f32c27fc700 1 -- 172.21.15.130:0/194462921 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32d0011e50 con 0x55bbdd7bbc00 2023-09-12T16:11:37.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.444+0000 7f32c27fc700 1 -- 172.21.15.130:0/194462921 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d000be70 con 0x55bbdd7bbc00 2023-09-12T16:11:37.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.448+0000 7f32eeabdb80 1 -- 172.21.15.130:0/194462921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 msgr2=0x55bbdd7bc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:37.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.448+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/194462921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7bc020 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009820 tx=0x7f32d0009b30 comp rx=0 tx=0).stop 2023-09-12T16:11:37.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.453+0000 7f32dbfff700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32c40a0710 0x7f32c40a2bd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b830 tx=0x7f32d4005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:37.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.471+0000 7f32eeabdb80 1 -- 172.21.15.130:0/194462921 shutdown_connections 2023-09-12T16:11:37.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.471+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/194462921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7bb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.471+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/194462921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7bc020 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.471+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/194462921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.471+0000 7f32eeabdb80 1 -- 172.21.15.130:0/194462921 >> 172.21.15.130:0/194462921 conn(0x55bbdd72f090 msgr2=0x55bbdd7b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:37.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.487+0000 7f32eeabdb80 1 -- 172.21.15.130:0/194462921 shutdown_connections 2023-09-12T16:11:37.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.490+0000 7f32eeabdb80 1 -- 172.21.15.130:0/194462921 wait complete. 2023-09-12T16:11:37.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.490+0000 7f32eeabdb80 1 -- start start 2023-09-12T16:11:37.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.491+0000 7f32eeabdb80 1 Processor -- start 2023-09-12T16:11:37.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.493+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.493+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.493+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.493+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd6aab70 con 0x55bbdd7bbc00 2023-09-12T16:11:37.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.493+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd6aa870 con 0x55bbdd7bc5a0 2023-09-12T16:11:37.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.493+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd58e920 con 0x55bbdd7bae00 2023-09-12T16:11:37.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.493+0000 7f32eeabdb80 10 client.0 _ll_register_callbacks cb 0x55bbdd5f4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:11:37.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.495+0000 7f32e0e94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.497+0000 7f32db7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.497+0000 7f32db7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56228/0 (socket says 172.21.15.130:56228) 2023-09-12T16:11:37.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.497+0000 7f32db7fe700 1 -- 172.21.15.130:0/3362919998 learned_addr learned my addr 172.21.15.130:0/3362919998 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:37.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.497+0000 7f32db7fe700 1 -- 172.21.15.130:0/3362919998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 msgr2=0x55bbdd7c46b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:11:37.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.497+0000 7f32db7fe700 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7c46b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.497+0000 7f32db7fe700 1 -- 172.21.15.130:0/3362919998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 msgr2=0x55bbdd7cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:37.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.497+0000 7f32db7fe700 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.497+0000 7f32db7fe700 1 -- 172.21.15.130:0/3362919998 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d0009450 con 0x55bbdd7bbc00 2023-09-12T16:11:37.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.497+0000 7f32db7fe700 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c4bf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0a0920 tx=0x7f32cc0109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6724ed53b7eb91 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:37.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.501+0000 7f32bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:11:37.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.501+0000 7f32bb7fe700 1 -- 172.21.15.130:0/3362919998 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0b1200 con 0x55bbdd7bbc00 2023-09-12T16:11:37.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.501+0000 7f32bb7fe700 1 -- 172.21.15.130:0/3362919998 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f32cc0a0e50 con 0x55bbdd7bbc00 2023-09-12T16:11:37.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.504+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3362919998 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbdd6644f0 con 0x55bbdd7bbc00 2023-09-12T16:11:37.514 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.515+0000 7f32bb7fe700 1 -- 172.21.15.130:0/3362919998 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32cc097180 con 0x55bbdd7bbc00 2023-09-12T16:11:37.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.525+0000 7f32eeabdb80 10 client.16032 fetch_fsmap learned FSMap version 32 2023-09-12T16:11:37.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.525+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3362919998 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bbdd521c60 con 0x55bbdd7bbc00 2023-09-12T16:11:37.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.533+0000 7f32bb7fe700 1 -- 172.21.15.130:0/3362919998 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bbdd6644f0 con 0x55bbdd7bbc00 2023-09-12T16:11:37.534 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.533+0000 7f32bb7fe700 1 -- 172.21.15.130:0/3362919998 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f32cc097d40 con 0x55bbdd7bbc00 2023-09-12T16:11:37.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.535+0000 7f32eeabdb80 10 client.16032 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:11:37.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.535+0000 7f32eeabdb80 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:11:37.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.535+0000 7f32eeabdb80 20 client.16032 populate_metadata read hostname 'smithi130' 2023-09-12T16:11:37.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.535+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55bbdd7d0400 0x55bbdd7d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.535+0000 7f32eeabdb80 4 client.16032 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:11:37.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.535+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3362919998 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbdd7218a0 con 0x55bbdd7d0400 2023-09-12T16:11:37.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.535+0000 7f32dbfff700 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55bbdd7d0400 0x55bbdd7d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.535+0000 7f32dbfff700 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55bbdd7d0400 0x55bbdd7d2820 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=528b24158fb0c9fb server_cookie=8e7a5b514c1b77c0 in_seq=0 out_seq=0 2023-09-12T16:11:37.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.539+0000 7f32bb7fe700 10 client.16032 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:11:37.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.539+0000 7f32bb7fe700 1 -- 172.21.15.130:0/3362919998 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bbdd7218a0 con 0x55bbdd7d0400 2023-09-12T16:11:37.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.539+0000 7f32bb7fe700 10 client.16032 handle_command_reply: tid=0 2023-09-12T16:11:37.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.539+0000 7f32eeabdb80 1 client.16032 shutdown 2023-09-12T16:11:37.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.544+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3362919998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 msgr2=0x55bbdd7c4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:37.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.544+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c4bf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0a0920 tx=0x7f32cc0109f0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.546+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3362919998 shutdown_connections 2023-09-12T16:11:37.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.546+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55bbdd7d0400 0x55bbdd7d2820 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.546+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.546+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.546+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3362919998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c4bf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.546+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3362919998 >> 172.21.15.130:0/3362919998 conn(0x55bbdd72f090 msgr2=0x55bbdd7b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:37.549 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.550+0000 7f32bb7fe700 0 client.16032 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:11:37.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.551+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3362919998 shutdown_connections 2023-09-12T16:11:37.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.551+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3362919998 wait complete. 2023-09-12T16:11:37.561 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.556+0000 7f32eeabdb80 20 client.16032 trim_cache size 0 max 16384 2023-09-12T16:11:37.592 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.586+0000 7f32eeabdb80 1 Processor -- start 2023-09-12T16:11:37.592 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32eeabdb80 1 -- start start 2023-09-12T16:11:37.592 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bae00 0x55bbdd7ba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd6644f0 con 0x55bbdd7bae00 2023-09-12T16:11:37.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd7ad400 con 0x55bbdd7bbc00 2023-09-12T16:11:37.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd7c5830 con 0x55bbdd7bc5a0 2023-09-12T16:11:37.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32db7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32db7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39490/0 (socket says 172.21.15.130:39490) 2023-09-12T16:11:37.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32e0e94700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32db7fe700 1 -- 172.21.15.130:0/3253921656 learned_addr learned my addr 172.21.15.130:0/3253921656 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:37.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.592+0000 7f32db7fe700 1 -- 172.21.15.130:0/3253921656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bc5a0 msgr2=0x55bbdd7c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:37.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.593+0000 7f32db7fe700 1 --2- 172.21.15.130:0/3253921656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.593+0000 7f32db7fe700 1 -- 172.21.15.130:0/3253921656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bae00 msgr2=0x55bbdd7ba710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:37.595 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.593+0000 7f32db7fe700 1 --2- 172.21.15.130:0/3253921656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bae00 0x55bbdd7ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.593+0000 7f32db7fe700 1 -- 172.21.15.130:0/3253921656 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbdd7d7410 con 0x55bbdd7bbc00 2023-09-12T16:11:37.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.593+0000 7f32db7fe700 1 --2- 172.21.15.130:0/3253921656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7cbb50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0c9140 tx=0x7f32cc09cd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732d8bfa99cd644c server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:37.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.597+0000 7f32c2ffd700 1 -- 172.21.15.130:0/3253921656 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc00d040 con 0x55bbdd7bbc00 2023-09-12T16:11:37.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.597+0000 7f32c2ffd700 1 -- 172.21.15.130:0/3253921656 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32cc0b1200 con 0x55bbdd7bbc00 2023-09-12T16:11:37.597 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.597+0000 7f32c2ffd700 1 -- 172.21.15.130:0/3253921656 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc097ed0 con 0x55bbdd7bbc00 2023-09-12T16:11:37.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.597+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3253921656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 msgr2=0x55bbdd7cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:37.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.597+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3253921656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7cbb50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0c9140 tx=0x7f32cc09cd90 comp rx=0 tx=0).stop 2023-09-12T16:11:37.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.598+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3253921656 shutdown_connections 2023-09-12T16:11:37.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.598+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3253921656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.598+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3253921656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bae00 0x55bbdd7ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.598+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/3253921656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7cbb50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.598+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3253921656 >> 172.21.15.130:0/3253921656 conn(0x55bbdd72f090 msgr2=0x55bbdd7d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:37.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.600+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3253921656 shutdown_connections 2023-09-12T16:11:37.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.604+0000 7f32eeabdb80 1 -- 172.21.15.130:0/3253921656 wait complete. 2023-09-12T16:11:37.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.604+0000 7f32eeabdb80 1 -- start start 2023-09-12T16:11:37.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.604+0000 7f32eeabdb80 1 Processor -- start 2023-09-12T16:11:37.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32eeabdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd7cb0e0 con 0x55bbdd7bbc00 2023-09-12T16:11:37.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd7cb250 con 0x55bbdd7bc5a0 2023-09-12T16:11:37.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32eeabdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdd7cb410 con 0x55bbdd7bae00 2023-09-12T16:11:37.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32eeabdb80 10 client.0 _ll_register_callbacks cb 0x7f32cc098740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:11:37.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32e0e94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32db7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32e0e94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39498/0 (socket says 172.21.15.130:39498) 2023-09-12T16:11:37.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32e0e94700 1 -- 172.21.15.130:0/2029793485 learned_addr learned my addr 172.21.15.130:0/2029793485 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:11:37.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.605+0000 7f32dbfff700 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32db7fe700 1 -- 172.21.15.130:0/2029793485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 msgr2=0x55bbdd7c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:37.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32db7fe700 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32db7fe700 1 -- 172.21.15.130:0/2029793485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 msgr2=0x55bbdd7caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:11:37.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32db7fe700 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32db7fe700 1 -- 172.21.15.130:0/2029793485 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbdd521c60 con 0x55bbdd7bbc00 2023-09-12T16:11:37.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32db7fe700 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c9b90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0952e0 tx=0x7f32cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f0f146fb1d6794 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:11:37.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:11:37.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32bb7fe700 1 -- 172.21.15.130:0/2029793485 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc097eb0 con 0x55bbdd7bbc00 2023-09-12T16:11:37.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32bb7fe700 1 -- 172.21.15.130:0/2029793485 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f32cc0a0e50 con 0x55bbdd7bbc00 2023-09-12T16:11:37.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.606+0000 7f32eeabdb80 1 -- 172.21.15.130:0/2029793485 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbdd705920 con 0x55bbdd7bbc00 2023-09-12T16:11:37.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.607+0000 7f32bb7fe700 1 -- 172.21.15.130:0/2029793485 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32cc098bb0 con 0x55bbdd7bbc00 2023-09-12T16:11:37.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.607+0000 7f32eeabdb80 10 client.16035 fetch_fsmap learned FSMap version 32 2023-09-12T16:11:37.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.607+0000 7f32eeabdb80 1 -- 172.21.15.130:0/2029793485 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bbdd55f250 con 0x55bbdd7bbc00 2023-09-12T16:11:37.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.608+0000 7f32bb7fe700 1 -- 172.21.15.130:0/2029793485 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bbdd705920 con 0x55bbdd7bbc00 2023-09-12T16:11:37.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.608+0000 7f32bb7fe700 1 -- 172.21.15.130:0/2029793485 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f32cc082170 con 0x55bbdd7bbc00 2023-09-12T16:11:37.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.608+0000 7f32eeabdb80 10 client.16035 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:11:37.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.608+0000 7f32eeabdb80 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:11:37.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.608+0000 7f32eeabdb80 20 client.16035 populate_metadata read hostname 'smithi130' 2023-09-12T16:11:37.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.608+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55bbdd824aa0 0x55bbdd826e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:11:37.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.608+0000 7f32eeabdb80 4 client.16035 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:11:37.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.608+0000 7f32eeabdb80 1 -- 172.21.15.130:0/2029793485 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bbdd6c60b0 con 0x55bbdd824aa0 2023-09-12T16:11:37.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.609+0000 7f32dbfff700 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55bbdd824aa0 0x55bbdd826e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:11:37.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.615+0000 7f32dbfff700 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55bbdd824aa0 0x55bbdd826e90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5caf5b5e89e824 server_cookie=507fbedfa39ae446 in_seq=0 out_seq=0 2023-09-12T16:11:37.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.615+0000 7f32bb7fe700 10 client.16035 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:11:37.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.615+0000 7f32bb7fe700 1 -- 172.21.15.130:0/2029793485 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55bbdd6c60b0 con 0x55bbdd824aa0 2023-09-12T16:11:37.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.615+0000 7f32bb7fe700 10 client.16035 handle_command_reply: tid=0 2023-09-12T16:11:37.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.615+0000 7f32eeabdb80 1 client.16035 shutdown 2023-09-12T16:11:37.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.617+0000 7f32eeabdb80 1 -- 172.21.15.130:0/2029793485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 msgr2=0x55bbdd7c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:37.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.617+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c9b90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0952e0 tx=0x7f32cc004750 comp rx=0 tx=0).stop 2023-09-12T16:11:37.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.617+0000 7f32eeabdb80 1 -- 172.21.15.130:0/2029793485 shutdown_connections 2023-09-12T16:11:37.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.617+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55bbdd824aa0 0x55bbdd826e90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.617+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdd7bc5a0 0x55bbdd7caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.617+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdd7bae00 0x55bbdd7c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.617+0000 7f32eeabdb80 1 --2- 172.21.15.130:0/2029793485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbdd7bbc00 0x55bbdd7c9b90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.617+0000 7f32eeabdb80 1 -- 172.21.15.130:0/2029793485 >> 172.21.15.130:0/2029793485 conn(0x55bbdd72f090 msgr2=0x55bbdd7c3360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:37.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.617+0000 7f32bb7fe700 0 client.16035 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:11:37.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.629+0000 7f32eeabdb80 1 -- 172.21.15.130:0/2029793485 shutdown_connections 2023-09-12T16:11:37.636 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.636+0000 7f32eeabdb80 1 -- 172.21.15.130:0/2029793485 wait complete. 2023-09-12T16:11:37.637 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.637+0000 7f32eeabdb80 20 client.16035 trim_cache size 0 max 16384 2023-09-12T16:11:37.639 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:11:37.639 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:11:37.640 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:11:37.640 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:11:37.640 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:11:37.640 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:11:37.641 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:11:37.641 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:11:37.641 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:11:37.642 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:11:37.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 -- 172.21.15.130:0/2429184975 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32c40a0710 msgr2=0x7f32c40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:37.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32c40a0710 0x7f32c40a2bd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b830 tx=0x7f32d4005a40 comp rx=0 tx=0).stop 2023-09-12T16:11:37.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 -- 172.21.15.130:0/2429184975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32dc109f70 msgr2=0x7f32dc1140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:11:37.648 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32dc109f70 0x7f32dc1140e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00b530 tx=0x7f32cc00b8f0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 -- 172.21.15.130:0/2429184975 shutdown_connections 2023-09-12T16:11:37.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32c40a0710 0x7f32c40a2bd0 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b830 tx=0x7f32d4005a40 comp rx=0 tx=0).stop 2023-09-12T16:11:37.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32dc10c940 0x7f32dc1b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32dc107600 0x7f32dc113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 --2- 172.21.15.130:0/2429184975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32dc109f70 0x7f32dc1140e0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00b530 tx=0x7f32cc00b8f0 comp rx=0 tx=0).stop 2023-09-12T16:11:37.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.648+0000 7f32e28f7700 1 -- 172.21.15.130:0/2429184975 >> 172.21.15.130:0/2429184975 conn(0x7f32dc06d3b0 msgr2=0x7f32dc10ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:11:37.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.666+0000 7f32e28f7700 1 -- 172.21.15.130:0/2429184975 shutdown_connections 2023-09-12T16:11:37.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:11:37.666+0000 7f32e28f7700 1 -- 172.21.15.130:0/2429184975 wait complete. 2023-09-12T16:11:37.727 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:11:38.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:38 smithi130 ceph-mon[98670]: pgmap v553: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-09-12T16:11:38.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:38 smithi159 ceph-mon[102522]: pgmap v553: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-09-12T16:11:38.507 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:38 smithi203 ceph-mon[103431]: pgmap v553: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-09-12T16:11:40.092 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 350.0 s, 518.3 tps, lat 190.201 ms stddev 161.434 2023-09-12T16:11:40.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:40 smithi203 ceph-mon[103431]: pgmap v554: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-09-12T16:11:40.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:40 smithi130 ceph-mon[98670]: pgmap v554: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-09-12T16:11:40.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:40 smithi159 ceph-mon[102522]: pgmap v554: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-09-12T16:11:42.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:42 smithi159 ceph-mon[102522]: pgmap v555: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 18 MiB/s wr, 1.38k op/s 2023-09-12T16:11:42.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:42 smithi130 ceph-mon[98670]: pgmap v555: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 18 MiB/s wr, 1.38k op/s 2023-09-12T16:11:42.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:42 smithi203 ceph-mon[103431]: pgmap v555: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 18 MiB/s wr, 1.38k op/s 2023-09-12T16:11:44.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:44 smithi130 ceph-mon[98670]: pgmap v556: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 11 MiB/s wr, 804 op/s 2023-09-12T16:11:44.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:44 smithi159 ceph-mon[102522]: pgmap v556: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 11 MiB/s wr, 804 op/s 2023-09-12T16:11:44.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:44 smithi203 ceph-mon[103431]: pgmap v556: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 11 MiB/s wr, 804 op/s 2023-09-12T16:11:45.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 355.0 s, 1473.3 tps, lat 70.013 ms stddev 65.250 2023-09-12T16:11:46.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:46 smithi130 ceph-mon[98670]: pgmap v557: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 16 MiB/s wr, 940 op/s 2023-09-12T16:11:46.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:46 smithi159 ceph-mon[102522]: pgmap v557: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 16 MiB/s wr, 940 op/s 2023-09-12T16:11:46.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:46 smithi203 ceph-mon[103431]: pgmap v557: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 16 MiB/s wr, 940 op/s 2023-09-12T16:11:48.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:48 smithi130 ceph-mon[98670]: pgmap v558: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 15 MiB/s wr, 727 op/s 2023-09-12T16:11:48.483 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:48 smithi159 ceph-mon[102522]: pgmap v558: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 15 MiB/s wr, 727 op/s 2023-09-12T16:11:48.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:48 smithi203 ceph-mon[103431]: pgmap v558: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 15 MiB/s wr, 727 op/s 2023-09-12T16:11:50.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 360.0 s, 797.1 tps, lat 120.240 ms stddev 194.462 2023-09-12T16:11:50.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:50 smithi159 ceph-mon[102522]: pgmap v559: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 810 op/s 2023-09-12T16:11:50.734 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:50 smithi130 ceph-mon[98670]: pgmap v559: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 810 op/s 2023-09-12T16:11:50.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:50 smithi203 ceph-mon[103431]: pgmap v559: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 810 op/s 2023-09-12T16:11:52.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:52 smithi159 ceph-mon[102522]: pgmap v560: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 24 MiB/s wr, 1.50k op/s 2023-09-12T16:11:52.733 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:52 smithi130 ceph-mon[98670]: pgmap v560: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 24 MiB/s wr, 1.50k op/s 2023-09-12T16:11:52.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:52 smithi203 ceph-mon[103431]: pgmap v560: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 24 MiB/s wr, 1.50k op/s 2023-09-12T16:11:54.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:54 smithi130 ceph-mon[98670]: pgmap v561: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 22 MiB/s wr, 1.40k op/s 2023-09-12T16:11:54.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:54 smithi159 ceph-mon[102522]: pgmap v561: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 22 MiB/s wr, 1.40k op/s 2023-09-12T16:11:54.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:54 smithi203 ceph-mon[103431]: pgmap v561: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 22 MiB/s wr, 1.40k op/s 2023-09-12T16:11:55.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 365.0 s, 1449.5 tps, lat 72.149 ms stddev 92.665 2023-09-12T16:11:56.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:56 smithi130 ceph-mon[98670]: pgmap v562: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 27 MiB/s wr, 1.77k op/s 2023-09-12T16:11:56.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:56 smithi159 ceph-mon[102522]: pgmap v562: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 27 MiB/s wr, 1.77k op/s 2023-09-12T16:11:56.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:56 smithi203 ceph-mon[103431]: pgmap v562: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 27 MiB/s wr, 1.77k op/s 2023-09-12T16:11:58.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:11:58 smithi159 ceph-mon[102522]: pgmap v563: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 1.68k op/s 2023-09-12T16:11:58.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:11:58 smithi130 ceph-mon[98670]: pgmap v563: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 1.68k op/s 2023-09-12T16:11:58.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:11:58 smithi203 ceph-mon[103431]: pgmap v563: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 1.68k op/s 2023-09-12T16:12:00.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 370.0 s, 1233.3 tps, lat 60.300 ms stddev 75.650 2023-09-12T16:12:00.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:00 smithi159 ceph-mon[102522]: pgmap v564: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 23 MiB/s wr, 1.59k op/s 2023-09-12T16:12:00.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:00 smithi130 ceph-mon[98670]: pgmap v564: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 23 MiB/s wr, 1.59k op/s 2023-09-12T16:12:00.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:00 smithi203 ceph-mon[103431]: pgmap v564: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 23 MiB/s wr, 1.59k op/s 2023-09-12T16:12:02.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:02 smithi130 ceph-mon[98670]: pgmap v565: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 31 MiB/s wr, 2.19k op/s 2023-09-12T16:12:02.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:02 smithi159 ceph-mon[102522]: pgmap v565: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 31 MiB/s wr, 2.19k op/s 2023-09-12T16:12:02.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:02 smithi203 ceph-mon[103431]: pgmap v565: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 31 MiB/s wr, 2.19k op/s 2023-09-12T16:12:04.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:04 smithi130 ceph-mon[98670]: pgmap v566: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 1.66k op/s 2023-09-12T16:12:04.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:04 smithi159 ceph-mon[102522]: pgmap v566: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 1.66k op/s 2023-09-12T16:12:04.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:04 smithi203 ceph-mon[103431]: pgmap v566: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 1.66k op/s 2023-09-12T16:12:05.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 375.0 s, 1131.5 tps, lat 105.544 ms stddev 327.955 2023-09-12T16:12:06.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:06 smithi159 ceph-mon[102522]: pgmap v567: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 26 MiB/s wr, 1.73k op/s 2023-09-12T16:12:06.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:06 smithi130 ceph-mon[98670]: pgmap v567: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 26 MiB/s wr, 1.73k op/s 2023-09-12T16:12:06.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:06 smithi203 ceph-mon[103431]: pgmap v567: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 26 MiB/s wr, 1.73k op/s 2023-09-12T16:12:07.728 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:12:08.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.065+0000 7f75dc691700 1 -- 172.21.15.130:0/4077063866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75d4077d40 msgr2=0x7f75d407a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:08.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.065+0000 7f75dc691700 1 --2- 172.21.15.130:0/4077063866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75d4077d40 0x7f75d407a1d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009f90 tx=0x7f75cc00b780 comp rx=0 tx=0).stop 2023-09-12T16:12:08.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.065+0000 7f75dc691700 1 -- 172.21.15.130:0/4077063866 shutdown_connections 2023-09-12T16:12:08.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.065+0000 7f75dc691700 1 --2- 172.21.15.130:0/4077063866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75d4077d40 0x7f75d407a1d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.065+0000 7f75dc691700 1 --2- 172.21.15.130:0/4077063866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d4072490 0x7f75d40728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.065+0000 7f75dc691700 1 --2- 172.21.15.130:0/4077063866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75d4075370 0x7f75d4077800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.065+0000 7f75dc691700 1 -- 172.21.15.130:0/4077063866 >> 172.21.15.130:0/4077063866 conn(0x7f75d406d3b0 msgr2=0x7f75d406f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:08.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.066+0000 7f75dc691700 1 -- 172.21.15.130:0/4077063866 shutdown_connections 2023-09-12T16:12:08.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.066+0000 7f75dc691700 1 -- 172.21.15.130:0/4077063866 wait complete. 2023-09-12T16:12:08.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.066+0000 7f75dc691700 1 Processor -- start 2023-09-12T16:12:08.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.066+0000 7f75dc691700 1 -- start start 2023-09-12T16:12:08.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75dc691700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d4072490 0x7f75d41b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75dc691700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75d4075370 0x7f75d41b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75dc691700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75d4077d40 0x7f75d41b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75dc691700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d41b76e0 con 0x7f75d4077d40 2023-09-12T16:12:08.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75dc691700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d41b7850 con 0x7f75d4075370 2023-09-12T16:12:08.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75dc691700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d41b7a10 con 0x7f75d4072490 2023-09-12T16:12:08.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75da42d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d4072490 0x7f75d41b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75da42d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d4072490 0x7f75d41b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35864/0 (socket says 172.21.15.130:35864) 2023-09-12T16:12:08.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75dac2e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75d4077d40 0x7f75d41b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75da42d700 1 -- 172.21.15.130:0/2171951883 learned_addr learned my addr 172.21.15.130:0/2171951883 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:08.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75d9c2c700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75d4075370 0x7f75d41b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75da42d700 1 -- 172.21.15.130:0/2171951883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75d4075370 msgr2=0x7f75d41b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:08.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75da42d700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75d4075370 0x7f75d41b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75da42d700 1 -- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75d4077d40 msgr2=0x7f75d41b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:08.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75da42d700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75d4077d40 0x7f75d41b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75da42d700 1 -- 172.21.15.130:0/2171951883 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc009c40 con 0x7f75d4072490 2023-09-12T16:12:08.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.067+0000 7f75dac2e700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75d4077d40 0x7f75d41b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:12:08.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.068+0000 7f75da42d700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d4072490 0x7f75d41b2540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f75d000e9c0 tx=0x7f75d000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:08.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.068+0000 7f75cb7fe700 1 -- 172.21.15.130:0/2171951883 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0004a30 con 0x7f75d4072490 2023-09-12T16:12:08.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.068+0000 7f75cb7fe700 1 -- 172.21.15.130:0/2171951883 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75d000cc80 con 0x7f75d4072490 2023-09-12T16:12:08.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.068+0000 7f75dc691700 1 -- 172.21.15.130:0/2171951883 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d41ae380 con 0x7f75d4072490 2023-09-12T16:12:08.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.068+0000 7f75cb7fe700 1 -- 172.21.15.130:0/2171951883 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d00054e0 con 0x7f75d4072490 2023-09-12T16:12:08.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.068+0000 7f75dc691700 1 -- 172.21.15.130:0/2171951883 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d41ae890 con 0x7f75d4072490 2023-09-12T16:12:08.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75cb7fe700 1 -- 172.21.15.130:0/2171951883 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f75d0004b90 con 0x7f75d4072490 2023-09-12T16:12:08.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75e8857b80 1 Processor -- start 2023-09-12T16:12:08.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75e8857b80 1 -- start start 2023-09-12T16:12:08.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 0x55f63e8e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2da0 0x55f63e8e1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75e8857b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e831730 con 0x55f63e8e1630 2023-09-12T16:12:08.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75e8857b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e82f050 con 0x55f63e8e2440 2023-09-12T16:12:08.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75e8857b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e82a940 con 0x55f63e8e2da0 2023-09-12T16:12:08.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.072+0000 7f75dac2e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 0x55f63e8e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75dac2e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 0x55f63e8e2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39516/0 (socket says 172.21.15.130:39516) 2023-09-12T16:12:08.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75dac2e700 1 -- 172.21.15.130:0/2616753601 learned_addr learned my addr 172.21.15.130:0/2616753601 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:08.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75d9c2c700 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75da42d700 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2da0 0x55f63e8e1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75cb7fe700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f75c00b1630 0x7f75c00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75dac2e700 1 -- 172.21.15.130:0/2616753601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2da0 msgr2=0x55f63e8e1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:08.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75dac2e700 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2da0 0x55f63e8e1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75dac2e700 1 -- 172.21.15.130:0/2616753601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 msgr2=0x55f63e8e1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:08.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75dac2e700 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8e1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75dac2e700 1 -- 172.21.15.130:0/2616753601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f63e656de0 con 0x55f63e8e2440 2023-09-12T16:12:08.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75d9c2c700 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8e1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:12:08.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75cb7fe700 1 -- 172.21.15.130:0/2171951883 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f75d0014070 con 0x7f75d4072490 2023-09-12T16:12:08.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75dac2e700 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 0x55f63e8e2860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009ac0 tx=0x7f75cc009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa0880f68076265b server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:08.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.073+0000 7f75dac2e700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f75c00b1630 0x7f75c00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.074+0000 7f75c8ff9700 1 -- 172.21.15.130:0/2616753601 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00e040 con 0x55f63e8e2440 2023-09-12T16:12:08.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.074+0000 7f75c8ff9700 1 -- 172.21.15.130:0/2616753601 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75cc0078e0 con 0x55f63e8e2440 2023-09-12T16:12:08.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.074+0000 7f75c8ff9700 1 -- 172.21.15.130:0/2616753601 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0059c0 con 0x55f63e8e2440 2023-09-12T16:12:08.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 -- 172.21.15.130:0/2616753601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 msgr2=0x55f63e8e2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:08.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75dac2e700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f75c00b1630 0x7f75c00b3af0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f75cc028000 tx=0x7f75cc01c5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:08.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 0x55f63e8e2860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009ac0 tx=0x7f75cc009af0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 -- 172.21.15.130:0/2616753601 shutdown_connections 2023-09-12T16:12:08.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8e1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 0x55f63e8e2860 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 --2- 172.21.15.130:0/2616753601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2da0 0x55f63e8e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 -- 172.21.15.130:0/2616753601 >> 172.21.15.130:0/2616753601 conn(0x55f63e84d090 msgr2=0x55f63e8d35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:08.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 -- 172.21.15.130:0/2616753601 shutdown_connections 2023-09-12T16:12:08.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 -- 172.21.15.130:0/2616753601 wait complete. 2023-09-12T16:12:08.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 -- start start 2023-09-12T16:12:08.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.075+0000 7f75e8857b80 1 Processor -- start 2023-09-12T16:12:08.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2440 0x55f63e8da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.088 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8df2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75e8857b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e7c8b70 con 0x55f63e8e1630 2023-09-12T16:12:08.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75e8857b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e7c8870 con 0x55f63e8e2da0 2023-09-12T16:12:08.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75e8857b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e6ac920 con 0x55f63e8e2440 2023-09-12T16:12:08.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75e8857b80 10 client.0 _ll_register_callbacks cb 0x55f63e712a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:12:08.089 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75da42d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75d9c2c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2440 0x55f63e8da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75da42d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8d9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56266/0 (socket says 172.21.15.130:56266) 2023-09-12T16:12:08.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75da42d700 1 -- 172.21.15.130:0/451280618 learned_addr learned my addr 172.21.15.130:0/451280618 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:08.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75dac2e700 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8df2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75dac2e700 1 -- 172.21.15.130:0/451280618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2440 msgr2=0x55f63e8da2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:08.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75dac2e700 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2440 0x55f63e8da2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75dac2e700 1 -- 172.21.15.130:0/451280618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 msgr2=0x55f63e8d9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:08.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75dac2e700 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8d9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75dac2e700 1 -- 172.21.15.130:0/451280618 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d407ac00 con 0x55f63e8e2da0 2023-09-12T16:12:08.091 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75dac2e700 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8df2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009790 tx=0x7f75cc020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25940bf01ee7ed38 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:08.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.076+0000 7f75bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:12:08.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75bdffb700 1 -- 172.21.15.130:0/451280618 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00e070 con 0x55f63e8e2da0 2023-09-12T16:12:08.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75e8857b80 1 -- 172.21.15.130:0/451280618 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f63e7824f0 con 0x55f63e8e2da0 2023-09-12T16:12:08.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75bdffb700 1 -- 172.21.15.130:0/451280618 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f75cc00be40 con 0x55f63e8e2da0 2023-09-12T16:12:08.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75bdffb700 1 -- 172.21.15.130:0/451280618 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75cc004bd0 con 0x55f63e8e2da0 2023-09-12T16:12:08.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75e8857b80 10 client.25532 fetch_fsmap learned FSMap version 32 2023-09-12T16:12:08.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75e8857b80 1 -- 172.21.15.130:0/451280618 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f63e63fc60 con 0x55f63e8e2da0 2023-09-12T16:12:08.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75bdffb700 1 -- 172.21.15.130:0/451280618 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f63e7824f0 con 0x55f63e8e2da0 2023-09-12T16:12:08.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75bdffb700 1 -- 172.21.15.130:0/451280618 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75cc03d070 con 0x55f63e8e2da0 2023-09-12T16:12:08.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75e8857b80 10 client.25532 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:12:08.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75e8857b80 10 client.25532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:12:08.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75e8857b80 20 client.25532 populate_metadata read hostname 'smithi130' 2023-09-12T16:12:08.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75e8857b80 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f63e8edf20 0x55f63e8f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75e8857b80 4 client.25532 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:12:08.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.077+0000 7f75e8857b80 1 -- 172.21.15.130:0/451280618 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f63e83f8a0 con 0x55f63e8edf20 2023-09-12T16:12:08.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.078+0000 7f75da42d700 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f63e8edf20 0x55f63e8f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.078+0000 7f75bdffb700 10 client.25532 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:12:08.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.078+0000 7f75da42d700 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f63e8edf20 0x55f63e8f0310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e905b896491fde84 server_cookie=d08e219f844dbd05 in_seq=0 out_seq=0 2023-09-12T16:12:08.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.078+0000 7f75bdffb700 1 -- 172.21.15.130:0/451280618 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f63e83f8a0 con 0x55f63e8edf20 2023-09-12T16:12:08.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.078+0000 7f75bdffb700 10 client.25532 handle_command_reply: tid=0 2023-09-12T16:12:08.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.078+0000 7f75e8857b80 1 client.25532 shutdown 2023-09-12T16:12:08.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 -- 172.21.15.130:0/451280618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 msgr2=0x55f63e8df2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:08.096 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8df2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009790 tx=0x7f75cc020000 comp rx=0 tx=0).stop 2023-09-12T16:12:08.096 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 -- 172.21.15.130:0/451280618 shutdown_connections 2023-09-12T16:12:08.096 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f63e8edf20 0x55f63e8f0310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.096 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8df2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.096 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75bdffb700 0 client.25532 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:12:08.097 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8d9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.097 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 --2- 172.21.15.130:0/451280618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2440 0x55f63e8da2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.097 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 -- 172.21.15.130:0/451280618 >> 172.21.15.130:0/451280618 conn(0x55f63e84d090 msgr2=0x55f63e8e5480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:08.097 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 -- 172.21.15.130:0/451280618 shutdown_connections 2023-09-12T16:12:08.097 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 1 -- 172.21.15.130:0/451280618 wait complete. 2023-09-12T16:12:08.098 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.079+0000 7f75e8857b80 20 client.25532 trim_cache size 0 max 16384 2023-09-12T16:12:08.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.114+0000 7f75e8857b80 1 Processor -- start 2023-09-12T16:12:08.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75e8857b80 1 -- start start 2023-09-12T16:12:08.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2440 0x55f63e8d3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8e7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75e8857b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e7824f0 con 0x55f63e8e1630 2023-09-12T16:12:08.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75e8857b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e8daf70 con 0x55f63e8e2da0 2023-09-12T16:12:08.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75e8857b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e8e0d10 con 0x55f63e8e2440 2023-09-12T16:12:08.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75dac2e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75dac2e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8e7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39534/0 (socket says 172.21.15.130:39534) 2023-09-12T16:12:08.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.118+0000 7f75dac2e700 1 -- 172.21.15.130:0/2642761622 learned_addr learned my addr 172.21.15.130:0/2642761622 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:08.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.119+0000 7f75dac2e700 1 -- 172.21.15.130:0/2642761622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2440 msgr2=0x55f63e8d3300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:12:08.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.119+0000 7f75dac2e700 1 --2- 172.21.15.130:0/2642761622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2440 0x55f63e8d3300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.119+0000 7f75dac2e700 1 -- 172.21.15.130:0/2642761622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 msgr2=0x55f63e8d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:08.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.119+0000 7f75dac2e700 1 --2- 172.21.15.130:0/2642761622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.119+0000 7f75dac2e700 1 -- 172.21.15.130:0/2642761622 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f63e67d250 con 0x55f63e8e2da0 2023-09-12T16:12:08.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.119+0000 7f75dac2e700 1 --2- 172.21.15.130:0/2642761622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8e7490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005270 tx=0x7f75cc00b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee3b2582a14e65b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:08.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.121+0000 7f75c97fa700 1 -- 172.21.15.130:0/2642761622 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00e030 con 0x55f63e8e2da0 2023-09-12T16:12:08.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.122+0000 7f75c97fa700 1 -- 172.21.15.130:0/2642761622 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75cc005360 con 0x55f63e8e2da0 2023-09-12T16:12:08.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.122+0000 7f75c97fa700 1 -- 172.21.15.130:0/2642761622 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc003c40 con 0x55f63e8e2da0 2023-09-12T16:12:08.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.122+0000 7f75e8857b80 1 -- 172.21.15.130:0/2642761622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 msgr2=0x55f63e8e7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:08.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.122+0000 7f75e8857b80 1 --2- 172.21.15.130:0/2642761622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8e7490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005270 tx=0x7f75cc00b780 comp rx=0 tx=0).stop 2023-09-12T16:12:08.128 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.124+0000 7f75e8857b80 1 -- 172.21.15.130:0/2642761622 shutdown_connections 2023-09-12T16:12:08.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.124+0000 7f75e8857b80 1 --2- 172.21.15.130:0/2642761622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2da0 0x55f63e8e7490 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.124+0000 7f75e8857b80 1 --2- 172.21.15.130:0/2642761622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e8d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.129 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.124+0000 7f75e8857b80 1 --2- 172.21.15.130:0/2642761622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2440 0x55f63e8d3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.124+0000 7f75e8857b80 1 -- 172.21.15.130:0/2642761622 >> 172.21.15.130:0/2642761622 conn(0x55f63e84d090 msgr2=0x55f63e8d9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:08.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.128+0000 7f75e8857b80 1 -- 172.21.15.130:0/2642761622 shutdown_connections 2023-09-12T16:12:08.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.128+0000 7f75e8857b80 1 -- 172.21.15.130:0/2642761622 wait complete. 2023-09-12T16:12:08.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.128+0000 7f75e8857b80 1 -- start start 2023-09-12T16:12:08.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.128+0000 7f75e8857b80 1 Processor -- start 2023-09-12T16:12:08.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e939ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 0x55f63e93a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75e8857b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2da0 0x55f63e9412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75e8857b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e941890 con 0x55f63e8e1630 2023-09-12T16:12:08.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75e8857b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e941a00 con 0x55f63e8e2440 2023-09-12T16:12:08.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75e8857b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e941bc0 con 0x55f63e8e2da0 2023-09-12T16:12:08.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75e8857b80 10 client.0 _ll_register_callbacks cb 0x7f75cc009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:12:08.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75da42d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e939ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75da42d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e939ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56278/0 (socket says 172.21.15.130:56278) 2023-09-12T16:12:08.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75da42d700 1 -- 172.21.15.130:0/1308166830 learned_addr learned my addr 172.21.15.130:0/1308166830 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:08.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75da42d700 1 -- 172.21.15.130:0/1308166830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2da0 msgr2=0x55f63e9412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:08.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75da42d700 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2da0 0x55f63e9412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75da42d700 1 -- 172.21.15.130:0/1308166830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 msgr2=0x55f63e93a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:08.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75da42d700 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 0x55f63e93a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.129+0000 7f75da42d700 1 -- 172.21.15.130:0/1308166830 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f63e63fc60 con 0x55f63e8e1630 2023-09-12T16:12:08.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.130+0000 7f75da42d700 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e939ee0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f75d0095e70 tx=0x7f75d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34575f6b57165090 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:08.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.130+0000 7f75bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:12:08.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.130+0000 7f75bdffb700 1 -- 172.21.15.130:0/1308166830 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0098d60 con 0x55f63e8e1630 2023-09-12T16:12:08.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.130+0000 7f75bdffb700 1 -- 172.21.15.130:0/1308166830 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f75d000cdf0 con 0x55f63e8e1630 2023-09-12T16:12:08.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.130+0000 7f75e8857b80 1 -- 172.21.15.130:0/1308166830 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f63e823920 con 0x55f63e8e1630 2023-09-12T16:12:08.139 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.131+0000 7f75bdffb700 1 -- 172.21.15.130:0/1308166830 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75d0099b00 con 0x55f63e8e1630 2023-09-12T16:12:08.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.136+0000 7f75e8857b80 10 client.16050 fetch_fsmap learned FSMap version 32 2023-09-12T16:12:08.140 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.136+0000 7f75e8857b80 1 -- 172.21.15.130:0/1308166830 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f63e6552b0 con 0x55f63e8e1630 2023-09-12T16:12:08.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.137+0000 7f75bdffb700 1 -- 172.21.15.130:0/1308166830 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f63e823920 con 0x55f63e8e1630 2023-09-12T16:12:08.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.137+0000 7f75bdffb700 1 -- 172.21.15.130:0/1308166830 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75d00acd20 con 0x55f63e8e1630 2023-09-12T16:12:08.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.137+0000 7f75e8857b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:12:08.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.137+0000 7f75e8857b80 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:12:08.141 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.137+0000 7f75e8857b80 20 client.16050 populate_metadata read hostname 'smithi130' 2023-09-12T16:12:08.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.137+0000 7f75e8857b80 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f63e8edf20 0x55f63e9455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:08.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.137+0000 7f75e8857b80 4 client.16050 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:12:08.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.137+0000 7f75e8857b80 1 -- 172.21.15.130:0/1308166830 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f63e7e40b0 con 0x55f63e8edf20 2023-09-12T16:12:08.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.137+0000 7f75d9c2c700 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f63e8edf20 0x55f63e9455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:08.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.138+0000 7f75bdffb700 10 client.16050 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:12:08.143 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.138+0000 7f75d9c2c700 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f63e8edf20 0x55f63e9455d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9e69a2a1dc10cc server_cookie=2142688098ac9c0f in_seq=0 out_seq=0 2023-09-12T16:12:08.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75bdffb700 1 -- 172.21.15.130:0/1308166830 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f63e7e40b0 con 0x55f63e8edf20 2023-09-12T16:12:08.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75bdffb700 10 client.16050 handle_command_reply: tid=0 2023-09-12T16:12:08.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75e8857b80 1 client.16050 shutdown 2023-09-12T16:12:08.144 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75e8857b80 1 -- 172.21.15.130:0/1308166830 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 msgr2=0x55f63e939ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:08.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75e8857b80 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e939ee0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f75d0095e70 tx=0x7f75d0000bc0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75e8857b80 1 -- 172.21.15.130:0/1308166830 shutdown_connections 2023-09-12T16:12:08.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75e8857b80 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f63e8edf20 0x55f63e9455d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75e8857b80 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f63e8e2da0 0x55f63e9412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75e8857b80 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f63e8e1630 0x55f63e939ee0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.145 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75e8857b80 1 --2- 172.21.15.130:0/1308166830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f63e8e2440 0x55f63e93a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75e8857b80 1 -- 172.21.15.130:0/1308166830 >> 172.21.15.130:0/1308166830 conn(0x55f63e84d090 msgr2=0x55f63e8d2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:08.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.140+0000 7f75bdffb700 0 client.16050 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:12:08.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.141+0000 7f75e8857b80 1 -- 172.21.15.130:0/1308166830 shutdown_connections 2023-09-12T16:12:08.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.141+0000 7f75e8857b80 1 -- 172.21.15.130:0/1308166830 wait complete. 2023-09-12T16:12:08.146 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.141+0000 7f75e8857b80 20 client.16050 trim_cache size 0 max 16384 2023-09-12T16:12:08.147 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:12:08.147 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:12:08.147 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:12:08.147 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:12:08.148 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:12:08.148 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:12:08.148 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:12:08.148 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:12:08.148 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:12:08.149 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:12:08.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.141+0000 7f75dc691700 1 -- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f75c00b1630 msgr2=0x7f75c00b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:08.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.141+0000 7f75dc691700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f75c00b1630 0x7f75c00b3af0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f75cc028000 tx=0x7f75cc01c5f0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.141+0000 7f75dc691700 1 -- 172.21.15.130:0/2171951883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d4072490 msgr2=0x7f75d41b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:08.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.141+0000 7f75dc691700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d4072490 0x7f75d41b2540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f75d000e9c0 tx=0x7f75d000ecd0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.147+0000 7f75dc691700 1 -- 172.21.15.130:0/2171951883 shutdown_connections 2023-09-12T16:12:08.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.147+0000 7f75dc691700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f75c00b1630 0x7f75c00b3af0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.147+0000 7f75dc691700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75d4077d40 0x7f75d41b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.147+0000 7f75dc691700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d4072490 0x7f75d41b2540 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.147+0000 7f75dc691700 1 --2- 172.21.15.130:0/2171951883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75d4075370 0x7f75d41b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:08.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.147+0000 7f75dc691700 1 -- 172.21.15.130:0/2171951883 >> 172.21.15.130:0/2171951883 conn(0x7f75d406d3b0 msgr2=0x7f75d40760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:08.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.148+0000 7f75dc691700 1 -- 172.21.15.130:0/2171951883 shutdown_connections 2023-09-12T16:12:08.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:08.148+0000 7f75dc691700 1 -- 172.21.15.130:0/2171951883 wait complete. 2023-09-12T16:12:08.166 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:08 smithi130 ceph-mon[98670]: pgmap v568: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 27 MiB/s wr, 1.71k op/s 2023-09-12T16:12:08.167 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:12:08.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:08 smithi159 ceph-mon[102522]: pgmap v568: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 27 MiB/s wr, 1.71k op/s 2023-09-12T16:12:08.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:08 smithi203 ceph-mon[103431]: pgmap v568: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 27 MiB/s wr, 1.71k op/s 2023-09-12T16:12:10.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 380.0 s, 439.0 tps, lat 235.556 ms stddev 262.314 2023-09-12T16:12:10.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:10 smithi159 ceph-mon[102522]: pgmap v569: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 23 MiB/s wr, 1.59k op/s 2023-09-12T16:12:10.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:10 smithi130 ceph-mon[98670]: pgmap v569: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 23 MiB/s wr, 1.59k op/s 2023-09-12T16:12:10.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:10 smithi203 ceph-mon[103431]: pgmap v569: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 23 MiB/s wr, 1.59k op/s 2023-09-12T16:12:12.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:12 smithi130 ceph-mon[98670]: pgmap v570: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 26 MiB/s wr, 1.84k op/s 2023-09-12T16:12:12.735 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:12 smithi159 ceph-mon[102522]: pgmap v570: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 26 MiB/s wr, 1.84k op/s 2023-09-12T16:12:12.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:12 smithi203 ceph-mon[103431]: pgmap v570: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 26 MiB/s wr, 1.84k op/s 2023-09-12T16:12:14.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:14 smithi130 ceph-mon[98670]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2023-09-12T16:12:14.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:14 smithi159 ceph-mon[102522]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2023-09-12T16:12:14.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:14 smithi203 ceph-mon[103431]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2023-09-12T16:12:15.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 385.0 s, 1361.0 tps, lat 74.476 ms stddev 75.614 2023-09-12T16:12:16.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:16 smithi130 ceph-mon[98670]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 16 MiB/s wr, 975 op/s 2023-09-12T16:12:16.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:16 smithi159 ceph-mon[102522]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 16 MiB/s wr, 975 op/s 2023-09-12T16:12:16.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:16 smithi203 ceph-mon[103431]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 16 MiB/s wr, 975 op/s 2023-09-12T16:12:18.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:18 smithi130 ceph-mon[98670]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 20 MiB/s wr, 1.04k op/s 2023-09-12T16:12:18.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:18 smithi159 ceph-mon[102522]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 20 MiB/s wr, 1.04k op/s 2023-09-12T16:12:18.505 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:18 smithi203 ceph-mon[103431]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 20 MiB/s wr, 1.04k op/s 2023-09-12T16:12:20.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 390.0 s, 1110.8 tps, lat 83.262 ms stddev 165.589 2023-09-12T16:12:20.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:20 smithi130 ceph-mon[98670]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 649 op/s 2023-09-12T16:12:20.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:20 smithi159 ceph-mon[102522]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 649 op/s 2023-09-12T16:12:20.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:20 smithi203 ceph-mon[103431]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 649 op/s 2023-09-12T16:12:22.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:22 smithi130 ceph-mon[98670]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 24 MiB/s wr, 1.57k op/s 2023-09-12T16:12:22.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:22 smithi159 ceph-mon[102522]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 24 MiB/s wr, 1.57k op/s 2023-09-12T16:12:22.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:22 smithi203 ceph-mon[103431]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 24 MiB/s wr, 1.57k op/s 2023-09-12T16:12:24.735 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:24 smithi159 ceph-mon[102522]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 1.67k op/s 2023-09-12T16:12:24.738 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:24 smithi130 ceph-mon[98670]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 1.67k op/s 2023-09-12T16:12:24.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:24 smithi203 ceph-mon[103431]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 1.67k op/s 2023-09-12T16:12:25.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 395.0 s, 552.8 tps, lat 194.658 ms stddev 291.067 2023-09-12T16:12:26.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:26 smithi130 ceph-mon[98670]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 1.97k op/s 2023-09-12T16:12:26.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:26 smithi159 ceph-mon[102522]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 1.97k op/s 2023-09-12T16:12:26.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:26 smithi203 ceph-mon[103431]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 1.97k op/s 2023-09-12T16:12:28.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:28 smithi130 ceph-mon[98670]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 35 MiB/s wr, 2.78k op/s 2023-09-12T16:12:28.479 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:28 smithi159 ceph-mon[102522]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 35 MiB/s wr, 2.78k op/s 2023-09-12T16:12:28.504 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:28 smithi203 ceph-mon[103431]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 35 MiB/s wr, 2.78k op/s 2023-09-12T16:12:29.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:29 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:12:29.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:29 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:12:29.500 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:29 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:12:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 400.0 s, 1465.3 tps, lat 67.194 ms stddev 61.020 2023-09-12T16:12:30.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:30 smithi130 ceph-mon[98670]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 28 MiB/s wr, 2.53k op/s 2023-09-12T16:12:30.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:30 smithi159 ceph-mon[102522]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 28 MiB/s wr, 2.53k op/s 2023-09-12T16:12:30.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:30 smithi203 ceph-mon[103431]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 28 MiB/s wr, 2.53k op/s 2023-09-12T16:12:31.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:12:31.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:12:31.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:12:31.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:12:31.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:12:31.477 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:12:31.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:12:31.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:12:31.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:12:32.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:32 smithi130 ceph-mon[98670]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 34 MiB/s wr, 2.87k op/s 2023-09-12T16:12:32.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:32 smithi159 ceph-mon[102522]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 34 MiB/s wr, 2.87k op/s 2023-09-12T16:12:32.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:32 smithi203 ceph-mon[103431]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 34 MiB/s wr, 2.87k op/s 2023-09-12T16:12:34.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:34 smithi159 ceph-mon[102522]: pgmap v581: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.06k op/s 2023-09-12T16:12:34.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:34 smithi130 ceph-mon[98670]: pgmap v581: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.06k op/s 2023-09-12T16:12:34.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:34 smithi203 ceph-mon[103431]: pgmap v581: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.06k op/s 2023-09-12T16:12:35.094 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 405.0 s, 1663.0 tps, lat 59.536 ms stddev 61.118 2023-09-12T16:12:36.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:36 smithi130 ceph-mon[98670]: pgmap v582: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 25 MiB/s wr, 1.81k op/s 2023-09-12T16:12:36.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:36 smithi159 ceph-mon[102522]: pgmap v582: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 25 MiB/s wr, 1.81k op/s 2023-09-12T16:12:36.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:36 smithi203 ceph-mon[103431]: pgmap v582: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 25 MiB/s wr, 1.81k op/s 2023-09-12T16:12:38.168 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:12:38.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:38 smithi159 ceph-mon[102522]: pgmap v583: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 32 MiB/s wr, 2.18k op/s 2023-09-12T16:12:38.494 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:38 smithi130 ceph-mon[98670]: pgmap v583: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 32 MiB/s wr, 2.18k op/s 2023-09-12T16:12:38.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:38 smithi203 ceph-mon[103431]: pgmap v583: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 32 MiB/s wr, 2.18k op/s 2023-09-12T16:12:38.580 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.580+0000 7fdd22dd1700 1 -- 172.21.15.130:0/2370226682 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 msgr2=0x7fdd1c10c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:38.581 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.580+0000 7fdd22dd1700 1 --2- 172.21.15.130:0/2370226682 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 0x7fdd1c10c400 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0098f0 tx=0x7fdd0c009c00 comp rx=0 tx=0).stop 2023-09-12T16:12:38.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.594+0000 7fdd22dd1700 1 -- 172.21.15.130:0/2370226682 shutdown_connections 2023-09-12T16:12:38.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.594+0000 7fdd22dd1700 1 --2- 172.21.15.130:0/2370226682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1c10c940 0x7fdd1c10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.594+0000 7fdd22dd1700 1 --2- 172.21.15.130:0/2370226682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd1c107600 0x7fdd1c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.594+0000 7fdd22dd1700 1 --2- 172.21.15.130:0/2370226682 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 0x7fdd1c10c400 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.596 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.594+0000 7fdd22dd1700 1 -- 172.21.15.130:0/2370226682 >> 172.21.15.130:0/2370226682 conn(0x7fdd1c06d3b0 msgr2=0x7fdd1c06f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:38.598 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.599+0000 7fdd22dd1700 1 -- 172.21.15.130:0/2370226682 shutdown_connections 2023-09-12T16:12:38.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.606+0000 7fdd22dd1700 1 -- 172.21.15.130:0/2370226682 wait complete. 2023-09-12T16:12:38.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.607+0000 7fdd22dd1700 1 Processor -- start 2023-09-12T16:12:38.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.618+0000 7fdd22dd1700 1 -- start start 2023-09-12T16:12:38.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.618+0000 7fdd22dd1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1c107600 0x7fdd1c1a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.618+0000 7fdd22dd1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 0x7fdd1c1a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.618+0000 7fdd22dd1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd1c10c940 0x7fdd1c1ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.618+0000 7fdd22dd1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c1accd0 con 0x7fdd1c109f70 2023-09-12T16:12:38.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.618+0000 7fdd22dd1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c10ed30 con 0x7fdd1c107600 2023-09-12T16:12:38.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.618+0000 7fdd22dd1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c10eec0 con 0x7fdd1c10c940 2023-09-12T16:12:38.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.628+0000 7fdd1bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 0x7fdd1c1a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.628+0000 7fdd1bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 0x7fdd1c1a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56296/0 (socket says 172.21.15.130:56296) 2023-09-12T16:12:38.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.628+0000 7fdd1bfff700 1 -- 172.21.15.130:0/3137599449 learned_addr learned my addr 172.21.15.130:0/3137599449 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:38.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.631+0000 7fdd2136e700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd1c10c940 0x7fdd1c1ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.636+0000 7fdd20b6d700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1c107600 0x7fdd1c1a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.650+0000 7fdd1bfff700 1 -- 172.21.15.130:0/3137599449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd1c10c940 msgr2=0x7fdd1c1ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:38.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.650+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd1c10c940 0x7fdd1c1ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.650+0000 7fdd1bfff700 1 -- 172.21.15.130:0/3137599449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1c107600 msgr2=0x7fdd1c1a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:38.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.650+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1c107600 0x7fdd1c1a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.650+0000 7fdd1bfff700 1 -- 172.21.15.130:0/3137599449 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c009520 con 0x7fdd1c109f70 2023-09-12T16:12:38.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.655+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 0x7fdd1c1a5690 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009230 tx=0x7fdd0c00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:38.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.660+0000 7fdd19ffb700 1 -- 172.21.15.130:0/3137599449 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c01c070 con 0x7fdd1c109f70 2023-09-12T16:12:38.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.660+0000 7fdd19ffb700 1 -- 172.21.15.130:0/3137599449 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd0c00bba0 con 0x7fdd1c109f70 2023-09-12T16:12:38.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.660+0000 7fdd19ffb700 1 -- 172.21.15.130:0/3137599449 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c004da0 con 0x7fdd1c109f70 2023-09-12T16:12:38.660 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.660+0000 7fdd22dd1700 1 -- 172.21.15.130:0/3137599449 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c10f140 con 0x7fdd1c109f70 2023-09-12T16:12:38.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.661+0000 7fdd22dd1700 1 -- 172.21.15.130:0/3137599449 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c10f5a0 con 0x7fdd1c109f70 2023-09-12T16:12:38.676 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.677+0000 7fdd2ef97b80 1 Processor -- start 2023-09-12T16:12:38.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.678+0000 7fdd19ffb700 1 -- 172.21.15.130:0/3137599449 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fdd0c017400 con 0x7fdd1c109f70 2023-09-12T16:12:38.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.679+0000 7fdd19ffb700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdd040a0840 0x7fdd040a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.679+0000 7fdd19ffb700 1 -- 172.21.15.130:0/3137599449 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdd0c0cd0e0 con 0x7fdd1c109f70 2023-09-12T16:12:38.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.680+0000 7fdd2ef97b80 1 -- start start 2023-09-12T16:12:38.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.680+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18ba80 0x5568dc18bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.680+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18c3e0 0x5568dc18ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.680+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc18d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.680 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.680+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc0e4730 con 0x5568dc18c3e0 2023-09-12T16:12:38.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.680+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc0e2050 con 0x5568dc18ba80 2023-09-12T16:12:38.681 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.680+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc0dd940 con 0x5568dc18d010 2023-09-12T16:12:38.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.682+0000 7fdd1bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18c3e0 0x5568dc18ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.682+0000 7fdd1bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18c3e0 0x5568dc18ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56304/0 (socket says 172.21.15.130:56304) 2023-09-12T16:12:38.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.682+0000 7fdd1bfff700 1 -- 172.21.15.130:0/1095970560 learned_addr learned my addr 172.21.15.130:0/1095970560 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:38.682 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.682+0000 7fdd2136e700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdd040a0840 0x7fdd040a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.683+0000 7fdd1bfff700 1 -- 172.21.15.130:0/1095970560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 msgr2=0x5568dc18d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:38.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.683+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/1095970560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc18d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.683 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.683+0000 7fdd1bfff700 1 -- 172.21.15.130:0/1095970560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18ba80 msgr2=0x5568dc18bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:38.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.683+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/1095970560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18ba80 0x5568dc18bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.683+0000 7fdd1bfff700 1 -- 172.21.15.130:0/1095970560 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568dbf09de0 con 0x5568dc18c3e0 2023-09-12T16:12:38.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.683+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/1095970560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18c3e0 0x5568dc18ca10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c097000 tx=0x7fdd0c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdeff2f830b4355a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:38.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.684+0000 7fdd2136e700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdd040a0840 0x7fdd040a2d00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdd14007fd0 tx=0x7fdd14007610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:38.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.689+0000 7fdd22dd1700 1 -- 172.21.15.130:0/1095970560 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c09a590 con 0x5568dc18c3e0 2023-09-12T16:12:38.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.689+0000 7fdd22dd1700 1 -- 172.21.15.130:0/1095970560 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd0c09e3c0 con 0x5568dc18c3e0 2023-09-12T16:12:38.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.689+0000 7fdd22dd1700 1 -- 172.21.15.130:0/1095970560 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c09a7f0 con 0x5568dc18c3e0 2023-09-12T16:12:38.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.696+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/1095970560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18c3e0 msgr2=0x5568dc18ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:38.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.696+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/1095970560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18c3e0 0x5568dc18ca10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c097000 tx=0x7fdd0c004840 comp rx=0 tx=0).stop 2023-09-12T16:12:38.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.707+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/1095970560 shutdown_connections 2023-09-12T16:12:38.706 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.707+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/1095970560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc18d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.707+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/1095970560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18ba80 0x5568dc18bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.707+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/1095970560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18c3e0 0x5568dc18ca10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.707 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.707+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/1095970560 >> 172.21.15.130:0/1095970560 conn(0x5568dc100090 msgr2=0x5568dc186750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:38.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.710+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/1095970560 shutdown_connections 2023-09-12T16:12:38.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.710+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/1095970560 wait complete. 2023-09-12T16:12:38.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.710+0000 7fdd2ef97b80 1 -- start start 2023-09-12T16:12:38.710 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.710+0000 7fdd2ef97b80 1 Processor -- start 2023-09-12T16:12:38.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.711+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.711+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18c3e0 0x5568dc195800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.711+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18d010 0x5568dc19c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.711 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.711+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc07bb70 con 0x5568dc18ba80 2023-09-12T16:12:38.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.711+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc07b870 con 0x5568dc18d010 2023-09-12T16:12:38.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.711+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dbf5f920 con 0x5568dc18c3e0 2023-09-12T16:12:38.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.711+0000 7fdd2ef97b80 10 client.0 _ll_register_callbacks cb 0x5568dbfc5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:12:38.715 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.716+0000 7fdd2136e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18d010 0x5568dc19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.716+0000 7fdd20b6d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.716+0000 7fdd20b6d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56308/0 (socket says 172.21.15.130:56308) 2023-09-12T16:12:38.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.716+0000 7fdd20b6d700 1 -- 172.21.15.130:0/3326490352 learned_addr learned my addr 172.21.15.130:0/3326490352 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:38.716 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.716+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18c3e0 0x5568dc195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.733+0000 7fdd20b6d700 1 -- 172.21.15.130:0/3326490352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18c3e0 msgr2=0x5568dc195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:38.738 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.733+0000 7fdd20b6d700 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18c3e0 0x5568dc195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.733+0000 7fdd20b6d700 1 -- 172.21.15.130:0/3326490352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18d010 msgr2=0x5568dc19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:38.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.733+0000 7fdd20b6d700 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18d010 0x5568dc19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.733+0000 7fdd20b6d700 1 -- 172.21.15.130:0/3326490352 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c10f5a0 con 0x5568dc18ba80 2023-09-12T16:12:38.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.739+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18c3e0 0x5568dc195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:12:38.739 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.739+0000 7fdd2136e700 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18d010 0x5568dc19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:12:38.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.743+0000 7fdd20b6d700 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1952c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdd10004790 tx=0x7fdd1000bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6677e55afa7d31 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:38.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.743+0000 7fdcfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:12:38.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.751+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3326490352 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000be50 con 0x5568dc18ba80 2023-09-12T16:12:38.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.751+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3326490352 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdd1001a470 con 0x5568dc18ba80 2023-09-12T16:12:38.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.751+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3326490352 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568dc0354f0 con 0x5568dc18ba80 2023-09-12T16:12:38.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.754+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3326490352 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd1001a630 con 0x5568dc18ba80 2023-09-12T16:12:38.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.754+0000 7fdd2ef97b80 10 client.16062 fetch_fsmap learned FSMap version 32 2023-09-12T16:12:38.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.754+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3326490352 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5568dbef2c60 con 0x5568dc18ba80 2023-09-12T16:12:38.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.757+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3326490352 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568dc0354f0 con 0x5568dc18ba80 2023-09-12T16:12:38.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.757+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3326490352 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdd10011900 con 0x5568dc18ba80 2023-09-12T16:12:38.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.757+0000 7fdd2ef97b80 10 client.16062 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:12:38.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.757+0000 7fdd2ef97b80 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:12:38.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.757+0000 7fdd2ef97b80 20 client.16062 populate_metadata read hostname 'smithi130' 2023-09-12T16:12:38.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.757+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5568dc1a1120 0x5568dc1a3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.757+0000 7fdd2ef97b80 4 client.16062 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:12:38.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.757+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3326490352 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568dc0f28a0 con 0x5568dc1a1120 2023-09-12T16:12:38.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.760+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5568dc1a1120 0x5568dc1a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.787+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5568dc1a1120 0x5568dc1a3540 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536d8c63ac82db6f server_cookie=6d359ffe82613355 in_seq=0 out_seq=0 2023-09-12T16:12:38.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.788+0000 7fdcfbfff700 10 client.16062 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:12:38.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.799+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3326490352 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5568dc0f28a0 con 0x5568dc1a1120 2023-09-12T16:12:38.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.799+0000 7fdcfbfff700 10 client.16062 handle_command_reply: tid=0 2023-09-12T16:12:38.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.800+0000 7fdd2ef97b80 1 client.16062 shutdown 2023-09-12T16:12:38.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.803+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3326490352 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 msgr2=0x5568dc1952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:38.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.803+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1952c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdd10004790 tx=0x7fdd1000bc70 comp rx=0 tx=0).stop 2023-09-12T16:12:38.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.822+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3326490352 shutdown_connections 2023-09-12T16:12:38.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.822+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5568dc1a1120 0x5568dc1a3540 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.822+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18d010 0x5568dc19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.822+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1952c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.822+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3326490352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18c3e0 0x5568dc195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.822+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3326490352 >> 172.21.15.130:0/3326490352 conn(0x5568dc100090 msgr2=0x5568dc185df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:38.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.822+0000 7fdcfbfff700 0 client.16062 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:12:38.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.832+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3326490352 shutdown_connections 2023-09-12T16:12:38.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.839+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3326490352 wait complete. 2023-09-12T16:12:38.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.863+0000 7fdd2ef97b80 20 client.16062 trim_cache size 0 max 16384 2023-09-12T16:12:38.898 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.895+0000 7fdd2ef97b80 1 Processor -- start 2023-09-12T16:12:38.904 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.900+0000 7fdd2ef97b80 1 -- start start 2023-09-12T16:12:38.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.900+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.900+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 0x5568dc19c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.900+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc18ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.900+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc187530 con 0x5568dc18ba80 2023-09-12T16:12:38.905 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.900+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc18d8a0 con 0x5568dc18c3e0 2023-09-12T16:12:38.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.900+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc180ef0 con 0x5568dc18d010 2023-09-12T16:12:38.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd20b6d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd20b6d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1933f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56316/0 (socket says 172.21.15.130:56316) 2023-09-12T16:12:38.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd20b6d700 1 -- 172.21.15.130:0/4010532362 learned_addr learned my addr 172.21.15.130:0/4010532362 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:38.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd2136e700 1 --2- 172.21.15.130:0/4010532362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc18ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd2136e700 1 -- 172.21.15.130:0/4010532362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 msgr2=0x5568dc19c800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:12:38.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd2136e700 1 --2- 172.21.15.130:0/4010532362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 0x5568dc19c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd2136e700 1 -- 172.21.15.130:0/4010532362 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 msgr2=0x5568dc1933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:38.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd2136e700 1 --2- 172.21.15.130:0/4010532362 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd2136e700 1 -- 172.21.15.130:0/4010532362 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568dbf30250 con 0x5568dc18d010 2023-09-12T16:12:38.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd2136e700 1 --2- 172.21.15.130:0/4010532362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc18ff50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009050 tx=0x7fdd14013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea9d48764da7f6d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:38.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.901+0000 7fdd02ffd700 1 -- 172.21.15.130:0/4010532362 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd14003a00 con 0x5568dc18d010 2023-09-12T16:12:38.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.902+0000 7fdd02ffd700 1 -- 172.21.15.130:0/4010532362 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd1401e040 con 0x5568dc18d010 2023-09-12T16:12:38.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.902+0000 7fdd02ffd700 1 -- 172.21.15.130:0/4010532362 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd14004440 con 0x5568dc18d010 2023-09-12T16:12:38.908 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.903+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/4010532362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 msgr2=0x5568dc18ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:38.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.903+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/4010532362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc18ff50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009050 tx=0x7fdd14013000 comp rx=0 tx=0).stop 2023-09-12T16:12:38.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.914+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/4010532362 shutdown_connections 2023-09-12T16:12:38.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.914+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/4010532362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc18ff50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.915 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.914+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/4010532362 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc1933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.914+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/4010532362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 0x5568dc19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.916 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.914+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/4010532362 >> 172.21.15.130:0/4010532362 conn(0x5568dc100090 msgr2=0x5568dc1a3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:38.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.925+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/4010532362 shutdown_connections 2023-09-12T16:12:38.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.927+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/4010532362 wait complete. 2023-09-12T16:12:38.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.928+0000 7fdd2ef97b80 1 -- start start 2023-09-12T16:12:38.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.928+0000 7fdd2ef97b80 1 Processor -- start 2023-09-12T16:12:38.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.928+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.928+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 0x5568dc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.928+0000 7fdd2ef97b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.928+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc19c0a0 con 0x5568dc18ba80 2023-09-12T16:12:38.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.928+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc19c210 con 0x5568dc18c3e0 2023-09-12T16:12:38.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.928+0000 7fdd2ef97b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568dc19c3d0 con 0x5568dc18d010 2023-09-12T16:12:38.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.928+0000 7fdd2ef97b80 10 client.0 _ll_register_callbacks cb 0x7fdd100043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:12:38.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.929+0000 7fdd2136e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.929+0000 7fdd2136e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc19bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35932/0 (socket says 172.21.15.130:35932) 2023-09-12T16:12:38.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.929+0000 7fdd2136e700 1 -- 172.21.15.130:0/3259397119 learned_addr learned my addr 172.21.15.130:0/3259397119 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:12:38.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.929+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 0x5568dc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.929+0000 7fdd1bfff700 1 -- 172.21.15.130:0/3259397119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 msgr2=0x5568dc19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:12:38.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.929+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.929+0000 7fdd1bfff700 1 -- 172.21.15.130:0/3259397119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 msgr2=0x5568dc19a610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:12:38.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.929+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc19a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:38.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.929+0000 7fdd1bfff700 1 -- 172.21.15.130:0/3259397119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd14012040 con 0x5568dc18c3e0 2023-09-12T16:12:38.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.935+0000 7fdd1bfff700 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 0x5568dc19ab50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c097000 tx=0x7fdd0c00bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc98cb65f47d64d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:12:38.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.935+0000 7fdcfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:12:38.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.947+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3259397119 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c09a590 con 0x5568dc18c3e0 2023-09-12T16:12:38.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.947+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3259397119 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdd0c0b2040 con 0x5568dc18c3e0 2023-09-12T16:12:38.948 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.947+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3259397119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd0c09e3c0 con 0x5568dc18c3e0 2023-09-12T16:12:38.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.947+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3259397119 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd0c09e170 con 0x5568dc18c3e0 2023-09-12T16:12:38.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.947+0000 7fdd2ef97b80 10 client.25544 fetch_fsmap learned FSMap version 32 2023-09-12T16:12:38.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.947+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3259397119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5568dbf082b0 con 0x5568dc18c3e0 2023-09-12T16:12:38.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.947+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3259397119 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd0c0a6070 con 0x5568dc18c3e0 2023-09-12T16:12:38.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.948+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3259397119 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdd0c0a2170 con 0x5568dc18c3e0 2023-09-12T16:12:38.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.948+0000 7fdd2ef97b80 10 client.25544 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:12:38.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.948+0000 7fdd2ef97b80 10 client.25544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:12:38.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.948+0000 7fdd2ef97b80 20 client.25544 populate_metadata read hostname 'smithi130' 2023-09-12T16:12:38.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.948+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5568dc1f5a50 0x5568dc1f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:12:38.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.948+0000 7fdd2ef97b80 4 client.25544 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:12:38.951 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.948+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3259397119 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5568dc0d6920 con 0x5568dc1f5a50 2023-09-12T16:12:38.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.957+0000 7fdd20b6d700 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5568dc1f5a50 0x5568dc1f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:12:38.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.971+0000 7fdd20b6d700 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5568dc1f5a50 0x5568dc1f7e40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f991f1cfd2f3772 server_cookie=6d313afb416b35cd in_seq=0 out_seq=0 2023-09-12T16:12:38.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.971+0000 7fdcfbfff700 10 client.25544 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:12:38.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.971+0000 7fdcfbfff700 1 -- 172.21.15.130:0/3259397119 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5568dc0d6920 con 0x5568dc1f5a50 2023-09-12T16:12:38.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.971+0000 7fdcfbfff700 10 client.25544 handle_command_reply: tid=0 2023-09-12T16:12:38.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.971+0000 7fdd2ef97b80 1 client.25544 shutdown 2023-09-12T16:12:38.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.995+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3259397119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 msgr2=0x5568dc19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:38.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:38.995+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 0x5568dc19ab50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c097000 tx=0x7fdd0c00bfa0 comp rx=0 tx=0).stop 2023-09-12T16:12:39.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.006+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3259397119 shutdown_connections 2023-09-12T16:12:39.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.006+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5568dc1f5a50 0x5568dc1f7e40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:39.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.006+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5568dc18d010 0x5568dc19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:39.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.006+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568dc18ba80 0x5568dc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:39.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.006+0000 7fdd2ef97b80 1 --2- 172.21.15.130:0/3259397119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568dc18c3e0 0x5568dc19ab50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:39.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.006+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3259397119 >> 172.21.15.130:0/3259397119 conn(0x5568dc100090 msgr2=0x5568dc1933f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:39.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.006+0000 7fdcfbfff700 0 client.25544 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:12:39.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.021+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3259397119 shutdown_connections 2023-09-12T16:12:39.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.022+0000 7fdd2ef97b80 1 -- 172.21.15.130:0/3259397119 wait complete. 2023-09-12T16:12:39.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.023+0000 7fdd2ef97b80 20 client.25544 trim_cache size 0 max 16384 2023-09-12T16:12:39.034 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:12:39.034 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:12:39.034 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:12:39.035 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:12:39.035 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:12:39.035 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:12:39.035 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:12:39.035 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:12:39.035 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:12:39.036 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:12:39.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.041+0000 7fdd037fe700 1 -- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdd040a0840 msgr2=0x7fdd040a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:39.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.041+0000 7fdd037fe700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdd040a0840 0x7fdd040a2d00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdd14007fd0 tx=0x7fdd14007610 comp rx=0 tx=0).stop 2023-09-12T16:12:39.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.041+0000 7fdd037fe700 1 -- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 msgr2=0x7fdd1c1a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:12:39.045 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.041+0000 7fdd037fe700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 0x7fdd1c1a5690 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009230 tx=0x7fdd0c00b820 comp rx=0 tx=0).stop 2023-09-12T16:12:39.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.047+0000 7fdd037fe700 1 -- 172.21.15.130:0/3137599449 shutdown_connections 2023-09-12T16:12:39.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.047+0000 7fdd037fe700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdd040a0840 0x7fdd040a2d00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:39.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.047+0000 7fdd037fe700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd1c10c940 0x7fdd1c1ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:39.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.047+0000 7fdd037fe700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1c107600 0x7fdd1c1a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:39.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.047+0000 7fdd037fe700 1 --2- 172.21.15.130:0/3137599449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd1c109f70 0x7fdd1c1a5690 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:12:39.053 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.047+0000 7fdd037fe700 1 -- 172.21.15.130:0/3137599449 >> 172.21.15.130:0/3137599449 conn(0x7fdd1c06d3b0 msgr2=0x7fdd1c108290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:12:39.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.055+0000 7fdd037fe700 1 -- 172.21.15.130:0/3137599449 shutdown_connections 2023-09-12T16:12:39.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:12:39.058+0000 7fdd037fe700 1 -- 172.21.15.130:0/3137599449 wait complete. 2023-09-12T16:12:39.084 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:12:40.092 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 410.0 s, 366.1 tps, lat 271.079 ms stddev 384.360 2023-09-12T16:12:40.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:40 smithi130 ceph-mon[98670]: pgmap v584: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 22 MiB/s wr, 1.28k op/s 2023-09-12T16:12:40.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:40 smithi159 ceph-mon[102522]: pgmap v584: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 22 MiB/s wr, 1.28k op/s 2023-09-12T16:12:40.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:40 smithi203 ceph-mon[103431]: pgmap v584: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 22 MiB/s wr, 1.28k op/s 2023-09-12T16:12:42.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:42 smithi159 ceph-mon[102522]: pgmap v585: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 26 MiB/s wr, 1.63k op/s 2023-09-12T16:12:42.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:42 smithi130 ceph-mon[98670]: pgmap v585: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 26 MiB/s wr, 1.63k op/s 2023-09-12T16:12:42.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:42 smithi203 ceph-mon[103431]: pgmap v585: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 26 MiB/s wr, 1.63k op/s 2023-09-12T16:12:44.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:44 smithi130 ceph-mon[98670]: pgmap v586: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 22 MiB/s wr, 1.52k op/s 2023-09-12T16:12:44.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:44 smithi203 ceph-mon[103431]: pgmap v586: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 22 MiB/s wr, 1.52k op/s 2023-09-12T16:12:44.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:44 smithi159 ceph-mon[102522]: pgmap v586: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 22 MiB/s wr, 1.52k op/s 2023-09-12T16:12:45.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 415.0 s, 542.4 tps, lat 189.427 ms stddev 179.991 2023-09-12T16:12:46.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:46 smithi203 ceph-mon[103431]: pgmap v587: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-09-12T16:12:46.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:46 smithi130 ceph-mon[98670]: pgmap v587: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-09-12T16:12:46.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:46 smithi159 ceph-mon[102522]: pgmap v587: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-09-12T16:12:48.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:48 smithi159 ceph-mon[102522]: pgmap v588: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-09-12T16:12:48.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:48 smithi130 ceph-mon[98670]: pgmap v588: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-09-12T16:12:48.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:48 smithi203 ceph-mon[103431]: pgmap v588: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-09-12T16:12:50.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 420.0 s, 1380.4 tps, lat 73.783 ms stddev 70.237 2023-09-12T16:12:50.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:50 smithi130 ceph-mon[98670]: pgmap v589: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 12 MiB/s wr, 888 op/s 2023-09-12T16:12:50.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:50 smithi159 ceph-mon[102522]: pgmap v589: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 12 MiB/s wr, 888 op/s 2023-09-12T16:12:50.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:50 smithi203 ceph-mon[103431]: pgmap v589: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 12 MiB/s wr, 888 op/s 2023-09-12T16:12:52.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:52 smithi159 ceph-mon[102522]: pgmap v590: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 16 MiB/s wr, 1.04k op/s 2023-09-12T16:12:52.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:52 smithi130 ceph-mon[98670]: pgmap v590: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 16 MiB/s wr, 1.04k op/s 2023-09-12T16:12:52.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:52 smithi203 ceph-mon[103431]: pgmap v590: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 16 MiB/s wr, 1.04k op/s 2023-09-12T16:12:54.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:54 smithi130 ceph-mon[98670]: pgmap v591: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 17 MiB/s wr, 881 op/s 2023-09-12T16:12:54.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:54 smithi159 ceph-mon[102522]: pgmap v591: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 17 MiB/s wr, 881 op/s 2023-09-12T16:12:54.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:54 smithi203 ceph-mon[103431]: pgmap v591: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 17 MiB/s wr, 881 op/s 2023-09-12T16:12:55.093 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 425.0 s, 920.4 tps, lat 105.450 ms stddev 284.591 2023-09-12T16:12:56.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:56 smithi130 ceph-mon[98670]: pgmap v592: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 18 MiB/s wr, 943 op/s 2023-09-12T16:12:56.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:56 smithi159 ceph-mon[102522]: pgmap v592: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 18 MiB/s wr, 943 op/s 2023-09-12T16:12:56.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:56 smithi203 ceph-mon[103431]: pgmap v592: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 18 MiB/s wr, 943 op/s 2023-09-12T16:12:58.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:12:58 smithi159 ceph-mon[102522]: pgmap v593: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 26 MiB/s wr, 1.90k op/s 2023-09-12T16:12:58.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:12:58 smithi130 ceph-mon[98670]: pgmap v593: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 26 MiB/s wr, 1.90k op/s 2023-09-12T16:12:58.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:12:58 smithi203 ceph-mon[103431]: pgmap v593: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 26 MiB/s wr, 1.90k op/s 2023-09-12T16:13:00.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 430.0 s, 475.6 tps, lat 216.422 ms stddev 330.242 2023-09-12T16:13:00.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:00 smithi130 ceph-mon[98670]: pgmap v594: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 25 MiB/s wr, 1.88k op/s 2023-09-12T16:13:00.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:00 smithi159 ceph-mon[102522]: pgmap v594: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 25 MiB/s wr, 1.88k op/s 2023-09-12T16:13:00.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:00 smithi203 ceph-mon[103431]: pgmap v594: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 25 MiB/s wr, 1.88k op/s 2023-09-12T16:13:02.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:02 smithi130 ceph-mon[98670]: pgmap v595: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 28 MiB/s wr, 2.37k op/s 2023-09-12T16:13:02.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:02 smithi159 ceph-mon[102522]: pgmap v595: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 28 MiB/s wr, 2.37k op/s 2023-09-12T16:13:02.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:02 smithi203 ceph-mon[103431]: pgmap v595: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 28 MiB/s wr, 2.37k op/s 2023-09-12T16:13:04.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:04 smithi130 ceph-mon[98670]: pgmap v596: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 31 MiB/s wr, 2.76k op/s 2023-09-12T16:13:04.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:04 smithi159 ceph-mon[102522]: pgmap v596: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 31 MiB/s wr, 2.76k op/s 2023-09-12T16:13:04.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:04 smithi203 ceph-mon[103431]: pgmap v596: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 31 MiB/s wr, 2.76k op/s 2023-09-12T16:13:05.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 435.0 s, 1592.8 tps, lat 62.865 ms stddev 65.497 2023-09-12T16:13:06.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:06 smithi159 ceph-mon[102522]: pgmap v597: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 28 MiB/s wr, 2.69k op/s 2023-09-12T16:13:06.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:06 smithi130 ceph-mon[98670]: pgmap v597: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 28 MiB/s wr, 2.69k op/s 2023-09-12T16:13:06.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:06 smithi203 ceph-mon[103431]: pgmap v597: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 28 MiB/s wr, 2.69k op/s 2023-09-12T16:13:08.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:08 smithi130 ceph-mon[98670]: pgmap v598: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 29 MiB/s wr, 2.63k op/s 2023-09-12T16:13:08.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:08 smithi159 ceph-mon[102522]: pgmap v598: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 29 MiB/s wr, 2.63k op/s 2023-09-12T16:13:08.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:08 smithi203 ceph-mon[103431]: pgmap v598: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 29 MiB/s wr, 2.63k op/s 2023-09-12T16:13:09.085 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:13:09.401 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.401+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/4249099643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab41087b0 msgr2=0x7f9ab410f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:09.402 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.401+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/4249099643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab41087b0 0x7f9ab410f060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b0a0 tx=0x7f9aac00b3b0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.407+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/4249099643 shutdown_connections 2023-09-12T16:13:09.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.407+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/4249099643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab41087b0 0x7f9ab410f060 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.407+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/4249099643 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab4106c20 0x7f9ab4107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.407+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/4249099643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab4107e10 0x7f9ab4108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.407+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/4249099643 >> 172.21.15.130:0/4249099643 conn(0x7f9ab4075bc0 msgr2=0x7f9ab4078000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:09.417 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.414+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/4249099643 shutdown_connections 2023-09-12T16:13:09.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.422+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/4249099643 wait complete. 2023-09-12T16:13:09.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.422+0000 7f9ab9a0e700 1 Processor -- start 2023-09-12T16:13:09.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.422+0000 7f9ab9a0e700 1 -- start start 2023-09-12T16:13:09.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.423+0000 7f9ab9a0e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab4106c20 0x7f9ab406ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.423+0000 7f9ab9a0e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab4107e10 0x7f9ab406f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.423+0000 7f9ab9a0e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab41087b0 0x7f9ab419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.423+0000 7f9ab9a0e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40724e0 con 0x7f9ab4107e10 2023-09-12T16:13:09.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.423+0000 7f9ab9a0e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4072650 con 0x7f9ab41087b0 2023-09-12T16:13:09.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.423+0000 7f9ab9a0e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab419c170 con 0x7f9ab4106c20 2023-09-12T16:13:09.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.423+0000 7f9ab2ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab4106c20 0x7f9ab406ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.423+0000 7f9ab2ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab4106c20 0x7f9ab406ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35944/0 (socket says 172.21.15.130:35944) 2023-09-12T16:13:09.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.423+0000 7f9ab2ffd700 1 -- 172.21.15.130:0/3743006203 learned_addr learned my addr 172.21.15.130:0/3743006203 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:09.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.424+0000 7f9aabfff700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab4107e10 0x7f9ab406f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.424+0000 7f9ab2ffd700 1 -- 172.21.15.130:0/3743006203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab41087b0 msgr2=0x7f9ab419bc30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-12T16:13:09.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.424+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab41087b0 0x7f9ab419bc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.424+0000 7f9ab2ffd700 1 -- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab4107e10 msgr2=0x7f9ab406f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:09.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.424+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab4107e10 0x7f9ab406f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.424+0000 7f9ab2ffd700 1 -- 172.21.15.130:0/3743006203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac009d00 con 0x7f9ab4106c20 2023-09-12T16:13:09.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.425+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab4106c20 0x7f9ab406ee50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00b500 tx=0x7f9a9c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:09.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.434+0000 7f9ab8a0c700 1 -- 172.21.15.130:0/3743006203 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c011830 con 0x7f9ab4106c20 2023-09-12T16:13:09.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.434+0000 7f9ab8a0c700 1 -- 172.21.15.130:0/3743006203 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a9c011e50 con 0x7f9ab4106c20 2023-09-12T16:13:09.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.434+0000 7f9ab8a0c700 1 -- 172.21.15.130:0/3743006203 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c00f2e0 con 0x7f9ab4106c20 2023-09-12T16:13:09.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.441+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/3743006203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab419c450 con 0x7f9ab4106c20 2023-09-12T16:13:09.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.441+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/3743006203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab419c960 con 0x7f9ab4106c20 2023-09-12T16:13:09.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.454+0000 7f9ac5bd4b80 1 Processor -- start 2023-09-12T16:13:09.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.455+0000 7f9ab8a0c700 1 -- 172.21.15.130:0/3743006203 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f9a9c010560 con 0x7f9ab4106c20 2023-09-12T16:13:09.460 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.456+0000 7f9ab8a0c700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa00a07c0 0x7f9aa00a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.456+0000 7f9ab8a0c700 1 -- 172.21.15.130:0/3743006203 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9a9c0cb8e0 con 0x7f9ab4106c20 2023-09-12T16:13:09.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9ac5bd4b80 1 -- start start 2023-09-12T16:13:09.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd62c70 0x55fbcfd63090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd61850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.461 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfca9730 con 0x55fbcfd635d0 2023-09-12T16:13:09.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfca7050 con 0x55fbcfd61e30 2023-09-12T16:13:09.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfca2940 con 0x55fbcfd62c70 2023-09-12T16:13:09.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9aabfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd61850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.462 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9ab37fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9aabfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd61850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56340/0 (socket says 172.21.15.130:56340) 2023-09-12T16:13:09.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9ab37fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39600/0 (socket says 172.21.15.130:39600) 2023-09-12T16:13:09.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.458+0000 7f9aabfff700 1 -- 172.21.15.130:0/2770473586 learned_addr learned my addr 172.21.15.130:0/2770473586 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:09.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.459+0000 7f9ab37fe700 1 -- 172.21.15.130:0/2770473586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd62c70 msgr2=0x55fbcfd63090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:09.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.459+0000 7f9ab37fe700 1 --2- 172.21.15.130:0/2770473586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd62c70 0x55fbcfd63090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.459+0000 7f9ab37fe700 1 -- 172.21.15.130:0/2770473586 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 msgr2=0x55fbcfd61850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:09.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.459+0000 7f9ab37fe700 1 --2- 172.21.15.130:0/2770473586 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd61850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.459+0000 7f9ab37fe700 1 -- 172.21.15.130:0/2770473586 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbcfacede0 con 0x55fbcfd61e30 2023-09-12T16:13:09.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.459+0000 7f9ab37fe700 1 --2- 172.21.15.130:0/2770473586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd622b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b830 tx=0x7f9aac008e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c382ccbc72b70b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:09.464 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.459+0000 7f9aa9ffb700 1 -- 172.21.15.130:0/2770473586 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac00e040 con 0x55fbcfd61e30 2023-09-12T16:13:09.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.459+0000 7f9aa9ffb700 1 -- 172.21.15.130:0/2770473586 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aac0092d0 con 0x55fbcfd61e30 2023-09-12T16:13:09.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.459+0000 7f9aa9ffb700 1 -- 172.21.15.130:0/2770473586 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac01c6a0 con 0x55fbcfd61e30 2023-09-12T16:13:09.465 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.460+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/2770473586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 msgr2=0x55fbcfd622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:09.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.460+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/2770473586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd622b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b830 tx=0x7f9aac008e10 comp rx=0 tx=0).stop 2023-09-12T16:13:09.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.460+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/2770473586 shutdown_connections 2023-09-12T16:13:09.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.460+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/2770473586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd622b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.466 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.460+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/2770473586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd62c70 0x55fbcfd63090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.460+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/2770473586 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd61850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.460+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/2770473586 >> 172.21.15.130:0/2770473586 conn(0x55fbcfcc5090 msgr2=0x55fbcfd4b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:09.467 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.461+0000 7f9ab37fe700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa00a07c0 0x7f9aa00a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.463+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/2770473586 shutdown_connections 2023-09-12T16:13:09.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.463+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/2770473586 wait complete. 2023-09-12T16:13:09.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.463+0000 7f9ab37fe700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa00a07c0 0x7f9aa00a2c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9aac007b30 tx=0x7f9aac006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:09.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.464+0000 7f9ac5bd4b80 1 -- start start 2023-09-12T16:13:09.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.464+0000 7f9ac5bd4b80 1 Processor -- start 2023-09-12T16:13:09.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.464+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd61e30 0x55fbcfd51f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.464+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd52460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.464+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 0x55fbcfd59920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.464+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfc40b70 con 0x55fbcfd635d0 2023-09-12T16:13:09.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.464+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfc40870 con 0x55fbcfd57490 2023-09-12T16:13:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.464+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfb24920 con 0x55fbcfd61e30 2023-09-12T16:13:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.464+0000 7f9ac5bd4b80 10 client.0 _ll_register_callbacks cb 0x55fbcfb8aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:13:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.465+0000 7f9aabfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd52460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.465+0000 7f9aabfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd52460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56346/0 (socket says 172.21.15.130:56346) 2023-09-12T16:13:09.471 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.465+0000 7f9aabfff700 1 -- 172.21.15.130:0/677623713 learned_addr learned my addr 172.21.15.130:0/677623713 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.465+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd61e30 0x55fbcfd51f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.465+0000 7f9aabfff700 1 -- 172.21.15.130:0/677623713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd61e30 msgr2=0x55fbcfd51f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.465+0000 7f9aabfff700 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd61e30 0x55fbcfd51f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.465+0000 7f9aabfff700 1 -- 172.21.15.130:0/677623713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 msgr2=0x55fbcfd59920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:13:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.465+0000 7f9aabfff700 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 0x55fbcfd59920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.472 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.465+0000 7f9aabfff700 1 -- 172.21.15.130:0/677623713 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab410f9b0 con 0x55fbcfd635d0 2023-09-12T16:13:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.466+0000 7f9aabfff700 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd52460 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009f80 tx=0x7f9aa4032540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c94892a052ae5e server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.466+0000 7f9a92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:13:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.466+0000 7f9a92ffd700 1 -- 172.21.15.130:0/677623713 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4036640 con 0x55fbcfd635d0 2023-09-12T16:13:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.466+0000 7f9a92ffd700 1 -- 172.21.15.130:0/677623713 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9aa4032e80 con 0x55fbcfd635d0 2023-09-12T16:13:09.473 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.466+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/677623713 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbcfbfa4f0 con 0x55fbcfd635d0 2023-09-12T16:13:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.466+0000 7f9a92ffd700 1 -- 172.21.15.130:0/677623713 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aa40367a0 con 0x55fbcfd635d0 2023-09-12T16:13:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.466+0000 7f9ac5bd4b80 10 client.16074 fetch_fsmap learned FSMap version 32 2023-09-12T16:13:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.466+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/677623713 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fbcfab7c60 con 0x55fbcfd635d0 2023-09-12T16:13:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.466+0000 7f9a92ffd700 1 -- 172.21.15.130:0/677623713 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fbcfbfa4f0 con 0x55fbcfd635d0 2023-09-12T16:13:09.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.467+0000 7f9a92ffd700 1 -- 172.21.15.130:0/677623713 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9aa4035a60 con 0x55fbcfd635d0 2023-09-12T16:13:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.467+0000 7f9ac5bd4b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:13:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.467+0000 7f9ac5bd4b80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:13:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.467+0000 7f9ac5bd4b80 20 client.16074 populate_metadata read hostname 'smithi130' 2023-09-12T16:13:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.467+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55fbcfd5e130 0x55fbcfd60550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.467+0000 7f9ac5bd4b80 4 client.16074 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:13:09.475 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.467+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/677623713 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbcfcb78a0 con 0x55fbcfd5e130 2023-09-12T16:13:09.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.467+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55fbcfd5e130 0x55fbcfd60550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.468+0000 7f9a92ffd700 10 client.16074 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:13:09.476 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.468+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55fbcfd5e130 0x55fbcfd60550 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90ee5ba3ddff47d server_cookie=f0156eee5df9e4d8 in_seq=0 out_seq=0 2023-09-12T16:13:09.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.468+0000 7f9a92ffd700 1 -- 172.21.15.130:0/677623713 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fbcfcb78a0 con 0x55fbcfd5e130 2023-09-12T16:13:09.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.468+0000 7f9a92ffd700 10 client.16074 handle_command_reply: tid=0 2023-09-12T16:13:09.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.469+0000 7f9ac5bd4b80 1 client.16074 shutdown 2023-09-12T16:13:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.469+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/677623713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 msgr2=0x55fbcfd52460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.469+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd52460 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009f80 tx=0x7f9aa4032540 comp rx=0 tx=0).stop 2023-09-12T16:13:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/677623713 shutdown_connections 2023-09-12T16:13:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55fbcfd5e130 0x55fbcfd60550 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 0x55fbcfd59920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd61e30 0x55fbcfd51f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/677623713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd635d0 0x55fbcfd52460 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/677623713 >> 172.21.15.130:0/677623713 conn(0x55fbcfcc5090 msgr2=0x55fbcfd4adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:09.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9a92ffd700 0 client.16074 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:13:09.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/677623713 shutdown_connections 2023-09-12T16:13:09.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/677623713 wait complete. 2023-09-12T16:13:09.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.470+0000 7f9ac5bd4b80 20 client.16074 trim_cache size 0 max 16384 2023-09-12T16:13:09.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ac5bd4b80 1 Processor -- start 2023-09-12T16:13:09.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ac5bd4b80 1 -- start start 2023-09-12T16:13:09.499 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd57490 0x55fbcfd509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 0x55fbcfd51f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfd513d0 con 0x55fbcfd57490 2023-09-12T16:13:09.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfd51a90 con 0x55fbcfd61e30 2023-09-12T16:13:09.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfd5b540 con 0x55fbcfd635d0 2023-09-12T16:13:09.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd57490 0x55fbcfd509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9aabfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd57490 0x55fbcfd509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56352/0 (socket says 172.21.15.130:56352) 2023-09-12T16:13:09.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab2ffd700 1 -- 172.21.15.130:0/1024669860 learned_addr learned my addr 172.21.15.130:0/1024669860 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:09.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab37fe700 1 --2- 172.21.15.130:0/1024669860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 0x55fbcfd51f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab2ffd700 1 -- 172.21.15.130:0/1024669860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 msgr2=0x55fbcfd51f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:09.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/1024669860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 0x55fbcfd51f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab2ffd700 1 -- 172.21.15.130:0/1024669860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 msgr2=0x55fbcfd6df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:09.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/1024669860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab2ffd700 1 -- 172.21.15.130:0/1024669860 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbcfaf5250 con 0x55fbcfd57490 2023-09-12T16:13:09.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9aabfff700 1 --2- 172.21.15.130:0/1024669860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:13:09.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.497+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/1024669860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd57490 0x55fbcfd509e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c098a60 tx=0x7f9a9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9347130df189cec server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:09.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9aaa7fc700 1 -- 172.21.15.130:0/1024669860 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c098d60 con 0x55fbcfd57490 2023-09-12T16:13:09.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9aaa7fc700 1 -- 172.21.15.130:0/1024669860 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a9c09cb90 con 0x55fbcfd57490 2023-09-12T16:13:09.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9aaa7fc700 1 -- 172.21.15.130:0/1024669860 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c098250 con 0x55fbcfd57490 2023-09-12T16:13:09.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/1024669860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd57490 msgr2=0x55fbcfd509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:09.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/1024669860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd57490 0x55fbcfd509e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c098a60 tx=0x7f9a9c000bc0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/1024669860 shutdown_connections 2023-09-12T16:13:09.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/1024669860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 0x55fbcfd51f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/1024669860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd57490 0x55fbcfd509e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/1024669860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/1024669860 >> 172.21.15.130:0/1024669860 conn(0x55fbcfcc5090 msgr2=0x55fbcfd60e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:09.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/1024669860 shutdown_connections 2023-09-12T16:13:09.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.498+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/1024669860 wait complete. 2023-09-12T16:13:09.505 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ac5bd4b80 1 -- start start 2023-09-12T16:13:09.509 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ac5bd4b80 1 Processor -- start 2023-09-12T16:13:09.509 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 0x55fbcfd6da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ac5bd4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 0x55fbcfd6c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfd6c660 con 0x55fbcfd61e30 2023-09-12T16:13:09.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfd6c7d0 con 0x55fbcfd57490 2023-09-12T16:13:09.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ac5bd4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcfd6c990 con 0x55fbcfd635d0 2023-09-12T16:13:09.510 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ac5bd4b80 10 client.0 _ll_register_callbacks cb 0x7f9aa4009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:13:09.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9aabfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.499+0000 7f9ab2ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 0x55fbcfd6da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9aabfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56360/0 (socket says 172.21.15.130:56360) 2023-09-12T16:13:09.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9ab2ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 0x55fbcfd6da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39616/0 (socket says 172.21.15.130:39616) 2023-09-12T16:13:09.511 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9aabfff700 1 -- 172.21.15.130:0/3000423455 learned_addr learned my addr 172.21.15.130:0/3000423455 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:09.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9ab37fe700 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 0x55fbcfd6c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9aabfff700 1 -- 172.21.15.130:0/3000423455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 msgr2=0x55fbcfd6c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:09.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9aabfff700 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 0x55fbcfd6c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9aabfff700 1 -- 172.21.15.130:0/3000423455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 msgr2=0x55fbcfd6da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:09.512 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9aabfff700 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 0x55fbcfd6da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.513 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9aabfff700 1 -- 172.21.15.130:0/3000423455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbcfab7c60 con 0x55fbcfd61e30 2023-09-12T16:13:09.513 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 0x55fbcfd6da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:13:09.513 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9aabfff700 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400a010 tx=0x7f9aa403e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e8cc95cbfc4cdf server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:09.513 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9a92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:13:09.513 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9a92ffd700 1 -- 172.21.15.130:0/3000423455 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4004450 con 0x55fbcfd61e30 2023-09-12T16:13:09.513 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9a92ffd700 1 -- 172.21.15.130:0/3000423455 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9aa40045b0 con 0x55fbcfd61e30 2023-09-12T16:13:09.514 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/3000423455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9aac0092d0 con 0x55fbcfd61e30 2023-09-12T16:13:09.514 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9a92ffd700 1 -- 172.21.15.130:0/3000423455 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aa403e7b0 con 0x55fbcfd61e30 2023-09-12T16:13:09.514 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9ac5bd4b80 10 client.16080 fetch_fsmap learned FSMap version 32 2023-09-12T16:13:09.514 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/3000423455 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f9ab419c960 con 0x55fbcfd61e30 2023-09-12T16:13:09.514 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.500+0000 7f9a92ffd700 1 -- 172.21.15.130:0/3000423455 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aac0092d0 con 0x55fbcfd61e30 2023-09-12T16:13:09.515 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9a92ffd700 1 -- 172.21.15.130:0/3000423455 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9aa4003a00 con 0x55fbcfd61e30 2023-09-12T16:13:09.515 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 10 client.16080 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:13:09.515 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:13:09.515 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 20 client.16080 populate_metadata read hostname 'smithi130' 2023-09-12T16:13:09.515 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55fbcfdbc3b0 0x55fbcfdbe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:09.516 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 4 client.16080 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:13:09.516 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/3000423455 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fbcfc9b920 con 0x55fbcfdbc3b0 2023-09-12T16:13:09.516 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55fbcfdbc3b0 0x55fbcfdbe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:09.516 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9a92ffd700 10 client.16080 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:13:09.516 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ab2ffd700 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55fbcfdbc3b0 0x55fbcfdbe7d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f0e5e6b14a112c server_cookie=49cf673ef7ca6785 in_seq=0 out_seq=0 2023-09-12T16:13:09.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9a92ffd700 1 -- 172.21.15.130:0/3000423455 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55fbcfc9b920 con 0x55fbcfdbc3b0 2023-09-12T16:13:09.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9a92ffd700 10 client.16080 handle_command_reply: tid=0 2023-09-12T16:13:09.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 1 client.16080 shutdown 2023-09-12T16:13:09.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/3000423455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd61e30 msgr2=0x55fbcfd6df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:09.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400a010 tx=0x7f9aa403e420 comp rx=0 tx=0).stop 2023-09-12T16:13:09.517 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/3000423455 shutdown_connections 2023-09-12T16:13:09.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.501+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55fbcfdbc3b0 0x55fbcfdbe7d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbcfd635d0 0x55fbcfd6c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fbcfd57490 0x55fbcfd6da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ac5bd4b80 1 --2- 172.21.15.130:0/3000423455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fbcfd61e30 0x55fbcfd6df70 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.518 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/3000423455 >> 172.21.15.130:0/3000423455 conn(0x55fbcfcc5090 msgr2=0x55fbcfd509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:09.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9a92ffd700 0 client.16080 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:13:09.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/3000423455 shutdown_connections 2023-09-12T16:13:09.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ac5bd4b80 1 -- 172.21.15.130:0/3000423455 wait complete. 2023-09-12T16:13:09.519 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ac5bd4b80 20 client.16080 trim_cache size 0 max 16384 2023-09-12T16:13:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:13:09.520 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:13:09.520 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:13:09.520 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:13:09.520 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:13:09.521 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:13:09.521 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:13:09.521 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:13:09.521 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:13:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:13:09.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa00a07c0 msgr2=0x7f9aa00a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:09.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa00a07c0 0x7f9aa00a2c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9aac007b30 tx=0x7f9aac006210 comp rx=0 tx=0).stop 2023-09-12T16:13:09.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/3743006203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab4106c20 msgr2=0x7f9ab406ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:09.523 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab4106c20 0x7f9ab406ee50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00b500 tx=0x7f9a9c00b8c0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/3743006203 shutdown_connections 2023-09-12T16:13:09.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f9aa00a07c0 0x7f9aa00a2c80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ab41087b0 0x7f9ab419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab4106c20 0x7f9ab406ee50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 --2- 172.21.15.130:0/3743006203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ab4107e10 0x7f9ab406f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:09.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/3743006203 >> 172.21.15.130:0/3743006203 conn(0x7f9ab4075bc0 msgr2=0x7f9ab410b340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:09.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/3743006203 shutdown_connections 2023-09-12T16:13:09.525 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:09.502+0000 7f9ab9a0e700 1 -- 172.21.15.130:0/3743006203 wait complete. 2023-09-12T16:13:09.525 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:13:10.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 440.0 s, 823.2 tps, lat 120.121 ms stddev 349.986 2023-09-12T16:13:10.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:10 smithi159 ceph-mon[102522]: pgmap v599: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 2.01k op/s 2023-09-12T16:13:10.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:10 smithi130 ceph-mon[98670]: pgmap v599: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 2.01k op/s 2023-09-12T16:13:10.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:10 smithi203 ceph-mon[103431]: pgmap v599: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 2.01k op/s 2023-09-12T16:13:12.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:12 smithi159 ceph-mon[102522]: pgmap v600: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 25 MiB/s wr, 2.00k op/s 2023-09-12T16:13:12.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:12 smithi130 ceph-mon[98670]: pgmap v600: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 25 MiB/s wr, 2.00k op/s 2023-09-12T16:13:12.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:12 smithi203 ceph-mon[103431]: pgmap v600: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 25 MiB/s wr, 2.00k op/s 2023-09-12T16:13:14.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:14 smithi159 ceph-mon[102522]: pgmap v601: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 1.89k op/s 2023-09-12T16:13:14.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:14 smithi130 ceph-mon[98670]: pgmap v601: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 1.89k op/s 2023-09-12T16:13:14.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:14 smithi203 ceph-mon[103431]: pgmap v601: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 1.89k op/s 2023-09-12T16:13:15.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 445.0 s, 1210.5 tps, lat 71.819 ms stddev 58.215 2023-09-12T16:13:16.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:16 smithi159 ceph-mon[102522]: pgmap v602: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 23 MiB/s wr, 1.44k op/s 2023-09-12T16:13:16.983 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:16 smithi130 ceph-mon[98670]: pgmap v602: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 23 MiB/s wr, 1.44k op/s 2023-09-12T16:13:16.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:16 smithi203 ceph-mon[103431]: pgmap v602: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 23 MiB/s wr, 1.44k op/s 2023-09-12T16:13:18.478 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:18 smithi159 ceph-mon[102522]: pgmap v603: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 22 MiB/s wr, 1.42k op/s 2023-09-12T16:13:18.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:18 smithi130 ceph-mon[98670]: pgmap v603: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 22 MiB/s wr, 1.42k op/s 2023-09-12T16:13:18.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:18 smithi203 ceph-mon[103431]: pgmap v603: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 22 MiB/s wr, 1.42k op/s 2023-09-12T16:13:20.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 450.0 s, 704.5 tps, lat 161.289 ms stddev 230.742 2023-09-12T16:13:20.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:20 smithi130 ceph-mon[98670]: pgmap v604: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 21 MiB/s wr, 1.36k op/s 2023-09-12T16:13:20.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:20 smithi159 ceph-mon[102522]: pgmap v604: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 21 MiB/s wr, 1.36k op/s 2023-09-12T16:13:20.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:20 smithi203 ceph-mon[103431]: pgmap v604: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 21 MiB/s wr, 1.36k op/s 2023-09-12T16:13:22.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:22 smithi159 ceph-mon[102522]: pgmap v605: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 17 MiB/s wr, 1.03k op/s 2023-09-12T16:13:22.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:22 smithi130 ceph-mon[98670]: pgmap v605: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 17 MiB/s wr, 1.03k op/s 2023-09-12T16:13:22.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:22 smithi203 ceph-mon[103431]: pgmap v605: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 17 MiB/s wr, 1.03k op/s 2023-09-12T16:13:24.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:24 smithi130 ceph-mon[98670]: pgmap v606: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2023-09-12T16:13:24.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:24 smithi159 ceph-mon[102522]: pgmap v606: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2023-09-12T16:13:24.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:24 smithi203 ceph-mon[103431]: pgmap v606: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2023-09-12T16:13:25.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 455.0 s, 1654.5 tps, lat 59.361 ms stddev 51.288 2023-09-12T16:13:26.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:26 smithi130 ceph-mon[98670]: pgmap v607: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 766 op/s 2023-09-12T16:13:26.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:26 smithi159 ceph-mon[102522]: pgmap v607: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 766 op/s 2023-09-12T16:13:26.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:26 smithi203 ceph-mon[103431]: pgmap v607: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 766 op/s 2023-09-12T16:13:28.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:28 smithi130 ceph-mon[98670]: pgmap v608: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 769 op/s 2023-09-12T16:13:28.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:28 smithi159 ceph-mon[102522]: pgmap v608: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 769 op/s 2023-09-12T16:13:28.500 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:28 smithi203 ceph-mon[103431]: pgmap v608: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 769 op/s 2023-09-12T16:13:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 460.0 s, 571.1 tps, lat 152.134 ms stddev 375.568 2023-09-12T16:13:30.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:30 smithi130 ceph-mon[98670]: pgmap v609: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 24 MiB/s wr, 1.38k op/s 2023-09-12T16:13:30.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:30 smithi159 ceph-mon[102522]: pgmap v609: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 24 MiB/s wr, 1.38k op/s 2023-09-12T16:13:30.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:30 smithi203 ceph-mon[103431]: pgmap v609: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 24 MiB/s wr, 1.38k op/s 2023-09-12T16:13:31.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:31 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:13:31.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:31 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:13:31.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:31 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:13:32.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:32 smithi159 ceph-mon[102522]: pgmap v610: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 24 MiB/s wr, 1.59k op/s 2023-09-12T16:13:32.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:32 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:13:32.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:32 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:13:32.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:32 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:13:32.984 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:32 smithi130 ceph-mon[98670]: pgmap v610: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 24 MiB/s wr, 1.59k op/s 2023-09-12T16:13:32.984 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:32 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:13:32.984 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:32 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:13:32.985 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:32 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:13:32.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:32 smithi203 ceph-mon[103431]: pgmap v610: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 24 MiB/s wr, 1.59k op/s 2023-09-12T16:13:32.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:32 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:13:32.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:32 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:13:32.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:32 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:13:34.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:34 smithi130 ceph-mon[98670]: pgmap v611: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 33 MiB/s wr, 2.73k op/s 2023-09-12T16:13:34.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:34 smithi159 ceph-mon[102522]: pgmap v611: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 33 MiB/s wr, 2.73k op/s 2023-09-12T16:13:35.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 465.0 s, 805.6 tps, lat 142.950 ms stddev 341.528 2023-09-12T16:13:35.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:34 smithi203 ceph-mon[103431]: pgmap v611: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 33 MiB/s wr, 2.73k op/s 2023-09-12T16:13:36.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:36 smithi159 ceph-mon[102522]: pgmap v612: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 31 MiB/s wr, 2.79k op/s 2023-09-12T16:13:36.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:36 smithi130 ceph-mon[98670]: pgmap v612: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 31 MiB/s wr, 2.79k op/s 2023-09-12T16:13:37.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:36 smithi203 ceph-mon[103431]: pgmap v612: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 31 MiB/s wr, 2.79k op/s 2023-09-12T16:13:38.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:38 smithi130 ceph-mon[98670]: pgmap v613: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 32 MiB/s wr, 2.90k op/s 2023-09-12T16:13:38.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:38 smithi159 ceph-mon[102522]: pgmap v613: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 32 MiB/s wr, 2.90k op/s 2023-09-12T16:13:38.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:38 smithi203 ceph-mon[103431]: pgmap v613: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 32 MiB/s wr, 2.90k op/s 2023-09-12T16:13:39.527 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:13:39.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.759+0000 7f0026766700 1 -- 172.21.15.130:0/977525091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 msgr2=0x7f002010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:39.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.759+0000 7f0026766700 1 --2- 172.21.15.130:0/977525091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 0x7f002010cdc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0010009990 tx=0x7f0010009ca0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.762+0000 7f0026766700 1 -- 172.21.15.130:0/977525091 shutdown_connections 2023-09-12T16:13:39.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.762+0000 7f0026766700 1 --2- 172.21.15.130:0/977525091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 0x7f002010cdc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.762+0000 7f0026766700 1 --2- 172.21.15.130:0/977525091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0020107600 0x7f0020109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.762+0000 7f0026766700 1 --2- 172.21.15.130:0/977525091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020109f70 0x7f002010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.762+0000 7f0026766700 1 -- 172.21.15.130:0/977525091 >> 172.21.15.130:0/977525091 conn(0x7f002006d3b0 msgr2=0x7f002006f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:39.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.776+0000 7f0026766700 1 -- 172.21.15.130:0/977525091 shutdown_connections 2023-09-12T16:13:39.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.785+0000 7f0026766700 1 -- 172.21.15.130:0/977525091 wait complete. 2023-09-12T16:13:39.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.786+0000 7f0026766700 1 Processor -- start 2023-09-12T16:13:39.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.786+0000 7f0026766700 1 -- start start 2023-09-12T16:13:39.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.786+0000 7f0026766700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0020107600 0x7f0020113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.786+0000 7f0026766700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020109f70 0x7f0020114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.786+0000 7f0026766700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 0x7f00201a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0026766700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00201171d0 con 0x7f0020107600 2023-09-12T16:13:39.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0026766700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0020117340 con 0x7f0020109f70 2023-09-12T16:13:39.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0026766700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00201a4920 con 0x7f002010c940 2023-09-12T16:13:39.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0024d03700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 0x7f00201a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0024d03700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 0x7f00201a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:35994/0 (socket says 172.21.15.130:35994) 2023-09-12T16:13:39.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0024d03700 1 -- 172.21.15.130:0/1204217702 learned_addr learned my addr 172.21.15.130:0/1204217702 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:39.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f001ffff700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0020107600 0x7f0020113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f001f7fe700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020109f70 0x7f0020114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0024d03700 1 -- 172.21.15.130:0/1204217702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020109f70 msgr2=0x7f0020114080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:39.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0024d03700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020109f70 0x7f0020114080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0024d03700 1 -- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0020107600 msgr2=0x7f0020113b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:39.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0024d03700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0020107600 0x7f0020113b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0024d03700 1 -- 172.21.15.130:0/1204217702 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00100095f0 con 0x7f002010c940 2023-09-12T16:13:39.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.787+0000 7f0024d03700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 0x7f00201a43e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0010003cd0 tx=0x7f0010003d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:39.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.788+0000 7f001d7fa700 1 -- 172.21.15.130:0/1204217702 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001001d070 con 0x7f002010c940 2023-09-12T16:13:39.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.788+0000 7f001d7fa700 1 -- 172.21.15.130:0/1204217702 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0010003e20 con 0x7f002010c940 2023-09-12T16:13:39.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.788+0000 7f0026766700 1 -- 172.21.15.130:0/1204217702 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00201a4ba0 con 0x7f002010c940 2023-09-12T16:13:39.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.788+0000 7f001d7fa700 1 -- 172.21.15.130:0/1204217702 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00100173f0 con 0x7f002010c940 2023-09-12T16:13:39.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.788+0000 7f0026766700 1 -- 172.21.15.130:0/1204217702 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00201a5120 con 0x7f002010c940 2023-09-12T16:13:39.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.790+0000 7f001d7fa700 1 -- 172.21.15.130:0/1204217702 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f00100052d0 con 0x7f002010c940 2023-09-12T16:13:39.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.796+0000 7f001d7fa700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00080a0740 0x7f00080a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.796+0000 7f001d7fa700 1 -- 172.21.15.130:0/1204217702 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f00100cd140 con 0x7f002010c940 2023-09-12T16:13:39.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.797+0000 7f001ffff700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00080a0740 0x7f00080a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.797+0000 7f003292cb80 1 Processor -- start 2023-09-12T16:13:39.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.797+0000 7f003292cb80 1 -- start start 2023-09-12T16:13:39.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.797+0000 7f003292cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66bc00 0x5652bf66c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.797+0000 7f003292cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66c5a0 0x5652bf66a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.797+0000 7f003292cb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf66b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.797+0000 7f003292cb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf5c3730 con 0x5652bf66ae00 2023-09-12T16:13:39.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.797+0000 7f003292cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf5c1050 con 0x5652bf66bc00 2023-09-12T16:13:39.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.797+0000 7f003292cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf5bc940 con 0x5652bf66c5a0 2023-09-12T16:13:39.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.801+0000 7f001ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66bc00 0x5652bf66c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.801+0000 7f001ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66bc00 0x5652bf66c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39646/0 (socket says 172.21.15.130:39646) 2023-09-12T16:13:39.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.801+0000 7f001ffff700 1 -- 172.21.15.130:0/2195345110 learned_addr learned my addr 172.21.15.130:0/2195345110 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:39.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.802+0000 7f0024d03700 1 --2- 172.21.15.130:0/2195345110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf66b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.803+0000 7f001ffff700 1 -- 172.21.15.130:0/2195345110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66c5a0 msgr2=0x5652bf66a800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:13:39.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.803+0000 7f001ffff700 1 --2- 172.21.15.130:0/2195345110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66c5a0 0x5652bf66a800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.803+0000 7f001ffff700 1 -- 172.21.15.130:0/2195345110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 msgr2=0x5652bf66b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:39.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.803+0000 7f001ffff700 1 --2- 172.21.15.130:0/2195345110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf66b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.803+0000 7f001ffff700 1 -- 172.21.15.130:0/2195345110 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652bf3e8de0 con 0x5652bf66bc00 2023-09-12T16:13:39.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.803+0000 7f001ffff700 1 --2- 172.21.15.130:0/2195345110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66bc00 0x5652bf66c020 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00180154d0 tx=0x7f0018018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e37e31fed6136a8d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:39.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.803+0000 7f001ffff700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00080a0740 0x7f00080a2c00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0018009fd0 tx=0x7f001800ac80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:39.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.808+0000 7f00067fc700 1 -- 172.21.15.130:0/2195345110 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018010050 con 0x5652bf66bc00 2023-09-12T16:13:39.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.808+0000 7f00067fc700 1 -- 172.21.15.130:0/2195345110 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0018023910 con 0x5652bf66bc00 2023-09-12T16:13:39.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.808+0000 7f00067fc700 1 -- 172.21.15.130:0/2195345110 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001801a990 con 0x5652bf66bc00 2023-09-12T16:13:39.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.809+0000 7f003292cb80 1 -- 172.21.15.130:0/2195345110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66bc00 msgr2=0x5652bf66c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:39.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.809+0000 7f003292cb80 1 --2- 172.21.15.130:0/2195345110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66bc00 0x5652bf66c020 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00180154d0 tx=0x7f0018018000 comp rx=0 tx=0).stop 2023-09-12T16:13:39.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.809+0000 7f003292cb80 1 -- 172.21.15.130:0/2195345110 shutdown_connections 2023-09-12T16:13:39.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.809+0000 7f003292cb80 1 --2- 172.21.15.130:0/2195345110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf66b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.809+0000 7f003292cb80 1 --2- 172.21.15.130:0/2195345110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66bc00 0x5652bf66c020 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00180154d0 tx=0x7f0018018000 comp rx=0 tx=0).stop 2023-09-12T16:13:39.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.809+0000 7f003292cb80 1 --2- 172.21.15.130:0/2195345110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66c5a0 0x5652bf66a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.809+0000 7f003292cb80 1 -- 172.21.15.130:0/2195345110 >> 172.21.15.130:0/2195345110 conn(0x5652bf5df090 msgr2=0x5652bf665720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:39.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.818+0000 7f003292cb80 1 -- 172.21.15.130:0/2195345110 shutdown_connections 2023-09-12T16:13:39.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.818+0000 7f003292cb80 1 -- 172.21.15.130:0/2195345110 wait complete. 2023-09-12T16:13:39.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.819+0000 7f003292cb80 1 -- start start 2023-09-12T16:13:39.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.819+0000 7f003292cb80 1 Processor -- start 2023-09-12T16:13:39.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.819+0000 7f003292cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66ae00 0x5652bf6747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.819+0000 7f003292cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 0x5652bf674d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f003292cb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf679d50 0x5652bf67c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f003292cb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf55ab70 con 0x5652bf679d50 2023-09-12T16:13:39.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f003292cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf55a870 con 0x5652bf66c5a0 2023-09-12T16:13:39.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f003292cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf43e920 con 0x5652bf66ae00 2023-09-12T16:13:39.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f003292cb80 10 client.0 _ll_register_callbacks cb 0x5652bf4a4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:13:39.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f0024d03700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf679d50 0x5652bf67c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f0024d03700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf679d50 0x5652bf67c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56398/0 (socket says 172.21.15.130:56398) 2023-09-12T16:13:39.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f0024d03700 1 -- 172.21.15.130:0/200451242 learned_addr learned my addr 172.21.15.130:0/200451242 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:39.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f001f7fe700 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 0x5652bf674d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f0024d03700 1 -- 172.21.15.130:0/200451242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66ae00 msgr2=0x5652bf6747e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:13:39.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f0024d03700 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66ae00 0x5652bf6747e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f0024d03700 1 -- 172.21.15.130:0/200451242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 msgr2=0x5652bf674d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:39.825 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f0024d03700 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 0x5652bf674d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f0024d03700 1 -- 172.21.15.130:0/200451242 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0018013040 con 0x5652bf679d50 2023-09-12T16:13:39.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.820+0000 7f001f7fe700 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 0x5652bf674d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:13:39.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f0024d03700 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf679d50 0x5652bf67c140 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f00100ca870 tx=0x7f001000f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaff48fa1026c98 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:39.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7effff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:13:39.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7effff7fe700 1 -- 172.21.15.130:0/200451242 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001009acf0 con 0x5652bf679d50 2023-09-12T16:13:39.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f003292cb80 1 -- 172.21.15.130:0/200451242 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652bf5144f0 con 0x5652bf679d50 2023-09-12T16:13:39.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7effff7fe700 1 -- 172.21.15.130:0/200451242 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0010083e40 con 0x5652bf679d50 2023-09-12T16:13:39.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7effff7fe700 1 -- 172.21.15.130:0/200451242 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f001009a5f0 con 0x5652bf679d50 2023-09-12T16:13:39.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f003292cb80 10 client.16089 fetch_fsmap learned FSMap version 32 2023-09-12T16:13:39.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f003292cb80 1 -- 172.21.15.130:0/200451242 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5652bf3d1c60 con 0x5652bf679d50 2023-09-12T16:13:39.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7effff7fe700 1 -- 172.21.15.130:0/200451242 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001007d080 con 0x5652bf679d50 2023-09-12T16:13:39.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7effff7fe700 1 -- 172.21.15.130:0/200451242 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f001009e030 con 0x5652bf679d50 2023-09-12T16:13:39.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f003292cb80 10 client.16089 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:13:39.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f003292cb80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:13:39.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f003292cb80 20 client.16089 populate_metadata read hostname 'smithi130' 2023-09-12T16:13:39.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f003292cb80 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5652bf6a2420 0x5652bf6a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f003292cb80 4 client.16089 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:13:39.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.821+0000 7f003292cb80 1 -- 172.21.15.130:0/200451242 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652bf5d18a0 con 0x5652bf6a2420 2023-09-12T16:13:39.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.822+0000 7f001ffff700 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5652bf6a2420 0x5652bf6a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.823+0000 7effff7fe700 10 client.16089 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:13:39.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.823+0000 7f001ffff700 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5652bf6a2420 0x5652bf6a4810 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc542c80a31e00b9 server_cookie=916aa9b71db0149a in_seq=0 out_seq=0 2023-09-12T16:13:39.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.823+0000 7effff7fe700 1 -- 172.21.15.130:0/200451242 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5652bf5d18a0 con 0x5652bf6a2420 2023-09-12T16:13:39.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.823+0000 7effff7fe700 10 client.16089 handle_command_reply: tid=0 2023-09-12T16:13:39.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.823+0000 7f003292cb80 1 client.16089 shutdown 2023-09-12T16:13:39.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.823+0000 7f003292cb80 1 -- 172.21.15.130:0/200451242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf679d50 msgr2=0x5652bf67c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:39.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.823+0000 7f003292cb80 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf679d50 0x5652bf67c140 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f00100ca870 tx=0x7f001000f9d0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7f003292cb80 1 -- 172.21.15.130:0/200451242 shutdown_connections 2023-09-12T16:13:39.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7f003292cb80 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5652bf6a2420 0x5652bf6a4810 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7f003292cb80 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf679d50 0x5652bf67c140 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7f003292cb80 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66ae00 0x5652bf6747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7f003292cb80 1 --2- 172.21.15.130:0/200451242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 0x5652bf674d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7f003292cb80 1 -- 172.21.15.130:0/200451242 >> 172.21.15.130:0/200451242 conn(0x5652bf5df090 msgr2=0x5652bf664dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:39.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7effff7fe700 0 client.16089 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:13:39.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7f003292cb80 1 -- 172.21.15.130:0/200451242 shutdown_connections 2023-09-12T16:13:39.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7f003292cb80 1 -- 172.21.15.130:0/200451242 wait complete. 2023-09-12T16:13:39.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.833+0000 7f003292cb80 20 client.16089 trim_cache size 0 max 16384 2023-09-12T16:13:39.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f003292cb80 1 Processor -- start 2023-09-12T16:13:39.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f003292cb80 1 -- start start 2023-09-12T16:13:39.859 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f003292cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66ae00 0x5652bf66a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f003292cb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66c5a0 0x5652bf6b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f003292cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf679d50 0x5652bf6a4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f003292cb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf6759d0 con 0x5652bf66c5a0 2023-09-12T16:13:39.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f003292cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf67c5e0 con 0x5652bf679d50 2023-09-12T16:13:39.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f003292cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf65d400 con 0x5652bf66ae00 2023-09-12T16:13:39.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f001f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66c5a0 0x5652bf6b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f001f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66c5a0 0x5652bf6b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56404/0 (socket says 172.21.15.130:56404) 2023-09-12T16:13:39.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f001f7fe700 1 -- 172.21.15.130:0/1588603950 learned_addr learned my addr 172.21.15.130:0/1588603950 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:39.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.859+0000 7f001ffff700 1 --2- 172.21.15.130:0/1588603950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66ae00 0x5652bf66a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.860+0000 7f001f7fe700 1 -- 172.21.15.130:0/1588603950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66ae00 msgr2=0x5652bf66a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:39.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.860+0000 7f001f7fe700 1 --2- 172.21.15.130:0/1588603950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66ae00 0x5652bf66a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.866 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.860+0000 7f001f7fe700 1 -- 172.21.15.130:0/1588603950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf679d50 msgr2=0x5652bf6a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:39.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.860+0000 7f001f7fe700 1 --2- 172.21.15.130:0/1588603950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf679d50 0x5652bf6a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.860+0000 7f001f7fe700 1 -- 172.21.15.130:0/1588603950 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652bf40f250 con 0x5652bf66c5a0 2023-09-12T16:13:39.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.860+0000 7f001f7fe700 1 --2- 172.21.15.130:0/1588603950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66c5a0 0x5652bf6b2010 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0014009cd0 tx=0x7f00140112c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f9bf593d1e2eab server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:39.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.860+0000 7f0006ffd700 1 -- 172.21.15.130:0/1588603950 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014011b30 con 0x5652bf66c5a0 2023-09-12T16:13:39.867 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.860+0000 7f0006ffd700 1 -- 172.21.15.130:0/1588603950 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0014011c90 con 0x5652bf66c5a0 2023-09-12T16:13:39.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.860+0000 7f0006ffd700 1 -- 172.21.15.130:0/1588603950 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001401d660 con 0x5652bf66c5a0 2023-09-12T16:13:39.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.861+0000 7f003292cb80 1 -- 172.21.15.130:0/1588603950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66c5a0 msgr2=0x5652bf6b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:39.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.861+0000 7f003292cb80 1 --2- 172.21.15.130:0/1588603950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66c5a0 0x5652bf6b2010 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0014009cd0 tx=0x7f00140112c0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.861+0000 7f003292cb80 1 -- 172.21.15.130:0/1588603950 shutdown_connections 2023-09-12T16:13:39.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.861+0000 7f003292cb80 1 --2- 172.21.15.130:0/1588603950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf679d50 0x5652bf6a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.861+0000 7f003292cb80 1 --2- 172.21.15.130:0/1588603950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf66ae00 0x5652bf66a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.861+0000 7f003292cb80 1 --2- 172.21.15.130:0/1588603950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66c5a0 0x5652bf6b2010 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.861+0000 7f003292cb80 1 -- 172.21.15.130:0/1588603950 >> 172.21.15.130:0/1588603950 conn(0x5652bf5df090 msgr2=0x5652bf673190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:39.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.861+0000 7f003292cb80 1 -- 172.21.15.130:0/1588603950 shutdown_connections 2023-09-12T16:13:39.869 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.861+0000 7f003292cb80 1 -- 172.21.15.130:0/1588603950 wait complete. 2023-09-12T16:13:39.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f003292cb80 1 -- start start 2023-09-12T16:13:39.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f003292cb80 1 Processor -- start 2023-09-12T16:13:39.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f003292cb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf6b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f003292cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 0x5652bf6b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.870 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f003292cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf679d50 0x5652bf6b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f003292cb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf6b1b00 con 0x5652bf66ae00 2023-09-12T16:13:39.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f003292cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf6b1c70 con 0x5652bf66c5a0 2023-09-12T16:13:39.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f003292cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652bf6b1e30 con 0x5652bf679d50 2023-09-12T16:13:39.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f003292cb80 10 client.0 _ll_register_callbacks cb 0x5652bf4a4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:13:39.871 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f001ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf6b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f001ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf6b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56408/0 (socket says 172.21.15.130:56408) 2023-09-12T16:13:39.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f0024d03700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf679d50 0x5652bf6b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f001ffff700 1 -- 172.21.15.130:0/1411290328 learned_addr learned my addr 172.21.15.130:0/1411290328 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:13:39.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.862+0000 7f001f7fe700 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 0x5652bf6b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f0024d03700 1 -- 172.21.15.130:0/1411290328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 msgr2=0x5652bf6b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:39.872 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f0024d03700 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 0x5652bf6b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f0024d03700 1 -- 172.21.15.130:0/1411290328 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 msgr2=0x5652bf6b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:13:39.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f0024d03700 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf6b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f0024d03700 1 -- 172.21.15.130:0/1411290328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0014009980 con 0x5652bf679d50 2023-09-12T16:13:39.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f001ffff700 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf6b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:13:39.873 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f0024d03700 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf679d50 0x5652bf6b15c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0010097010 tx=0x7f001000f650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d70c081d51add7d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:13:39.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7effff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:13:39.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7effff7fe700 1 -- 172.21.15.130:0/1411290328 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001000f740 con 0x5652bf679d50 2023-09-12T16:13:39.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7effff7fe700 1 -- 172.21.15.130:0/1411290328 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0010083e40 con 0x5652bf679d50 2023-09-12T16:13:39.874 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f003292cb80 1 -- 172.21.15.130:0/1411290328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0018023910 con 0x5652bf679d50 2023-09-12T16:13:39.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7effff7fe700 1 -- 172.21.15.130:0/1411290328 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00100caab0 con 0x5652bf679d50 2023-09-12T16:13:39.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f003292cb80 10 client.25783 fetch_fsmap learned FSMap version 32 2023-09-12T16:13:39.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.863+0000 7f003292cb80 1 -- 172.21.15.130:0/1411290328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5652bf3e72b0 con 0x5652bf679d50 2023-09-12T16:13:39.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.864+0000 7effff7fe700 1 -- 172.21.15.130:0/1411290328 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001007d070 con 0x5652bf679d50 2023-09-12T16:13:39.875 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.864+0000 7effff7fe700 1 -- 172.21.15.130:0/1411290328 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0010099ae0 con 0x5652bf679d50 2023-09-12T16:13:39.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.864+0000 7f003292cb80 10 client.25783 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:13:39.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.864+0000 7f003292cb80 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:13:39.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.864+0000 7f003292cb80 20 client.25783 populate_metadata read hostname 'smithi130' 2023-09-12T16:13:39.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.864+0000 7f003292cb80 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5652bf6f7590 0x5652bf6f9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:13:39.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.864+0000 7f003292cb80 4 client.25783 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:13:39.876 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.864+0000 7f003292cb80 1 -- 172.21.15.130:0/1411290328 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652bf5b5920 con 0x5652bf6f7590 2023-09-12T16:13:39.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.864+0000 7f001ffff700 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5652bf6f7590 0x5652bf6f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:13:39.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.865+0000 7effff7fe700 10 client.25783 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:13:39.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.865+0000 7f001ffff700 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5652bf6f7590 0x5652bf6f9980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4b96a92df25722f server_cookie=dfa55738c170d75f in_seq=0 out_seq=0 2023-09-12T16:13:39.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.865+0000 7effff7fe700 1 -- 172.21.15.130:0/1411290328 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5652bf5b5920 con 0x5652bf6f7590 2023-09-12T16:13:39.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.865+0000 7effff7fe700 10 client.25783 handle_command_reply: tid=0 2023-09-12T16:13:39.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.865+0000 7f003292cb80 1 client.25783 shutdown 2023-09-12T16:13:39.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 -- 172.21.15.130:0/1411290328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf679d50 msgr2=0x5652bf6b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:39.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf679d50 0x5652bf6b15c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0010097010 tx=0x7f001000f650 comp rx=0 tx=0).stop 2023-09-12T16:13:39.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 -- 172.21.15.130:0/1411290328 shutdown_connections 2023-09-12T16:13:39.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5652bf6f7590 0x5652bf6f9980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5652bf679d50 0x5652bf6b15c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5652bf66ae00 0x5652bf6b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 --2- 172.21.15.130:0/1411290328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652bf66c5a0 0x5652bf6b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 -- 172.21.15.130:0/1411290328 >> 172.21.15.130:0/1411290328 conn(0x5652bf5df090 msgr2=0x5652bf6a4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:39.879 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7effff7fe700 0 client.25783 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:13:39.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 -- 172.21.15.130:0/1411290328 shutdown_connections 2023-09-12T16:13:39.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 1 -- 172.21.15.130:0/1411290328 wait complete. 2023-09-12T16:13:39.880 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.866+0000 7f003292cb80 20 client.25783 trim_cache size 0 max 16384 2023-09-12T16:13:39.880 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:13:39.881 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:13:39.881 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:13:39.881 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:13:39.881 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:13:39.881 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:13:39.882 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:13:39.882 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:13:39.882 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:13:39.882 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:13:39.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.870+0000 7f0026766700 1 -- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00080a0740 msgr2=0x7f00080a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:39.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.870+0000 7f0026766700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00080a0740 0x7f00080a2c00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0018009fd0 tx=0x7f001800ac80 comp rx=0 tx=0).stop 2023-09-12T16:13:39.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.870+0000 7f0026766700 1 -- 172.21.15.130:0/1204217702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 msgr2=0x7f00201a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:13:39.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.870+0000 7f0026766700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 0x7f00201a43e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0010003cd0 tx=0x7f0010003d00 comp rx=0 tx=0).stop 2023-09-12T16:13:39.883 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.871+0000 7f0026766700 1 -- 172.21.15.130:0/1204217702 shutdown_connections 2023-09-12T16:13:39.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.871+0000 7f0026766700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00080a0740 0x7f00080a2c00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.871+0000 7f0026766700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f002010c940 0x7f00201a43e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.871+0000 7f0026766700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0020107600 0x7f0020113b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.871+0000 7f0026766700 1 --2- 172.21.15.130:0/1204217702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020109f70 0x7f0020114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:13:39.884 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.871+0000 7f0026766700 1 -- 172.21.15.130:0/1204217702 >> 172.21.15.130:0/1204217702 conn(0x7f002006d3b0 msgr2=0x7f002010aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:13:39.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.871+0000 7f0026766700 1 -- 172.21.15.130:0/1204217702 shutdown_connections 2023-09-12T16:13:39.885 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:13:39.871+0000 7f0026766700 1 -- 172.21.15.130:0/1204217702 wait complete. 2023-09-12T16:13:39.888 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:13:40.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 470.0 s, 1258.2 tps, lat 79.640 ms stddev 190.334 2023-09-12T16:13:40.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:40 smithi159 ceph-mon[102522]: pgmap v614: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 36 MiB/s wr, 3.18k op/s 2023-09-12T16:13:40.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:40 smithi130 ceph-mon[98670]: pgmap v614: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 36 MiB/s wr, 3.18k op/s 2023-09-12T16:13:40.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:40 smithi203 ceph-mon[103431]: pgmap v614: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 36 MiB/s wr, 3.18k op/s 2023-09-12T16:13:42.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:42 smithi130 ceph-mon[98670]: pgmap v615: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 28 MiB/s wr, 2.62k op/s 2023-09-12T16:13:42.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:42 smithi159 ceph-mon[102522]: pgmap v615: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 28 MiB/s wr, 2.62k op/s 2023-09-12T16:13:42.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:42 smithi203 ceph-mon[103431]: pgmap v615: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 28 MiB/s wr, 2.62k op/s 2023-09-12T16:13:44.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:44 smithi203 ceph-mon[103431]: pgmap v616: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 30 MiB/s wr, 2.60k op/s 2023-09-12T16:13:44.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:44 smithi130 ceph-mon[98670]: pgmap v616: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 30 MiB/s wr, 2.60k op/s 2023-09-12T16:13:44.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:44 smithi159 ceph-mon[102522]: pgmap v616: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 30 MiB/s wr, 2.60k op/s 2023-09-12T16:13:45.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 475.0 s, 1464.0 tps, lat 67.238 ms stddev 83.320 2023-09-12T16:13:46.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:46 smithi159 ceph-mon[102522]: pgmap v617: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 23 MiB/s wr, 1.50k op/s 2023-09-12T16:13:46.981 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:46 smithi130 ceph-mon[98670]: pgmap v617: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 23 MiB/s wr, 1.50k op/s 2023-09-12T16:13:46.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:46 smithi203 ceph-mon[103431]: pgmap v617: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 23 MiB/s wr, 1.50k op/s 2023-09-12T16:13:48.505 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:48 smithi203 ceph-mon[103431]: pgmap v618: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 22 MiB/s wr, 1.28k op/s 2023-09-12T16:13:48.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:48 smithi159 ceph-mon[102522]: pgmap v618: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 22 MiB/s wr, 1.28k op/s 2023-09-12T16:13:48.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:48 smithi130 ceph-mon[98670]: pgmap v618: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 22 MiB/s wr, 1.28k op/s 2023-09-12T16:13:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 480.0 s, 659.6 tps, lat 139.774 ms stddev 216.841 2023-09-12T16:13:50.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:50 smithi130 ceph-mon[98670]: pgmap v619: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 1.33k op/s 2023-09-12T16:13:50.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:50 smithi159 ceph-mon[102522]: pgmap v619: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 1.33k op/s 2023-09-12T16:13:50.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:50 smithi203 ceph-mon[103431]: pgmap v619: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 1.33k op/s 2023-09-12T16:13:52.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:52 smithi159 ceph-mon[102522]: pgmap v620: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2023-09-12T16:13:52.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:52 smithi130 ceph-mon[98670]: pgmap v620: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2023-09-12T16:13:52.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:52 smithi203 ceph-mon[103431]: pgmap v620: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2023-09-12T16:13:54.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:54 smithi130 ceph-mon[98670]: pgmap v621: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 18 MiB/s wr, 1.07k op/s 2023-09-12T16:13:54.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:54 smithi159 ceph-mon[102522]: pgmap v621: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 18 MiB/s wr, 1.07k op/s 2023-09-12T16:13:54.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:54 smithi203 ceph-mon[103431]: pgmap v621: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 18 MiB/s wr, 1.07k op/s 2023-09-12T16:13:55.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 485.0 s, 915.8 tps, lat 117.082 ms stddev 149.963 2023-09-12T16:13:56.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:56 smithi130 ceph-mon[98670]: pgmap v622: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 16 MiB/s wr, 819 op/s 2023-09-12T16:13:56.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:56 smithi159 ceph-mon[102522]: pgmap v622: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 16 MiB/s wr, 819 op/s 2023-09-12T16:13:56.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:56 smithi203 ceph-mon[103431]: pgmap v622: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 16 MiB/s wr, 819 op/s 2023-09-12T16:13:58.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:13:58 smithi159 ceph-mon[102522]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 789 op/s 2023-09-12T16:13:58.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:13:58 smithi130 ceph-mon[98670]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 789 op/s 2023-09-12T16:13:58.500 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:13:58 smithi203 ceph-mon[103431]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 789 op/s 2023-09-12T16:14:00.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 490.0 s, 1739.1 tps, lat 57.976 ms stddev 52.438 2023-09-12T16:14:00.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:00 smithi130 ceph-mon[98670]: pgmap v624: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 17 MiB/s wr, 931 op/s 2023-09-12T16:14:00.734 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:00 smithi159 ceph-mon[102522]: pgmap v624: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 17 MiB/s wr, 931 op/s 2023-09-12T16:14:00.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:00 smithi203 ceph-mon[103431]: pgmap v624: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 17 MiB/s wr, 931 op/s 2023-09-12T16:14:02.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:02 smithi159 ceph-mon[102522]: pgmap v625: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 719 op/s 2023-09-12T16:14:02.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:02 smithi130 ceph-mon[98670]: pgmap v625: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 719 op/s 2023-09-12T16:14:02.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:02 smithi203 ceph-mon[103431]: pgmap v625: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 719 op/s 2023-09-12T16:14:04.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:04 smithi203 ceph-mon[103431]: pgmap v626: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 24 MiB/s wr, 1.51k op/s 2023-09-12T16:14:04.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:04 smithi159 ceph-mon[102522]: pgmap v626: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 24 MiB/s wr, 1.51k op/s 2023-09-12T16:14:04.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:04 smithi130 ceph-mon[98670]: pgmap v626: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 24 MiB/s wr, 1.51k op/s 2023-09-12T16:14:05.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 495.0 s, 238.5 tps, lat 340.726 ms stddev 526.773 2023-09-12T16:14:06.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:06 smithi130 ceph-mon[98670]: pgmap v627: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 26 MiB/s wr, 1.87k op/s 2023-09-12T16:14:06.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:06 smithi159 ceph-mon[102522]: pgmap v627: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 26 MiB/s wr, 1.87k op/s 2023-09-12T16:14:06.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:06 smithi203 ceph-mon[103431]: pgmap v627: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 26 MiB/s wr, 1.87k op/s 2023-09-12T16:14:08.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:08 smithi203 ceph-mon[103431]: pgmap v628: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 26 MiB/s wr, 2.16k op/s 2023-09-12T16:14:08.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:08 smithi159 ceph-mon[102522]: pgmap v628: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 26 MiB/s wr, 2.16k op/s 2023-09-12T16:14:08.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:08 smithi130 ceph-mon[98670]: pgmap v628: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 26 MiB/s wr, 2.16k op/s 2023-09-12T16:14:09.889 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:14:10.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 500.0 s, 842.0 tps, lat 142.602 ms stddev 318.639 2023-09-12T16:14:10.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.242+0000 7f8cab91b700 1 -- 172.21.15.130:0/876244324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4107600 msgr2=0x7f8ca4109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:10.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.242+0000 7f8cab91b700 1 --2- 172.21.15.130:0/876244324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4107600 0x7f8ca4109a30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00098f0 tx=0x7f8ca0009c00 comp rx=0 tx=0).stop 2023-09-12T16:14:10.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.242+0000 7f8cab91b700 1 -- 172.21.15.130:0/876244324 shutdown_connections 2023-09-12T16:14:10.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.242+0000 7f8cab91b700 1 --2- 172.21.15.130:0/876244324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ca410c940 0x7f8ca410cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.242+0000 7f8cab91b700 1 --2- 172.21.15.130:0/876244324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4107600 0x7f8ca4109a30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.242+0000 7f8cab91b700 1 --2- 172.21.15.130:0/876244324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ca4109f70 0x7f8ca410c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.242+0000 7f8cab91b700 1 -- 172.21.15.130:0/876244324 >> 172.21.15.130:0/876244324 conn(0x7f8ca406d3b0 msgr2=0x7f8ca406f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:10.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.242+0000 7f8cab91b700 1 -- 172.21.15.130:0/876244324 shutdown_connections 2023-09-12T16:14:10.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.242+0000 7f8cab91b700 1 -- 172.21.15.130:0/876244324 wait complete. 2023-09-12T16:14:10.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.243+0000 7f8cab91b700 1 Processor -- start 2023-09-12T16:14:10.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.245+0000 7f8cab91b700 1 -- start start 2023-09-12T16:14:10.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.245+0000 7f8cab91b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ca4107600 0x7f8ca4113c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.245+0000 7f8cab91b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4109f70 0x7f8ca4114170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.245+0000 7f8cab91b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ca410c940 0x7f8ca41b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.245+0000 7f8cab91b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4117250 con 0x7f8ca4107600 2023-09-12T16:14:10.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.245+0000 7f8cab91b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca41173c0 con 0x7f8ca4109f70 2023-09-12T16:14:10.247 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.245+0000 7f8cab91b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca41b57f0 con 0x7f8ca410c940 2023-09-12T16:14:10.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.245+0000 7f8ca9eb8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ca410c940 0x7f8ca41b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.248 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.245+0000 7f8ca9eb8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ca410c940 0x7f8ca41b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36034/0 (socket says 172.21.15.130:36034) 2023-09-12T16:14:10.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.246+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/3153261560 learned_addr learned my addr 172.21.15.130:0/3153261560 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:10.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.246+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ca4107600 0x7f8ca4113c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.246+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/3153261560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4109f70 msgr2=0x7f8ca4114170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-12T16:14:10.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.246+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4109f70 0x7f8ca4114170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.246+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ca4107600 msgr2=0x7f8ca4113c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:10.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.246+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ca4107600 0x7f8ca4113c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.246+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/3153261560 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca0009520 con 0x7f8ca410c940 2023-09-12T16:14:10.250 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.247+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ca410c940 0x7f8ca41b52b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c9a0 tx=0x7f8c9400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:10.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.247+0000 7f8c9a7fc700 1 -- 172.21.15.130:0/3153261560 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94007a60 con 0x7f8ca410c940 2023-09-12T16:14:10.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.247+0000 7f8c9a7fc700 1 -- 172.21.15.130:0/3153261560 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c9400ce90 con 0x7f8ca410c940 2023-09-12T16:14:10.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.247+0000 7f8c9a7fc700 1 -- 172.21.15.130:0/3153261560 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940053d0 con 0x7f8ca410c940 2023-09-12T16:14:10.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.247+0000 7f8cab91b700 1 -- 172.21.15.130:0/3153261560 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca41b5ad0 con 0x7f8ca410c940 2023-09-12T16:14:10.251 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.247+0000 7f8cab91b700 1 -- 172.21.15.130:0/3153261560 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca41b5f90 con 0x7f8ca410c940 2023-09-12T16:14:10.252 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.251+0000 7f8c9a7fc700 1 -- 172.21.15.130:0/3153261560 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f8c940055b0 con 0x7f8ca410c940 2023-09-12T16:14:10.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.251+0000 7f8c9a7fc700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8c900a06c0 0x7f8c900a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.254 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8c9a7fc700 1 -- 172.21.15.130:0/3153261560 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8c940cb920 con 0x7f8ca410c940 2023-09-12T16:14:10.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8c900a06c0 0x7f8c900a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8cb7ae1b80 1 Processor -- start 2023-09-12T16:14:10.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8cb7ae1b80 1 -- start start 2023-09-12T16:14:10.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 0x561e33f09020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f095a0 0x561e33f07800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.255 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33e60730 con 0x561e33f08c00 2023-09-12T16:14:10.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33e5e050 con 0x561e33f07e00 2023-09-12T16:14:10.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33e59940 con 0x561e33f095a0 2023-09-12T16:14:10.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.252+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8c900a06c0 0x7f8c900a2b80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0005e90 tx=0x7f8ca0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:10.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca96b7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 0x561e33f09020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca96b7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 0x561e33f09020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56428/0 (socket says 172.21.15.130:56428) 2023-09-12T16:14:10.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca96b7700 1 -- 172.21.15.130:0/3441376484 learned_addr learned my addr 172.21.15.130:0/3441376484 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:10.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca8eb6700 1 --2- 172.21.15.130:0/3441376484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f095a0 0x561e33f07800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/3441376484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca96b7700 1 -- 172.21.15.130:0/3441376484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f095a0 msgr2=0x561e33f07800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:10.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/3441376484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f095a0 0x561e33f07800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca96b7700 1 -- 172.21.15.130:0/3441376484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 msgr2=0x561e33f082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:10.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/3441376484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f082b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca96b7700 1 -- 172.21.15.130:0/3441376484 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e33c85de0 con 0x561e33f08c00 2023-09-12T16:14:10.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/3441376484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 0x561e33f09020 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0016000 tx=0x7f8ca001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6aeffb4fb8662f server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:10.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8c8f7fe700 1 -- 172.21.15.130:0/3441376484 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca002a070 con 0x561e33f08c00 2023-09-12T16:14:10.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8c8f7fe700 1 -- 172.21.15.130:0/3441376484 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ca001d040 con 0x561e33f08c00 2023-09-12T16:14:10.259 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.253+0000 7f8c8f7fe700 1 -- 172.21.15.130:0/3441376484 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00038e0 con 0x561e33f08c00 2023-09-12T16:14:10.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.254+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/3441376484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 msgr2=0x561e33f09020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:10.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.254+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/3441376484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 0x561e33f09020 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0016000 tx=0x7f8ca001e000 comp rx=0 tx=0).stop 2023-09-12T16:14:10.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.255+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/3441376484 shutdown_connections 2023-09-12T16:14:10.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.255+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/3441376484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.255+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/3441376484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 0x561e33f09020 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.255+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/3441376484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f095a0 0x561e33f07800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.255+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/3441376484 >> 172.21.15.130:0/3441376484 conn(0x561e33e7c090 msgr2=0x561e33f02720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:10.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.255+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/3441376484 shutdown_connections 2023-09-12T16:14:10.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.255+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/3441376484 wait complete. 2023-09-12T16:14:10.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.255+0000 7f8cb7ae1b80 1 -- start start 2023-09-12T16:14:10.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.255+0000 7f8cb7ae1b80 1 Processor -- start 2023-09-12T16:14:10.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f11560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f08c00 0x561e33f11aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33df7b70 con 0x561e33f095a0 2023-09-12T16:14:10.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33df7870 con 0x561e33f07e00 2023-09-12T16:14:10.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33cdb920 con 0x561e33f08c00 2023-09-12T16:14:10.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8cb7ae1b80 10 client.0 _ll_register_callbacks cb 0x561e33d41a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:14:10.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca96b7700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f11560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca96b7700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f11560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39692/0 (socket says 172.21.15.130:39692) 2023-09-12T16:14:10.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca96b7700 1 -- 172.21.15.130:0/877797052 learned_addr learned my addr 172.21.15.130:0/877797052 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:10.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca8eb6700 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f08c00 0x561e33f11aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca96b7700 1 -- 172.21.15.130:0/877797052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f08c00 msgr2=0x561e33f11aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:10.265 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f08c00 0x561e33f11aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca96b7700 1 -- 172.21.15.130:0/877797052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 msgr2=0x561e33f18ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:10.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f18ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.256+0000 7f8ca96b7700 1 -- 172.21.15.130:0/877797052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca410d940 con 0x561e33f07e00 2023-09-12T16:14:10.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.257+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f11560 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009810 tx=0x7f8ca0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee95d92525727a8f server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:10.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.257+0000 7f8c8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:14:10.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.257+0000 7f8c8cff9700 1 -- 172.21.15.130:0/877797052 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca002a030 con 0x561e33f07e00 2023-09-12T16:14:10.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.257+0000 7f8c8cff9700 1 -- 172.21.15.130:0/877797052 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8ca0005280 con 0x561e33f07e00 2023-09-12T16:14:10.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.257+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/877797052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e33db14f0 con 0x561e33f07e00 2023-09-12T16:14:10.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.260+0000 7f8c8cff9700 1 -- 172.21.15.130:0/877797052 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ca0003880 con 0x561e33f07e00 2023-09-12T16:14:10.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.261+0000 7f8cb7ae1b80 10 client.25574 fetch_fsmap learned FSMap version 32 2023-09-12T16:14:10.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.261+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/877797052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561e33c6ec60 con 0x561e33f07e00 2023-09-12T16:14:10.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.272+0000 7f8c8cff9700 1 -- 172.21.15.130:0/877797052 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e33db14f0 con 0x561e33f07e00 2023-09-12T16:14:10.272 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.272+0000 7f8c8cff9700 1 -- 172.21.15.130:0/877797052 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8ca0037b00 con 0x561e33f07e00 2023-09-12T16:14:10.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.272+0000 7f8cb7ae1b80 10 client.25574 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:14:10.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.272+0000 7f8cb7ae1b80 10 client.25574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:14:10.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.272+0000 7f8cb7ae1b80 20 client.25574 populate_metadata read hostname 'smithi130' 2023-09-12T16:14:10.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.272+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561e33f1ce40 0x561e33f1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.273 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.272+0000 7f8cb7ae1b80 4 client.25574 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:14:10.274 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.272+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/877797052 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e33e6e8a0 con 0x561e33f1ce40 2023-09-12T16:14:10.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.279+0000 7f8ca8eb6700 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561e33f1ce40 0x561e33f1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.305+0000 7f8ca8eb6700 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561e33f1ce40 0x561e33f1f260 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20b26b5ec0bba8d server_cookie=f013895a64f2cc0e in_seq=0 out_seq=0 2023-09-12T16:14:10.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.305+0000 7f8c8cff9700 10 client.25574 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:14:10.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.307+0000 7f8c8cff9700 1 -- 172.21.15.130:0/877797052 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561e33e6e8a0 con 0x561e33f1ce40 2023-09-12T16:14:10.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.307+0000 7f8c8cff9700 10 client.25574 handle_command_reply: tid=0 2023-09-12T16:14:10.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.308+0000 7f8cb7ae1b80 1 client.25574 shutdown 2023-09-12T16:14:10.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.317+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/877797052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 msgr2=0x561e33f11560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:10.317 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.317+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f11560 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009810 tx=0x7f8ca0005dc0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.318+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/877797052 shutdown_connections 2023-09-12T16:14:10.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.318+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561e33f1ce40 0x561e33f1f260 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.319 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.318+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f18ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.318+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f07e00 0x561e33f11560 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.318+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/877797052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f08c00 0x561e33f11aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.318+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/877797052 >> 172.21.15.130:0/877797052 conn(0x561e33e7c090 msgr2=0x561e33f01dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:10.320 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.319+0000 7f8c8cff9700 0 client.25574 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:14:10.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.319+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/877797052 shutdown_connections 2023-09-12T16:14:10.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.319+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/877797052 wait complete. 2023-09-12T16:14:10.321 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.319+0000 7f8cb7ae1b80 20 client.25574 trim_cache size 0 max 16384 2023-09-12T16:14:10.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.361+0000 7f8cb7ae1b80 1 Processor -- start 2023-09-12T16:14:10.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.361+0000 7f8cb7ae1b80 1 -- start start 2023-09-12T16:14:10.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.361+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f07e00 0x561e33f07710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.361+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 0x561e33f18aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.363 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.361+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f095a0 0x561e33f1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.361+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f07c50 con 0x561e33f08c00 2023-09-12T16:14:10.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.361+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f110b0 con 0x561e33f095a0 2023-09-12T16:14:10.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.361+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33ecc850 con 0x561e33f07e00 2023-09-12T16:14:10.364 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.362+0000 7f8ca9eb8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f095a0 0x561e33f1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.362+0000 7f8ca9eb8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f095a0 0x561e33f1ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39700/0 (socket says 172.21.15.130:39700) 2023-09-12T16:14:10.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.362+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/1545835664 learned_addr learned my addr 172.21.15.130:0/1545835664 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:10.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.362+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/1545835664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f07e00 msgr2=0x561e33f07710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:14:10.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.362+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/1545835664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f07e00 0x561e33f07710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.362+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/1545835664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 msgr2=0x561e33f18aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:10.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.362+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/1545835664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 0x561e33f18aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.362+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/1545835664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e33cac250 con 0x561e33f095a0 2023-09-12T16:14:10.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.362+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/1545835664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f095a0 0x561e33f1ee50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c94098aa0 tx=0x7f8c940cb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e810031ac63b3555 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:10.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.363+0000 7f8c8ffff700 1 -- 172.21.15.130:0/1545835664 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940b1160 con 0x561e33f095a0 2023-09-12T16:14:10.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.363+0000 7f8c8ffff700 1 -- 172.21.15.130:0/1545835664 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c94097140 con 0x561e33f095a0 2023-09-12T16:14:10.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.363+0000 7f8c8ffff700 1 -- 172.21.15.130:0/1545835664 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940984d0 con 0x561e33f095a0 2023-09-12T16:14:10.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.364+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/1545835664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f095a0 msgr2=0x561e33f1ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:10.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.364+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/1545835664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f095a0 0x561e33f1ee50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c94098aa0 tx=0x7f8c940cb040 comp rx=0 tx=0).stop 2023-09-12T16:14:10.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.366+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/1545835664 shutdown_connections 2023-09-12T16:14:10.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.366+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/1545835664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f095a0 0x561e33f1ee50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.366+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/1545835664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f07e00 0x561e33f07710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.366+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/1545835664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f08c00 0x561e33f18aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.368 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.366+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/1545835664 >> 172.21.15.130:0/1545835664 conn(0x561e33e7c090 msgr2=0x561e33f01dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:10.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.373+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/1545835664 shutdown_connections 2023-09-12T16:14:10.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.373+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/1545835664 wait complete. 2023-09-12T16:14:10.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.373+0000 7f8cb7ae1b80 1 -- start start 2023-09-12T16:14:10.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.373+0000 7f8cb7ae1b80 1 Processor -- start 2023-09-12T16:14:10.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.374+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f07e00 0x561e33f18710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.374+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f08c00 0x561e33f151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.374+0000 7f8cb7ae1b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.374+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f15c30 con 0x561e33f095a0 2023-09-12T16:14:10.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.374+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f15da0 con 0x561e33f08c00 2023-09-12T16:14:10.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.374+0000 7f8cb7ae1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f70c20 con 0x561e33f07e00 2023-09-12T16:14:10.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.374+0000 7f8cb7ae1b80 10 client.0 _ll_register_callbacks cb 0x7f8c94095100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:14:10.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.375+0000 7f8ca9eb8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.375+0000 7f8ca9eb8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f156f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56450/0 (socket says 172.21.15.130:56450) 2023-09-12T16:14:10.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.375+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/433121275 learned_addr learned my addr 172.21.15.130:0/433121275 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:10.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.376+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/433121275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f07e00 msgr2=0x561e33f18710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:14:10.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.376+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f07e00 0x561e33f18710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.376+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/433121275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f08c00 msgr2=0x561e33f151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:10.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.376+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f08c00 0x561e33f151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.376+0000 7f8ca9eb8700 1 -- 172.21.15.130:0/433121275 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca41b5f90 con 0x561e33f095a0 2023-09-12T16:14:10.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.376+0000 7f8ca9eb8700 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f156f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007eb0 tx=0x7f8c94099000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bb9f0fc0a3f134 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:10.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.376+0000 7f8c8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:14:10.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.377+0000 7f8c8cff9700 1 -- 172.21.15.130:0/433121275 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940a0b80 con 0x561e33f095a0 2023-09-12T16:14:10.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.377+0000 7f8c8cff9700 1 -- 172.21.15.130:0/433121275 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8c94094b90 con 0x561e33f095a0 2023-09-12T16:14:10.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.377+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/433121275 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ca001d040 con 0x561e33f095a0 2023-09-12T16:14:10.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.378+0000 7f8c8cff9700 1 -- 172.21.15.130:0/433121275 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c94094d50 con 0x561e33f095a0 2023-09-12T16:14:10.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8cb7ae1b80 10 client.16110 fetch_fsmap learned FSMap version 32 2023-09-12T16:14:10.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/433121275 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561e33c842b0 con 0x561e33f095a0 2023-09-12T16:14:10.387 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8c8cff9700 1 -- 172.21.15.130:0/433121275 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ca001d040 con 0x561e33f095a0 2023-09-12T16:14:10.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8c8cff9700 1 -- 172.21.15.130:0/433121275 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8c940b8060 con 0x561e33f095a0 2023-09-12T16:14:10.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8cb7ae1b80 10 client.16110 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:14:10.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8cb7ae1b80 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:14:10.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8cb7ae1b80 20 client.16110 populate_metadata read hostname 'smithi130' 2023-09-12T16:14:10.388 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561e33f720c0 0x561e33f744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:10.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8cb7ae1b80 4 client.16110 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:14:10.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.380+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/433121275 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561e33e52920 con 0x561e33f720c0 2023-09-12T16:14:10.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561e33f720c0 0x561e33f744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:10.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8ca96b7700 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561e33f720c0 0x561e33f744e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c95e24d56236ea7 server_cookie=89e3198491b686c8 in_seq=0 out_seq=0 2023-09-12T16:14:10.389 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8c8cff9700 10 client.16110 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:14:10.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8c8cff9700 1 -- 172.21.15.130:0/433121275 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561e33e52920 con 0x561e33f720c0 2023-09-12T16:14:10.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8c8cff9700 10 client.16110 handle_command_reply: tid=0 2023-09-12T16:14:10.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8cb7ae1b80 1 client.16110 shutdown 2023-09-12T16:14:10.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/433121275 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 msgr2=0x561e33f156f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:10.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f156f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007eb0 tx=0x7f8c94099000 comp rx=0 tx=0).stop 2023-09-12T16:14:10.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/433121275 shutdown_connections 2023-09-12T16:14:10.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.381+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x561e33f720c0 0x561e33f744e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.382+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e33f095a0 0x561e33f156f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.391 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.382+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e33f07e00 0x561e33f18710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.382+0000 7f8cb7ae1b80 1 --2- 172.21.15.130:0/433121275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e33f08c00 0x561e33f151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.382+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/433121275 >> 172.21.15.130:0/433121275 conn(0x561e33e7c090 msgr2=0x561e33f1ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:10.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.382+0000 7f8c8cff9700 0 client.16110 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:14:10.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.382+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/433121275 shutdown_connections 2023-09-12T16:14:10.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.386+0000 7f8cb7ae1b80 1 -- 172.21.15.130:0/433121275 wait complete. 2023-09-12T16:14:10.392 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.386+0000 7f8cb7ae1b80 20 client.16110 trim_cache size 0 max 16384 2023-09-12T16:14:10.393 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:14:10.393 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:14:10.393 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:14:10.394 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:14:10.394 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:14:10.394 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:14:10.394 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:14:10.394 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:14:10.395 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:14:10.395 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:14:10.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.386+0000 7f8cab91b700 1 -- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8c900a06c0 msgr2=0x7f8c900a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:10.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.386+0000 7f8cab91b700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8c900a06c0 0x7f8c900a2b80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0005e90 tx=0x7f8ca0019040 comp rx=0 tx=0).stop 2023-09-12T16:14:10.396 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.386+0000 7f8cab91b700 1 -- 172.21.15.130:0/3153261560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ca410c940 msgr2=0x7f8ca41b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:10.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.386+0000 7f8cab91b700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ca410c940 0x7f8ca41b52b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c9a0 tx=0x7f8c9400ccb0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.387+0000 7f8cab91b700 1 -- 172.21.15.130:0/3153261560 shutdown_connections 2023-09-12T16:14:10.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.387+0000 7f8cab91b700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f8c900a06c0 0x7f8c900a2b80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.387+0000 7f8cab91b700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ca410c940 0x7f8ca41b52b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.397 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.387+0000 7f8cab91b700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ca4107600 0x7f8ca4113c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.387+0000 7f8cab91b700 1 --2- 172.21.15.130:0/3153261560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4109f70 0x7f8ca4114170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:10.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.387+0000 7f8cab91b700 1 -- 172.21.15.130:0/3153261560 >> 172.21.15.130:0/3153261560 conn(0x7f8ca406d3b0 msgr2=0x7f8ca410b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:10.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.387+0000 7f8cab91b700 1 -- 172.21.15.130:0/3153261560 shutdown_connections 2023-09-12T16:14:10.398 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:10.387+0000 7f8cab91b700 1 -- 172.21.15.130:0/3153261560 wait complete. 2023-09-12T16:14:10.410 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:14:10.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:10 smithi130 ceph-mon[98670]: pgmap v629: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 36 MiB/s wr, 3.24k op/s 2023-09-12T16:14:10.739 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:10 smithi159 ceph-mon[102522]: pgmap v629: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 36 MiB/s wr, 3.24k op/s 2023-09-12T16:14:10.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:10 smithi203 ceph-mon[103431]: pgmap v629: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 36 MiB/s wr, 3.24k op/s 2023-09-12T16:14:12.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:12 smithi159 ceph-mon[102522]: pgmap v630: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 31 MiB/s wr, 3.10k op/s 2023-09-12T16:14:12.980 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:12 smithi130 ceph-mon[98670]: pgmap v630: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 31 MiB/s wr, 3.10k op/s 2023-09-12T16:14:12.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:12 smithi203 ceph-mon[103431]: pgmap v630: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 31 MiB/s wr, 3.10k op/s 2023-09-12T16:14:14.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:14 smithi130 ceph-mon[98670]: pgmap v631: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 35 MiB/s wr, 3.34k op/s 2023-09-12T16:14:14.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:14 smithi159 ceph-mon[102522]: pgmap v631: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 35 MiB/s wr, 3.34k op/s 2023-09-12T16:14:14.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:14 smithi203 ceph-mon[103431]: pgmap v631: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 35 MiB/s wr, 3.34k op/s 2023-09-12T16:14:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 505.0 s, 1401.6 tps, lat 71.292 ms stddev 76.186 2023-09-12T16:14:16.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:16 smithi130 ceph-mon[98670]: pgmap v632: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 27 MiB/s wr, 2.62k op/s 2023-09-12T16:14:16.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:16 smithi159 ceph-mon[102522]: pgmap v632: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 27 MiB/s wr, 2.62k op/s 2023-09-12T16:14:16.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:16 smithi203 ceph-mon[103431]: pgmap v632: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 27 MiB/s wr, 2.62k op/s 2023-09-12T16:14:18.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:18 smithi203 ceph-mon[103431]: pgmap v633: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 26 MiB/s wr, 2.21k op/s 2023-09-12T16:14:18.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:18 smithi130 ceph-mon[98670]: pgmap v633: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 26 MiB/s wr, 2.21k op/s 2023-09-12T16:14:18.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:18 smithi159 ceph-mon[102522]: pgmap v633: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 26 MiB/s wr, 2.21k op/s 2023-09-12T16:14:20.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 510.0 s, 1715.4 tps, lat 55.534 ms stddev 45.027 2023-09-12T16:14:20.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:20 smithi159 ceph-mon[102522]: pgmap v634: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 28 MiB/s wr, 2.04k op/s 2023-09-12T16:14:20.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:20 smithi130 ceph-mon[98670]: pgmap v634: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 28 MiB/s wr, 2.04k op/s 2023-09-12T16:14:20.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:20 smithi203 ceph-mon[103431]: pgmap v634: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 28 MiB/s wr, 2.04k op/s 2023-09-12T16:14:22.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:22 smithi130 ceph-mon[98670]: pgmap v635: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 920 op/s 2023-09-12T16:14:22.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:22 smithi159 ceph-mon[102522]: pgmap v635: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 920 op/s 2023-09-12T16:14:22.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:22 smithi203 ceph-mon[103431]: pgmap v635: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 920 op/s 2023-09-12T16:14:24.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:24 smithi130 ceph-mon[98670]: pgmap v636: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 1.04k op/s 2023-09-12T16:14:24.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:24 smithi159 ceph-mon[102522]: pgmap v636: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 1.04k op/s 2023-09-12T16:14:24.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:24 smithi203 ceph-mon[103431]: pgmap v636: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 1.04k op/s 2023-09-12T16:14:25.098 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 515.0 s, 555.1 tps, lat 188.007 ms stddev 208.987 2023-09-12T16:14:26.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:26 smithi130 ceph-mon[98670]: pgmap v637: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 883 op/s 2023-09-12T16:14:26.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:26 smithi159 ceph-mon[102522]: pgmap v637: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 883 op/s 2023-09-12T16:14:26.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:26 smithi203 ceph-mon[103431]: pgmap v637: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 883 op/s 2023-09-12T16:14:28.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:28 smithi203 ceph-mon[103431]: pgmap v638: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 840 op/s 2023-09-12T16:14:28.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:28 smithi159 ceph-mon[102522]: pgmap v638: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 840 op/s 2023-09-12T16:14:28.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:28 smithi130 ceph-mon[98670]: pgmap v638: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 840 op/s 2023-09-12T16:14:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 520.0 s, 1271.4 tps, lat 77.770 ms stddev 65.366 2023-09-12T16:14:30.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:30 smithi130 ceph-mon[98670]: pgmap v639: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 905 op/s 2023-09-12T16:14:30.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:30 smithi159 ceph-mon[102522]: pgmap v639: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 905 op/s 2023-09-12T16:14:30.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:30 smithi203 ceph-mon[103431]: pgmap v639: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 905 op/s 2023-09-12T16:14:32.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:32 smithi130 ceph-mon[98670]: pgmap v640: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 702 op/s 2023-09-12T16:14:32.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:32 smithi159 ceph-mon[102522]: pgmap v640: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 702 op/s 2023-09-12T16:14:32.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:32 smithi203 ceph-mon[103431]: pgmap v640: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 702 op/s 2023-09-12T16:14:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:33 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:14:33.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:33 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:14:33.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:33 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:14:34.980 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:34 smithi130 ceph-mon[98670]: pgmap v641: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 956 op/s 2023-09-12T16:14:34.984 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:34 smithi159 ceph-mon[102522]: pgmap v641: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 956 op/s 2023-09-12T16:14:34.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:34 smithi203 ceph-mon[103431]: pgmap v641: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 956 op/s 2023-09-12T16:14:35.097 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 525.0 s, 1209.0 tps, lat 71.544 ms stddev 81.452 2023-09-12T16:14:36.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:36 smithi159 ceph-mon[102522]: pgmap v642: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 17 MiB/s wr, 940 op/s 2023-09-12T16:14:36.980 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:36 smithi130 ceph-mon[98670]: pgmap v642: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 17 MiB/s wr, 940 op/s 2023-09-12T16:14:36.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:36 smithi203 ceph-mon[103431]: pgmap v642: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 17 MiB/s wr, 940 op/s 2023-09-12T16:14:38.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:38 smithi130 ceph-mon[98670]: pgmap v643: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 20 MiB/s wr, 1.28k op/s 2023-09-12T16:14:38.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:38 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:38.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:38 smithi159 ceph-mon[102522]: pgmap v643: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 20 MiB/s wr, 1.28k op/s 2023-09-12T16:14:38.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:38 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:38.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:38 smithi203 ceph-mon[103431]: pgmap v643: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 20 MiB/s wr, 1.28k op/s 2023-09-12T16:14:38.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:38 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:39 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:39 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:39.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:39 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:40.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 530.0 s, 119.9 tps, lat 863.007 ms stddev 799.111 2023-09-12T16:14:40.411 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:14:40.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:40.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:40 smithi159 ceph-mon[102522]: pgmap v644: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 26 MiB/s wr, 2.06k op/s 2023-09-12T16:14:40.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:14:40.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:14:40.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:40 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:40.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:40.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:40 smithi130 ceph-mon[98670]: pgmap v644: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 26 MiB/s wr, 2.06k op/s 2023-09-12T16:14:40.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:14:40.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:14:40.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:40 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:40.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:40.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:40 smithi203 ceph-mon[103431]: pgmap v644: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 26 MiB/s wr, 2.06k op/s 2023-09-12T16:14:40.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:14:40.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:14:40.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:40 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:14:40.807 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.804+0000 7f50331fa700 1 -- 172.21.15.130:0/1350084229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 msgr2=0x7f502c10c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:40.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.804+0000 7f50331fa700 1 --2- 172.21.15.130:0/1350084229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 0x7f502c10c400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5020009990 tx=0x7f5020009ca0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.808+0000 7f50331fa700 1 -- 172.21.15.130:0/1350084229 shutdown_connections 2023-09-12T16:14:40.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.808+0000 7f50331fa700 1 --2- 172.21.15.130:0/1350084229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f502c10c940 0x7f502c10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.808+0000 7f50331fa700 1 --2- 172.21.15.130:0/1350084229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f502c107600 0x7f502c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.808+0000 7f50331fa700 1 --2- 172.21.15.130:0/1350084229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 0x7f502c10c400 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.808+0000 7f50331fa700 1 -- 172.21.15.130:0/1350084229 >> 172.21.15.130:0/1350084229 conn(0x7f502c06d3b0 msgr2=0x7f502c06f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:40.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.815+0000 7f50331fa700 1 -- 172.21.15.130:0/1350084229 shutdown_connections 2023-09-12T16:14:40.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.825+0000 7f50331fa700 1 -- 172.21.15.130:0/1350084229 wait complete. 2023-09-12T16:14:40.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.825+0000 7f50331fa700 1 Processor -- start 2023-09-12T16:14:40.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.828+0000 7f50331fa700 1 -- start start 2023-09-12T16:14:40.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.828+0000 7f50331fa700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f502c107600 0x7f502c1a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.828+0000 7f50331fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 0x7f502c1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.828+0000 7f50331fa700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f502c10c940 0x7f502c1ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.828+0000 7f50331fa700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1acc60 con 0x7f502c10c940 2023-09-12T16:14:40.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.828+0000 7f50331fa700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c10ed30 con 0x7f502c107600 2023-09-12T16:14:40.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.828+0000 7f50331fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c10eec0 con 0x7f502c109f70 2023-09-12T16:14:40.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.830+0000 7f502bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 0x7f502c1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.830+0000 7f502bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 0x7f502c1a5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36078/0 (socket says 172.21.15.130:36078) 2023-09-12T16:14:40.831 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.830+0000 7f502bfff700 1 -- 172.21.15.130:0/1191296727 learned_addr learned my addr 172.21.15.130:0/1191296727 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:40.832 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.832+0000 7f5030f96700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f502c107600 0x7f502c1a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.838+0000 7f5031797700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f502c10c940 0x7f502c1ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.840+0000 7f502bfff700 1 -- 172.21.15.130:0/1191296727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f502c107600 msgr2=0x7f502c1a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:40.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.840+0000 7f502bfff700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f502c107600 0x7f502c1a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.840+0000 7f502bfff700 1 -- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f502c10c940 msgr2=0x7f502c1ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:40.840 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.840+0000 7f502bfff700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f502c10c940 0x7f502c1ac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.841 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.840+0000 7f502bfff700 1 -- 172.21.15.130:0/1191296727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50200095f0 con 0x7f502c109f70 2023-09-12T16:14:40.851 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.843+0000 7f502bfff700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 0x7f502c1a5620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f502000f690 tx=0x7f502000f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:40.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.852+0000 7f5029ffb700 1 -- 172.21.15.130:0/1191296727 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502001d070 con 0x7f502c109f70 2023-09-12T16:14:40.860 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.852+0000 7f5029ffb700 1 -- 172.21.15.130:0/1191296727 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f502000be40 con 0x7f502c109f70 2023-09-12T16:14:40.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.852+0000 7f5029ffb700 1 -- 172.21.15.130:0/1191296727 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502000e640 con 0x7f502c109f70 2023-09-12T16:14:40.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.852+0000 7f50331fa700 1 -- 172.21.15.130:0/1191296727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c10f140 con 0x7f502c109f70 2023-09-12T16:14:40.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.852+0000 7f50331fa700 1 -- 172.21.15.130:0/1191296727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c10f5a0 con 0x7f502c109f70 2023-09-12T16:14:40.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.860+0000 7f503f3c0b80 1 Processor -- start 2023-09-12T16:14:40.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.860+0000 7f503f3c0b80 1 -- start start 2023-09-12T16:14:40.861 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.861+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.861+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.861+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.861+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa9105730 con 0x558fa91ae5a0 2023-09-12T16:14:40.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.861+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa9103050 con 0x558fa91adc00 2023-09-12T16:14:40.862 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.861+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa90fe940 con 0x558fa91ace00 2023-09-12T16:14:40.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.862+0000 7f5030f96700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.862+0000 7f5030f96700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56470/0 (socket says 172.21.15.130:56470) 2023-09-12T16:14:40.868 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.862+0000 7f5030f96700 1 -- 172.21.15.130:0/1813636023 learned_addr learned my addr 172.21.15.130:0/1813636023 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:40.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.871+0000 7f5031797700 1 --2- 172.21.15.130:0/1813636023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.875+0000 7f502bfff700 1 --2- 172.21.15.130:0/1813636023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.877 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.876+0000 7f5029ffb700 1 -- 172.21.15.130:0/1191296727 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f50200173f0 con 0x7f502c109f70 2023-09-12T16:14:40.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.877+0000 7f5029ffb700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50140a0790 0x7f50140a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.878 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.877+0000 7f5029ffb700 1 -- 172.21.15.130:0/1191296727 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f50200d3100 con 0x7f502c109f70 2023-09-12T16:14:40.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.880+0000 7f5030f96700 1 -- 172.21.15.130:0/1813636023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 msgr2=0x558fa91ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:40.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.880+0000 7f5030f96700 1 --2- 172.21.15.130:0/1813636023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.881 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.880+0000 7f5030f96700 1 -- 172.21.15.130:0/1813636023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 msgr2=0x558fa91ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:40.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.880+0000 7f5030f96700 1 --2- 172.21.15.130:0/1813636023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.882 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.880+0000 7f5030f96700 1 -- 172.21.15.130:0/1813636023 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fa8f2ade0 con 0x558fa91ae5a0 2023-09-12T16:14:40.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.891+0000 7f5031797700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50140a0790 0x7f50140a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.892 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.891+0000 7f5030f96700 1 --2- 172.21.15.130:0/1813636023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91ac800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f501c009d80 tx=0x7f501c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea391b2eed80e01 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:40.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.896+0000 7f5012ffd700 1 -- 172.21.15.130:0/1813636023 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c004230 con 0x558fa91ae5a0 2023-09-12T16:14:40.900 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.897+0000 7f5012ffd700 1 -- 172.21.15.130:0/1813636023 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f501c009e10 con 0x558fa91ae5a0 2023-09-12T16:14:40.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.897+0000 7f5012ffd700 1 -- 172.21.15.130:0/1813636023 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c019880 con 0x558fa91ae5a0 2023-09-12T16:14:40.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.897+0000 7f503f3c0b80 1 -- 172.21.15.130:0/1813636023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 msgr2=0x558fa91ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:40.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.897+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/1813636023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91ac800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f501c009d80 tx=0x7f501c011720 comp rx=0 tx=0).stop 2023-09-12T16:14:40.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.898+0000 7f503f3c0b80 1 -- 172.21.15.130:0/1813636023 shutdown_connections 2023-09-12T16:14:40.901 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.898+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/1813636023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.898+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/1813636023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.898+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/1813636023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91ac800 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.902 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.898+0000 7f503f3c0b80 1 -- 172.21.15.130:0/1813636023 >> 172.21.15.130:0/1813636023 conn(0x558fa9121090 msgr2=0x558fa91a7720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:40.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.904+0000 7f503f3c0b80 1 -- 172.21.15.130:0/1813636023 shutdown_connections 2023-09-12T16:14:40.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.905+0000 7f5031797700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50140a0790 0x7f50140a2c50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5024009da0 tx=0x7f502400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:40.906 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.906+0000 7f503f3c0b80 1 -- 172.21.15.130:0/1813636023 wait complete. 2023-09-12T16:14:40.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.907+0000 7f503f3c0b80 1 -- start start 2023-09-12T16:14:40.907 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.907+0000 7f503f3c0b80 1 Processor -- start 2023-09-12T16:14:40.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.909+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.909 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91d8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa909cb70 con 0x558fa91ae5a0 2023-09-12T16:14:40.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa909c870 con 0x558fa91adc00 2023-09-12T16:14:40.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa8f80920 con 0x558fa91ace00 2023-09-12T16:14:40.910 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f503f3c0b80 10 client.0 _ll_register_callbacks cb 0x558fa8fe6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:14:40.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f5030f96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f5030f96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91d85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36090/0 (socket says 172.21.15.130:36090) 2023-09-12T16:14:40.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f5030f96700 1 -- 172.21.15.130:0/459983355 learned_addr learned my addr 172.21.15.130:0/459983355 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:40.911 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f5030f96700 1 -- 172.21.15.130:0/459983355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 msgr2=0x558fa91d8b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:14:40.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f5030f96700 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91d8b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f5030f96700 1 -- 172.21.15.130:0/459983355 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 msgr2=0x558fa91dfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:40.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f5030f96700 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.910+0000 7f5030f96700 1 -- 172.21.15.130:0/459983355 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c0099e0 con 0x558fa91ace00 2023-09-12T16:14:40.912 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.911+0000 7f5030f96700 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91d85e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f501c01b040 tx=0x7f501c004280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9ee7d1748f701cc server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:40.919 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.911+0000 7f500bfff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:14:40.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.921+0000 7f500bfff700 1 -- 172.21.15.130:0/459983355 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c005240 con 0x558fa91ace00 2023-09-12T16:14:40.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.921+0000 7f500bfff700 1 -- 172.21.15.130:0/459983355 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f501c0053a0 con 0x558fa91ace00 2023-09-12T16:14:40.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.921+0000 7f503f3c0b80 1 -- 172.21.15.130:0/459983355 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fa90564f0 con 0x558fa91ace00 2023-09-12T16:14:40.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.923+0000 7f500bfff700 1 -- 172.21.15.130:0/459983355 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f501c003d60 con 0x558fa91ace00 2023-09-12T16:14:40.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f503f3c0b80 10 client.25804 fetch_fsmap learned FSMap version 32 2023-09-12T16:14:40.923 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f503f3c0b80 1 -- 172.21.15.130:0/459983355 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558fa8f13c60 con 0x558fa91ace00 2023-09-12T16:14:40.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f500bfff700 1 -- 172.21.15.130:0/459983355 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fa90564f0 con 0x558fa91ace00 2023-09-12T16:14:40.924 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f500bfff700 1 -- 172.21.15.130:0/459983355 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f501c021040 con 0x558fa91ace00 2023-09-12T16:14:40.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f503f3c0b80 10 client.25804 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:14:40.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f503f3c0b80 10 client.25804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:14:40.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f503f3c0b80 20 client.25804 populate_metadata read hostname 'smithi130' 2023-09-12T16:14:40.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558fa91e4380 0x558fa91e67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.925 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f503f3c0b80 4 client.25804 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:14:40.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.924+0000 7f503f3c0b80 1 -- 172.21.15.130:0/459983355 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fa91138a0 con 0x558fa91e4380 2023-09-12T16:14:40.926 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.925+0000 7f502bfff700 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558fa91e4380 0x558fa91e67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.943+0000 7f502bfff700 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558fa91e4380 0x558fa91e67a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a59828c4baeb354 server_cookie=df10344945742f0b in_seq=0 out_seq=0 2023-09-12T16:14:40.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.943+0000 7f500bfff700 10 client.25804 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:14:40.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.944+0000 7f500bfff700 1 -- 172.21.15.130:0/459983355 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558fa91138a0 con 0x558fa91e4380 2023-09-12T16:14:40.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.944+0000 7f500bfff700 10 client.25804 handle_command_reply: tid=0 2023-09-12T16:14:40.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.944+0000 7f503f3c0b80 1 client.25804 shutdown 2023-09-12T16:14:40.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 -- 172.21.15.130:0/459983355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 msgr2=0x558fa91d85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:40.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91d85e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f501c01b040 tx=0x7f501c004280 comp rx=0 tx=0).stop 2023-09-12T16:14:40.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 -- 172.21.15.130:0/459983355 shutdown_connections 2023-09-12T16:14:40.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558fa91e4380 0x558fa91e67a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91dfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91d85e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/459983355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91d8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 -- 172.21.15.130:0/459983355 >> 172.21.15.130:0/459983355 conn(0x558fa9121090 msgr2=0x558fa91a6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:40.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f500bfff700 0 client.25804 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:14:40.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 -- 172.21.15.130:0/459983355 shutdown_connections 2023-09-12T16:14:40.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 1 -- 172.21.15.130:0/459983355 wait complete. 2023-09-12T16:14:40.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.945+0000 7f503f3c0b80 20 client.25804 trim_cache size 0 max 16384 2023-09-12T16:14:40.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.973+0000 7f503f3c0b80 1 Processor -- start 2023-09-12T16:14:40.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f503f3c0b80 1 -- start start 2023-09-12T16:14:40.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ace00 0x558fa91ac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91adc00 0x558fa91dfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91ae5a0 0x558fa91a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa9056520 con 0x558fa91ace00 2023-09-12T16:14:40.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa919f430 con 0x558fa91ae5a0 2023-09-12T16:14:40.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa91d7130 con 0x558fa91adc00 2023-09-12T16:14:40.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f5030f96700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ace00 0x558fa91ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f5030f96700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ace00 0x558fa91ac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56486/0 (socket says 172.21.15.130:56486) 2023-09-12T16:14:40.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.974+0000 7f5030f96700 1 -- 172.21.15.130:0/2917741899 learned_addr learned my addr 172.21.15.130:0/2917741899 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:40.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.975+0000 7f5031797700 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91ae5a0 0x558fa91a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.975+0000 7f502bfff700 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91adc00 0x558fa91dfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.975+0000 7f502bfff700 1 -- 172.21.15.130:0/2917741899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91ae5a0 msgr2=0x558fa91a6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:40.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.975+0000 7f502bfff700 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91ae5a0 0x558fa91a6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.975+0000 7f502bfff700 1 -- 172.21.15.130:0/2917741899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ace00 msgr2=0x558fa91ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:40.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.975+0000 7f502bfff700 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ace00 0x558fa91ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.975+0000 7f502bfff700 1 -- 172.21.15.130:0/2917741899 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fa91eb3d0 con 0x558fa91adc00 2023-09-12T16:14:40.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.976+0000 7f5031797700 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91ae5a0 0x558fa91a6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:14:40.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.976+0000 7f502bfff700 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91adc00 0x558fa91dfb20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f502000b2e0 tx=0x7f50200d06e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6b67ac03192c5c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:40.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.976+0000 7f50137fe700 1 -- 172.21.15.130:0/2917741899 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502009fef0 con 0x558fa91adc00 2023-09-12T16:14:40.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.976+0000 7f50137fe700 1 -- 172.21.15.130:0/2917741899 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50200a0c90 con 0x558fa91adc00 2023-09-12T16:14:40.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.976+0000 7f50137fe700 1 -- 172.21.15.130:0/2917741899 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502009e930 con 0x558fa91adc00 2023-09-12T16:14:40.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.977+0000 7f503f3c0b80 1 -- 172.21.15.130:0/2917741899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91adc00 msgr2=0x558fa91dfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:40.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.977+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91adc00 0x558fa91dfb20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f502000b2e0 tx=0x7f50200d06e0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.978+0000 7f503f3c0b80 1 -- 172.21.15.130:0/2917741899 shutdown_connections 2023-09-12T16:14:40.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.978+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91ae5a0 0x558fa91a6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.978+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ace00 0x558fa91ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.978+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/2917741899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91adc00 0x558fa91dfb20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.978+0000 7f503f3c0b80 1 -- 172.21.15.130:0/2917741899 >> 172.21.15.130:0/2917741899 conn(0x558fa9121090 msgr2=0x558fa91e0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:40.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.984+0000 7f503f3c0b80 1 -- 172.21.15.130:0/2917741899 shutdown_connections 2023-09-12T16:14:40.988 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.984+0000 7f503f3c0b80 1 -- 172.21.15.130:0/2917741899 wait complete. 2023-09-12T16:14:40.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.984+0000 7f503f3c0b80 1 -- start start 2023-09-12T16:14:40.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.984+0000 7f503f3c0b80 1 Processor -- start 2023-09-12T16:14:40.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91dba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91dbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f503f3c0b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91defe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.989 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa91df520 con 0x558fa91ae5a0 2023-09-12T16:14:40.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa91df690 con 0x558fa91adc00 2023-09-12T16:14:40.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f503f3c0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa91df850 con 0x558fa91ace00 2023-09-12T16:14:40.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f503f3c0b80 10 client.0 _ll_register_callbacks cb 0x7f501c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:14:40.990 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f5030f96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91dba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f5030f96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91dba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36102/0 (socket says 172.21.15.130:36102) 2023-09-12T16:14:40.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f5030f96700 1 -- 172.21.15.130:0/353545501 learned_addr learned my addr 172.21.15.130:0/353545501 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:14:40.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.985+0000 7f502bfff700 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91dbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.987+0000 7f5030f96700 1 -- 172.21.15.130:0/353545501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 msgr2=0x558fa91dbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:40.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.987+0000 7f5030f96700 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91dbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.991 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.987+0000 7f5030f96700 1 -- 172.21.15.130:0/353545501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 msgr2=0x558fa91defe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:14:40.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.987+0000 7f5030f96700 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91defe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:40.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.987+0000 7f5030f96700 1 -- 172.21.15.130:0/353545501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c10f5a0 con 0x558fa91ace00 2023-09-12T16:14:40.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f5030f96700 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91dba80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f501c001eb0 tx=0x7f501c019650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcb124aaf0fb8d4b server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:14:40.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f500bfff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:14:40.992 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f500bfff700 1 -- 172.21.15.130:0/353545501 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c005450 con 0x558fa91ace00 2023-09-12T16:14:40.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f500bfff700 1 -- 172.21.15.130:0/353545501 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f501c0055b0 con 0x558fa91ace00 2023-09-12T16:14:40.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f503f3c0b80 1 -- 172.21.15.130:0/353545501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f501c009e10 con 0x558fa91ace00 2023-09-12T16:14:40.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f500bfff700 1 -- 172.21.15.130:0/353545501 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f501c005770 con 0x558fa91ace00 2023-09-12T16:14:40.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f503f3c0b80 10 client.25810 fetch_fsmap learned FSMap version 32 2023-09-12T16:14:40.993 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f503f3c0b80 1 -- 172.21.15.130:0/353545501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558fa8f51250 con 0x558fa91ace00 2023-09-12T16:14:40.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f500bfff700 1 -- 172.21.15.130:0/353545501 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f501c009e10 con 0x558fa91ace00 2023-09-12T16:14:40.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.988+0000 7f500bfff700 1 -- 172.21.15.130:0/353545501 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f501c011840 con 0x558fa91ace00 2023-09-12T16:14:40.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.989+0000 7f503f3c0b80 10 client.25810 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:14:40.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.989+0000 7f503f3c0b80 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:14:40.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.989+0000 7f503f3c0b80 20 client.25810 populate_metadata read hostname 'smithi130' 2023-09-12T16:14:40.994 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.989+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558fa9238eb0 0x558fa923b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:14:40.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.989+0000 7f503f3c0b80 4 client.25810 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:14:40.995 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.989+0000 7f503f3c0b80 1 -- 172.21.15.130:0/353545501 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558fa90f7920 con 0x558fa9238eb0 2023-09-12T16:14:40.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.990+0000 7f502bfff700 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558fa9238eb0 0x558fa923b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:14:40.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.990+0000 7f500bfff700 10 client.25810 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:14:40.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.990+0000 7f502bfff700 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558fa9238eb0 0x558fa923b2d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b7f236301f5a4b server_cookie=f48ad221b9abf0ce in_seq=0 out_seq=0 2023-09-12T16:14:40.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.991+0000 7f500bfff700 1 -- 172.21.15.130:0/353545501 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558fa90f7920 con 0x558fa9238eb0 2023-09-12T16:14:40.996 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.991+0000 7f500bfff700 10 client.25810 handle_command_reply: tid=0 2023-09-12T16:14:40.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.991+0000 7f503f3c0b80 1 client.25810 shutdown 2023-09-12T16:14:40.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.997+0000 7f503f3c0b80 1 -- 172.21.15.130:0/353545501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 msgr2=0x558fa91dba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:40.997 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:40.997+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91dba80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f501c001eb0 tx=0x7f501c019650 comp rx=0 tx=0).stop 2023-09-12T16:14:41.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.008+0000 7f503f3c0b80 1 -- 172.21.15.130:0/353545501 shutdown_connections 2023-09-12T16:14:41.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.008+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x558fa9238eb0 0x558fa923b2d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:41.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.008+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558fa91ae5a0 0x558fa91defe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:41.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.008+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558fa91ace00 0x558fa91dba80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:41.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.008+0000 7f503f3c0b80 1 --2- 172.21.15.130:0/353545501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fa91adc00 0x558fa91dbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:41.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.008+0000 7f503f3c0b80 1 -- 172.21.15.130:0/353545501 >> 172.21.15.130:0/353545501 conn(0x558fa9121090 msgr2=0x558fa91a6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:41.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.009+0000 7f500bfff700 0 client.25810 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:14:41.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.011+0000 7f503f3c0b80 1 -- 172.21.15.130:0/353545501 shutdown_connections 2023-09-12T16:14:41.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.014+0000 7f503f3c0b80 1 -- 172.21.15.130:0/353545501 wait complete. 2023-09-12T16:14:41.016 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:14:41.016 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:14:41.016 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:14:41.016 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:14:41.017 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:14:41.017 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:14:41.017 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:14:41.017 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:14:41.017 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:14:41.018 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:14:41.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.015+0000 7f503f3c0b80 20 client.25810 trim_cache size 0 max 16384 2023-09-12T16:14:41.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.015+0000 7f50331fa700 1 -- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50140a0790 msgr2=0x7f50140a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:41.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.015+0000 7f50331fa700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50140a0790 0x7f50140a2c50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5024009da0 tx=0x7f502400b040 comp rx=0 tx=0).stop 2023-09-12T16:14:41.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.015+0000 7f50331fa700 1 -- 172.21.15.130:0/1191296727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 msgr2=0x7f502c1a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:14:41.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.015+0000 7f50331fa700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 0x7f502c1a5620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f502000f690 tx=0x7f502000f6c0 comp rx=0 tx=0).stop 2023-09-12T16:14:41.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.016+0000 7f50331fa700 1 -- 172.21.15.130:0/1191296727 shutdown_connections 2023-09-12T16:14:41.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.016+0000 7f50331fa700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f50140a0790 0x7f50140a2c50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:41.019 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.016+0000 7f50331fa700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f502c10c940 0x7f502c1ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:41.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.016+0000 7f50331fa700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f502c107600 0x7f502c1a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:41.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.016+0000 7f50331fa700 1 --2- 172.21.15.130:0/1191296727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c109f70 0x7f502c1a5620 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:14:41.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.016+0000 7f50331fa700 1 -- 172.21.15.130:0/1191296727 >> 172.21.15.130:0/1191296727 conn(0x7f502c06d3b0 msgr2=0x7f502c108310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:14:41.020 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.017+0000 7f50331fa700 1 -- 172.21.15.130:0/1191296727 shutdown_connections 2023-09-12T16:14:41.021 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:14:41.017+0000 7f50331fa700 1 -- 172.21.15.130:0/1191296727 wait complete. 2023-09-12T16:14:41.055 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:14:42.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:42 smithi130 ceph-mon[98670]: pgmap v645: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 28 MiB/s wr, 2.42k op/s 2023-09-12T16:14:42.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:42 smithi159 ceph-mon[102522]: pgmap v645: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 28 MiB/s wr, 2.42k op/s 2023-09-12T16:14:42.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:42 smithi203 ceph-mon[103431]: pgmap v645: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 28 MiB/s wr, 2.42k op/s 2023-09-12T16:14:44.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:44 smithi159 ceph-mon[102522]: pgmap v646: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 33 MiB/s wr, 3.00k op/s 2023-09-12T16:14:44.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:44 smithi130 ceph-mon[98670]: pgmap v646: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 33 MiB/s wr, 3.00k op/s 2023-09-12T16:14:44.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:44 smithi203 ceph-mon[103431]: pgmap v646: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 33 MiB/s wr, 3.00k op/s 2023-09-12T16:14:45.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 535.0 s, 1105.7 tps, lat 99.096 ms stddev 230.566 2023-09-12T16:14:46.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:46 smithi130 ceph-mon[98670]: pgmap v647: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 33 MiB/s wr, 3.46k op/s 2023-09-12T16:14:46.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:46 smithi159 ceph-mon[102522]: pgmap v647: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 33 MiB/s wr, 3.46k op/s 2023-09-12T16:14:46.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:46 smithi203 ceph-mon[103431]: pgmap v647: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 33 MiB/s wr, 3.46k op/s 2023-09-12T16:14:48.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:48 smithi130 ceph-mon[98670]: pgmap v648: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 33 MiB/s wr, 3.37k op/s 2023-09-12T16:14:48.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:48 smithi159 ceph-mon[102522]: pgmap v648: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 33 MiB/s wr, 3.37k op/s 2023-09-12T16:14:48.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:48 smithi203 ceph-mon[103431]: pgmap v648: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 33 MiB/s wr, 3.37k op/s 2023-09-12T16:14:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 540.0 s, 1947.4 tps, lat 52.409 ms stddev 41.871 2023-09-12T16:14:50.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:50 smithi130 ceph-mon[98670]: pgmap v649: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 33 MiB/s wr, 3.08k op/s 2023-09-12T16:14:50.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:50 smithi159 ceph-mon[102522]: pgmap v649: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 33 MiB/s wr, 3.08k op/s 2023-09-12T16:14:50.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:50 smithi203 ceph-mon[103431]: pgmap v649: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 33 MiB/s wr, 3.08k op/s 2023-09-12T16:14:52.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:52 smithi130 ceph-mon[98670]: pgmap v650: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 26 MiB/s wr, 2.31k op/s 2023-09-12T16:14:52.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:52 smithi159 ceph-mon[102522]: pgmap v650: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 26 MiB/s wr, 2.31k op/s 2023-09-12T16:14:52.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:52 smithi203 ceph-mon[103431]: pgmap v650: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 26 MiB/s wr, 2.31k op/s 2023-09-12T16:14:54.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:54 smithi159 ceph-mon[102522]: pgmap v651: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 26 MiB/s wr, 1.90k op/s 2023-09-12T16:14:54.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:54 smithi130 ceph-mon[98670]: pgmap v651: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 26 MiB/s wr, 1.90k op/s 2023-09-12T16:14:54.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:54 smithi203 ceph-mon[103431]: pgmap v651: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 26 MiB/s wr, 1.90k op/s 2023-09-12T16:14:55.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 545.0 s, 1113.2 tps, lat 88.622 ms stddev 128.532 2023-09-12T16:14:56.981 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:56 smithi130 ceph-mon[98670]: pgmap v652: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 26 MiB/s wr, 1.53k op/s 2023-09-12T16:14:56.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:56 smithi159 ceph-mon[102522]: pgmap v652: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 26 MiB/s wr, 1.53k op/s 2023-09-12T16:14:56.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:56 smithi203 ceph-mon[103431]: pgmap v652: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 26 MiB/s wr, 1.53k op/s 2023-09-12T16:14:58.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:14:58 smithi203 ceph-mon[103431]: pgmap v653: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 886 op/s 2023-09-12T16:14:58.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:14:58 smithi130 ceph-mon[98670]: pgmap v653: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 886 op/s 2023-09-12T16:14:58.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:14:58 smithi159 ceph-mon[102522]: pgmap v653: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 886 op/s 2023-09-12T16:15:00.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 550.0 s, 1254.3 tps, lat 80.364 ms stddev 70.241 2023-09-12T16:15:00.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:00 smithi130 ceph-mon[98670]: pgmap v654: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 911 op/s 2023-09-12T16:15:00.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:00 smithi159 ceph-mon[102522]: pgmap v654: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 911 op/s 2023-09-12T16:15:00.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:00 smithi203 ceph-mon[103431]: pgmap v654: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 911 op/s 2023-09-12T16:15:02.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:02 smithi130 ceph-mon[98670]: pgmap v655: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 847 op/s 2023-09-12T16:15:02.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:02 smithi159 ceph-mon[102522]: pgmap v655: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 847 op/s 2023-09-12T16:15:02.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:02 smithi203 ceph-mon[103431]: pgmap v655: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 847 op/s 2023-09-12T16:15:04.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:04 smithi130 ceph-mon[98670]: pgmap v656: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 19 MiB/s wr, 851 op/s 2023-09-12T16:15:04.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:04 smithi159 ceph-mon[102522]: pgmap v656: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 19 MiB/s wr, 851 op/s 2023-09-12T16:15:04.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:04 smithi203 ceph-mon[103431]: pgmap v656: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 19 MiB/s wr, 851 op/s 2023-09-12T16:15:05.092 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 555.0 s, 1306.0 tps, lat 76.746 ms stddev 78.510 2023-09-12T16:15:06.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:06 smithi159 ceph-mon[102522]: pgmap v657: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 20 MiB/s wr, 1.04k op/s 2023-09-12T16:15:06.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:06 smithi130 ceph-mon[98670]: pgmap v657: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 20 MiB/s wr, 1.04k op/s 2023-09-12T16:15:06.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:06 smithi203 ceph-mon[103431]: pgmap v657: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 20 MiB/s wr, 1.04k op/s 2023-09-12T16:15:08.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:08 smithi130 ceph-mon[98670]: pgmap v658: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 20 MiB/s wr, 1.12k op/s 2023-09-12T16:15:08.729 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:08 smithi159 ceph-mon[102522]: pgmap v658: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 20 MiB/s wr, 1.12k op/s 2023-09-12T16:15:08.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:08 smithi203 ceph-mon[103431]: pgmap v658: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 20 MiB/s wr, 1.12k op/s 2023-09-12T16:15:10.092 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 560.0 s, 487.6 tps, lat 170.602 ms stddev 378.783 2023-09-12T16:15:10.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:10 smithi203 ceph-mon[103431]: pgmap v659: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 1.20k op/s 2023-09-12T16:15:10.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:10 smithi130 ceph-mon[98670]: pgmap v659: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 1.20k op/s 2023-09-12T16:15:10.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:10 smithi159 ceph-mon[102522]: pgmap v659: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 1.20k op/s 2023-09-12T16:15:11.057 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:15:11.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.462+0000 7f76430a8700 1 -- 172.21.15.130:0/508375174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f763c06b8b0 msgr2=0x7f763c069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:11.463 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.462+0000 7f76430a8700 1 --2- 172.21.15.130:0/508375174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f763c06b8b0 0x7f763c069d00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7630009990 tx=0x7f7630009ca0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.467+0000 7f76430a8700 1 -- 172.21.15.130:0/508375174 shutdown_connections 2023-09-12T16:15:11.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.467+0000 7f76430a8700 1 --2- 172.21.15.130:0/508375174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f763c115410 0x7f763c117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.467+0000 7f76430a8700 1 --2- 172.21.15.130:0/508375174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f763c06b8b0 0x7f763c069d00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.468 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.467+0000 7f76430a8700 1 --2- 172.21.15.130:0/508375174 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f763c06a2d0 0x7f763c06a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.467+0000 7f76430a8700 1 -- 172.21.15.130:0/508375174 >> 172.21.15.130:0/508375174 conn(0x7f763c06f5b0 msgr2=0x7f763c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:11.469 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.467+0000 7f76430a8700 1 -- 172.21.15.130:0/508375174 shutdown_connections 2023-09-12T16:15:11.470 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.468+0000 7f76430a8700 1 -- 172.21.15.130:0/508375174 wait complete. 2023-09-12T16:15:11.474 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.469+0000 7f76430a8700 1 Processor -- start 2023-09-12T16:15:11.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.479+0000 7f76430a8700 1 -- start start 2023-09-12T16:15:11.485 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.479+0000 7f76430a8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f763c06a2d0 0x7f763c1a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.479+0000 7f76430a8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f763c06b8b0 0x7f763c1a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.479+0000 7f76430a8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f763c115410 0x7f763c1ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.479+0000 7f76430a8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c1acda0 con 0x7f763c115410 2023-09-12T16:15:11.486 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.479+0000 7f76430a8700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c10cc00 con 0x7f763c06b8b0 2023-09-12T16:15:11.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.479+0000 7f76430a8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c10cd90 con 0x7f763c06a2d0 2023-09-12T16:15:11.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.485+0000 7f7641645700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f763c115410 0x7f763c1ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.485+0000 7f763bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f763c06b8b0 0x7f763c1a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.485+0000 7f7641645700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f763c115410 0x7f763c1ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56506/0 (socket says 172.21.15.130:56506) 2023-09-12T16:15:11.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.485+0000 7f7641645700 1 -- 172.21.15.130:0/2157389241 learned_addr learned my addr 172.21.15.130:0/2157389241 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:11.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.485+0000 7f7640e44700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f763c06a2d0 0x7f763c1a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.486+0000 7f7640e44700 1 -- 172.21.15.130:0/2157389241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f763c06b8b0 msgr2=0x7f763c1a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:11.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.486+0000 7f7640e44700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f763c06b8b0 0x7f763c1a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.486+0000 7f7640e44700 1 -- 172.21.15.130:0/2157389241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f763c115410 msgr2=0x7f763c1ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:11.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.486+0000 7f7640e44700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f763c115410 0x7f763c1ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.486+0000 7f7640e44700 1 -- 172.21.15.130:0/2157389241 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76300095f0 con 0x7f763c06a2d0 2023-09-12T16:15:11.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.486+0000 7f7640e44700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f763c06a2d0 0x7f763c1a5220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f76300098e0 tx=0x7f7630009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:11.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.486+0000 7f7639ffb700 1 -- 172.21.15.130:0/2157389241 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763001d070 con 0x7f763c06a2d0 2023-09-12T16:15:11.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.487+0000 7f7639ffb700 1 -- 172.21.15.130:0/2157389241 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7630004ca0 con 0x7f763c06a2d0 2023-09-12T16:15:11.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.487+0000 7f7639ffb700 1 -- 172.21.15.130:0/2157389241 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630005560 con 0x7f763c06a2d0 2023-09-12T16:15:11.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.487+0000 7f76430a8700 1 -- 172.21.15.130:0/2157389241 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c10d010 con 0x7f763c06a2d0 2023-09-12T16:15:11.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.488+0000 7f76430a8700 1 -- 172.21.15.130:0/2157389241 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c10d490 con 0x7f763c06a2d0 2023-09-12T16:15:11.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f764f26eb80 1 Processor -- start 2023-09-12T16:15:11.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f764f26eb80 1 -- start start 2023-09-12T16:15:11.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d98c00 0x559400d99020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 0x559400d97800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d97e00 0x559400d982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f764f26eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400cf0730 con 0x559400d995a0 2023-09-12T16:15:11.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f764f26eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400cee050 con 0x559400d98c00 2023-09-12T16:15:11.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f764f26eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400ce9940 con 0x559400d97e00 2023-09-12T16:15:11.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f7639ffb700 1 -- 172.21.15.130:0/2157389241 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f763000fbd0 con 0x7f763c06a2d0 2023-09-12T16:15:11.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f7641645700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d98c00 0x559400d99020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f7641645700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d98c00 0x559400d99020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39768/0 (socket says 172.21.15.130:39768) 2023-09-12T16:15:11.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.494+0000 7f7641645700 1 -- 172.21.15.130:0/1705612389 learned_addr learned my addr 172.21.15.130:0/1705612389 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:11.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.495+0000 7f7639ffb700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f762c0a06c0 0x7f762c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.495+0000 7f7639ffb700 1 -- 172.21.15.130:0/2157389241 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f76300cd1c0 con 0x7f763c06a2d0 2023-09-12T16:15:11.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.499+0000 7f763bfff700 1 --2- 172.21.15.130:0/1705612389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d97e00 0x559400d982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.499+0000 7f763bfff700 1 -- 172.21.15.130:0/1705612389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d98c00 msgr2=0x559400d99020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:11.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.499+0000 7f763bfff700 1 --2- 172.21.15.130:0/1705612389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d98c00 0x559400d99020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.499+0000 7f763bfff700 1 -- 172.21.15.130:0/1705612389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 msgr2=0x559400d97800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:11.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.499+0000 7f763bfff700 1 --2- 172.21.15.130:0/1705612389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 0x559400d97800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.504 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.499+0000 7f763bfff700 1 -- 172.21.15.130:0/1705612389 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559400b15de0 con 0x559400d97e00 2023-09-12T16:15:11.506 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.506+0000 7f763bfff700 1 --2- 172.21.15.130:0/1705612389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d97e00 0x559400d982b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7628007750 tx=0x7f7628007a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe76cf4e5a7a2ab server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:11.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.506+0000 7f7626ffd700 1 -- 172.21.15.130:0/1705612389 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628011590 con 0x559400d97e00 2023-09-12T16:15:11.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.506+0000 7f7626ffd700 1 -- 172.21.15.130:0/1705612389 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7628011bb0 con 0x559400d97e00 2023-09-12T16:15:11.507 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.506+0000 7f7626ffd700 1 -- 172.21.15.130:0/1705612389 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628012c60 con 0x559400d97e00 2023-09-12T16:15:11.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.506+0000 7f7641645700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f762c0a06c0 0x7f762c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.507+0000 7f764f26eb80 1 -- 172.21.15.130:0/1705612389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d97e00 msgr2=0x559400d982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:11.508 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.507+0000 7f764f26eb80 1 --2- 172.21.15.130:0/1705612389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d97e00 0x559400d982b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7628007750 tx=0x7f7628007a60 comp rx=0 tx=0).stop 2023-09-12T16:15:11.514 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.514+0000 7f764f26eb80 1 -- 172.21.15.130:0/1705612389 shutdown_connections 2023-09-12T16:15:11.515 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.514+0000 7f764f26eb80 1 --2- 172.21.15.130:0/1705612389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d97e00 0x559400d982b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.515 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.514+0000 7f764f26eb80 1 --2- 172.21.15.130:0/1705612389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d98c00 0x559400d99020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.515 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.514+0000 7f764f26eb80 1 --2- 172.21.15.130:0/1705612389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 0x559400d97800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.515 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.514+0000 7f764f26eb80 1 -- 172.21.15.130:0/1705612389 >> 172.21.15.130:0/1705612389 conn(0x559400d0c090 msgr2=0x559400d92720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:11.516 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.516+0000 7f7641645700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f762c0a06c0 0x7f762c0a2b80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7634009fd0 tx=0x7f7634004ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:11.524 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.521+0000 7f764f26eb80 1 -- 172.21.15.130:0/1705612389 shutdown_connections 2023-09-12T16:15:11.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.532+0000 7f764f26eb80 1 -- 172.21.15.130:0/1705612389 wait complete. 2023-09-12T16:15:11.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.532+0000 7f764f26eb80 1 -- start start 2023-09-12T16:15:11.535 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.532+0000 7f764f26eb80 1 Processor -- start 2023-09-12T16:15:11.536 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.535+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d97e00 0x559400da15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.535+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 0x559400da1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.535+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 0x559400da8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.535+0000 7f764f26eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400c87b70 con 0x559400d995a0 2023-09-12T16:15:11.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.535+0000 7f764f26eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400c87870 con 0x559400d97e00 2023-09-12T16:15:11.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.535+0000 7f764f26eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400b6b920 con 0x559400d98c00 2023-09-12T16:15:11.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.535+0000 7f764f26eb80 10 client.0 _ll_register_callbacks cb 0x559400bd1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:15:11.538 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.536+0000 7f7641645700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 0x559400da8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.536+0000 7f7641645700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 0x559400da8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56518/0 (socket says 172.21.15.130:56518) 2023-09-12T16:15:11.539 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.536+0000 7f7641645700 1 -- 172.21.15.130:0/3046263117 learned_addr learned my addr 172.21.15.130:0/3046263117 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:11.540 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.537+0000 7f7641645700 1 -- 172.21.15.130:0/3046263117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 msgr2=0x559400da1b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:15:11.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.537+0000 7f7641645700 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 0x559400da1b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.537+0000 7f7641645700 1 -- 172.21.15.130:0/3046263117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d97e00 msgr2=0x559400da15c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:15:11.541 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.537+0000 7f7641645700 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d97e00 0x559400da15c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.537+0000 7f7641645700 1 -- 172.21.15.130:0/3046263117 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762800e040 con 0x559400d995a0 2023-09-12T16:15:11.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.537+0000 7f7641645700 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 0x559400da8b40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7634039e70 tx=0x7f763400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31a7dde14146533 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:11.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.537+0000 7f761bfff700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:15:11.542 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.538+0000 7f761bfff700 1 -- 172.21.15.130:0/3046263117 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634049070 con 0x559400d995a0 2023-09-12T16:15:11.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.538+0000 7f761bfff700 1 -- 172.21.15.130:0/3046263117 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f763403e030 con 0x559400d995a0 2023-09-12T16:15:11.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.538+0000 7f764f26eb80 1 -- 172.21.15.130:0/3046263117 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559400c414f0 con 0x559400d995a0 2023-09-12T16:15:11.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.538+0000 7f761bfff700 1 -- 172.21.15.130:0/3046263117 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7634038e20 con 0x559400d995a0 2023-09-12T16:15:11.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.538+0000 7f764f26eb80 10 client.16128 fetch_fsmap learned FSMap version 32 2023-09-12T16:15:11.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.539+0000 7f764f26eb80 1 -- 172.21.15.130:0/3046263117 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559400afec60 con 0x559400d995a0 2023-09-12T16:15:11.543 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.539+0000 7f761bfff700 1 -- 172.21.15.130:0/3046263117 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559400c414f0 con 0x559400d995a0 2023-09-12T16:15:11.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.539+0000 7f761bfff700 1 -- 172.21.15.130:0/3046263117 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f763404c020 con 0x559400d995a0 2023-09-12T16:15:11.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.539+0000 7f764f26eb80 10 client.16128 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:15:11.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.539+0000 7f764f26eb80 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:15:11.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.539+0000 7f764f26eb80 20 client.16128 populate_metadata read hostname 'smithi130' 2023-09-12T16:15:11.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.539+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559400dad4a0 0x559400daf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.539+0000 7f764f26eb80 4 client.16128 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:15:11.545 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.539+0000 7f764f26eb80 1 -- 172.21.15.130:0/3046263117 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559400cfe8a0 con 0x559400dad4a0 2023-09-12T16:15:11.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.541+0000 7f7640e44700 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559400dad4a0 0x559400daf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.542+0000 7f7640e44700 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559400dad4a0 0x559400daf8c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b1d01b02e4b32e server_cookie=d1e233c56be46f5e in_seq=0 out_seq=0 2023-09-12T16:15:11.546 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.542+0000 7f761bfff700 10 client.16128 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:15:11.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.542+0000 7f761bfff700 1 -- 172.21.15.130:0/3046263117 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559400cfe8a0 con 0x559400dad4a0 2023-09-12T16:15:11.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.542+0000 7f761bfff700 10 client.16128 handle_command_reply: tid=0 2023-09-12T16:15:11.547 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.543+0000 7f764f26eb80 1 client.16128 shutdown 2023-09-12T16:15:11.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.549+0000 7f764f26eb80 1 -- 172.21.15.130:0/3046263117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 msgr2=0x559400da8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:11.550 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.549+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 0x559400da8b40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7634039e70 tx=0x7f763400b040 comp rx=0 tx=0).stop 2023-09-12T16:15:11.551 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.551+0000 7f764f26eb80 1 -- 172.21.15.130:0/3046263117 shutdown_connections 2023-09-12T16:15:11.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.551+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559400dad4a0 0x559400daf8c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.551+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d995a0 0x559400da8b40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.551+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d97e00 0x559400da15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.552 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.551+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3046263117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 0x559400da1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.553 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.551+0000 7f764f26eb80 1 -- 172.21.15.130:0/3046263117 >> 172.21.15.130:0/3046263117 conn(0x559400d0c090 msgr2=0x559400d91dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:11.553 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.551+0000 7f761bfff700 0 client.16128 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:15:11.555 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.554+0000 7f764f26eb80 1 -- 172.21.15.130:0/3046263117 shutdown_connections 2023-09-12T16:15:11.556 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.555+0000 7f764f26eb80 1 -- 172.21.15.130:0/3046263117 wait complete. 2023-09-12T16:15:11.560 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.559+0000 7f764f26eb80 20 client.16128 trim_cache size 0 max 16384 2023-09-12T16:15:11.599 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.594+0000 7f764f26eb80 1 Processor -- start 2023-09-12T16:15:11.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.594+0000 7f764f26eb80 1 -- start start 2023-09-12T16:15:11.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.594+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d97e00 0x559400da15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.594+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 0x559400da8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.594+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d995a0 0x559400da1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.594+0000 7f764f26eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400d986b0 con 0x559400d97e00 2023-09-12T16:15:11.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.594+0000 7f764f26eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400d8f400 con 0x559400d995a0 2023-09-12T16:15:11.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.594+0000 7f764f26eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400d93420 con 0x559400d98c00 2023-09-12T16:15:11.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.594+0000 7f763bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 0x559400da8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.602 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.595+0000 7f763bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 0x559400da8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36136/0 (socket says 172.21.15.130:36136) 2023-09-12T16:15:11.602 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.595+0000 7f763bfff700 1 -- 172.21.15.130:0/3173979271 learned_addr learned my addr 172.21.15.130:0/3173979271 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:11.602 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.595+0000 7f7641645700 1 --2- 172.21.15.130:0/3173979271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d995a0 0x559400da1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.602 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.595+0000 7f763bfff700 1 -- 172.21.15.130:0/3173979271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d995a0 msgr2=0x559400da1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:11.602 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.595+0000 7f763bfff700 1 --2- 172.21.15.130:0/3173979271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d995a0 0x559400da1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.603 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.595+0000 7f763bfff700 1 -- 172.21.15.130:0/3173979271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d97e00 msgr2=0x559400da15c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:15:11.603 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.595+0000 7f763bfff700 1 --2- 172.21.15.130:0/3173979271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d97e00 0x559400da15c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.603 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.595+0000 7f763bfff700 1 -- 172.21.15.130:0/3173979271 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559400b3c250 con 0x559400d98c00 2023-09-12T16:15:11.603 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.595+0000 7f763bfff700 1 --2- 172.21.15.130:0/3173979271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 0x559400da8b00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7628000c00 tx=0x7f7628007000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1e298491f70c7e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:11.603 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.598+0000 7f76277fe700 1 -- 172.21.15.130:0/3173979271 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628012420 con 0x559400d98c00 2023-09-12T16:15:11.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.598+0000 7f76277fe700 1 -- 172.21.15.130:0/3173979271 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7628020070 con 0x559400d98c00 2023-09-12T16:15:11.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.598+0000 7f76277fe700 1 -- 172.21.15.130:0/3173979271 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628027480 con 0x559400d98c00 2023-09-12T16:15:11.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.599+0000 7f764f26eb80 1 -- 172.21.15.130:0/3173979271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 msgr2=0x559400da8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:11.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.599+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3173979271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 0x559400da8b00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7628000c00 tx=0x7f7628007000 comp rx=0 tx=0).stop 2023-09-12T16:15:11.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.599+0000 7f764f26eb80 1 -- 172.21.15.130:0/3173979271 shutdown_connections 2023-09-12T16:15:11.605 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.599+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3173979271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d995a0 0x559400da1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.599+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3173979271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d97e00 0x559400da15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.599+0000 7f764f26eb80 1 --2- 172.21.15.130:0/3173979271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d98c00 0x559400da8b00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.599+0000 7f764f26eb80 1 -- 172.21.15.130:0/3173979271 >> 172.21.15.130:0/3173979271 conn(0x559400d0c090 msgr2=0x559400d9f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:11.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.607+0000 7f764f26eb80 1 -- 172.21.15.130:0/3173979271 shutdown_connections 2023-09-12T16:15:11.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.611+0000 7f764f26eb80 1 -- 172.21.15.130:0/3173979271 wait complete. 2023-09-12T16:15:11.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.611+0000 7f764f26eb80 1 -- start start 2023-09-12T16:15:11.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.611+0000 7f764f26eb80 1 Processor -- start 2023-09-12T16:15:11.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.612+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d97e00 0x559400da88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.612+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d98c00 0x559400da5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.612+0000 7f764f26eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d995a0 0x559400da58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.612+0000 7f764f26eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400da5ea0 con 0x559400d98c00 2023-09-12T16:15:11.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.612+0000 7f764f26eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400da6010 con 0x559400d97e00 2023-09-12T16:15:11.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.612+0000 7f764f26eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559400e01340 con 0x559400d995a0 2023-09-12T16:15:11.614 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.612+0000 7f764f26eb80 10 client.0 _ll_register_callbacks cb 0x7f7634039840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:15:11.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.613+0000 7f763bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d98c00 0x559400da5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.613+0000 7f763bfff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d98c00 0x559400da5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56530/0 (socket says 172.21.15.130:56530) 2023-09-12T16:15:11.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.613+0000 7f763bfff700 1 -- 172.21.15.130:0/270233274 learned_addr learned my addr 172.21.15.130:0/270233274 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:11.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.613+0000 7f7641645700 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d995a0 0x559400da58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.613+0000 7f7640e44700 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d97e00 0x559400da88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.614+0000 7f763bfff700 1 -- 172.21.15.130:0/270233274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d995a0 msgr2=0x559400da58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:11.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.614+0000 7f763bfff700 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d995a0 0x559400da58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.614+0000 7f763bfff700 1 -- 172.21.15.130:0/270233274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d97e00 msgr2=0x559400da88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:11.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.614+0000 7f763bfff700 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d97e00 0x559400da88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.614+0000 7f763bfff700 1 -- 172.21.15.130:0/270233274 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559400b15de0 con 0x559400d98c00 2023-09-12T16:15:11.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.614+0000 7f763bfff700 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d98c00 0x559400da5390 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7628004790 tx=0x7f7628011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66fa97ae003b6e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:11.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.614+0000 7f761bfff700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:15:11.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.618+0000 7f761bfff700 1 -- 172.21.15.130:0/270233274 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628012770 con 0x559400d98c00 2023-09-12T16:15:11.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.618+0000 7f761bfff700 1 -- 172.21.15.130:0/270233274 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7628022070 con 0x559400d98c00 2023-09-12T16:15:11.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.618+0000 7f764f26eb80 1 -- 172.21.15.130:0/270233274 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7628011bb0 con 0x559400d98c00 2023-09-12T16:15:11.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.618+0000 7f761bfff700 1 -- 172.21.15.130:0/270233274 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76280128d0 con 0x559400d98c00 2023-09-12T16:15:11.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.618+0000 7f764f26eb80 10 client.16131 fetch_fsmap learned FSMap version 32 2023-09-12T16:15:11.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.618+0000 7f764f26eb80 1 -- 172.21.15.130:0/270233274 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559400b142b0 con 0x559400d98c00 2023-09-12T16:15:11.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.618+0000 7f761bfff700 1 -- 172.21.15.130:0/270233274 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7628011bb0 con 0x559400d98c00 2023-09-12T16:15:11.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.619+0000 7f761bfff700 1 -- 172.21.15.130:0/270233274 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7628027c80 con 0x559400d98c00 2023-09-12T16:15:11.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.619+0000 7f764f26eb80 10 client.16131 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:15:11.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.619+0000 7f764f26eb80 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:15:11.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.619+0000 7f764f26eb80 20 client.16131 populate_metadata read hostname 'smithi130' 2023-09-12T16:15:11.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.619+0000 7f764f26eb80 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559400e02960 0x559400e04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:11.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.619+0000 7f764f26eb80 4 client.16131 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:15:11.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.619+0000 7f764f26eb80 1 -- 172.21.15.130:0/270233274 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559400ce2920 con 0x559400e02960 2023-09-12T16:15:11.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.623+0000 7f7640e44700 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559400e02960 0x559400e04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:11.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.624+0000 7f7640e44700 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559400e02960 0x559400e04d80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6822a087d22970ba server_cookie=b7e2484bf26a93c7 in_seq=0 out_seq=0 2023-09-12T16:15:11.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.624+0000 7f761bfff700 10 client.16131 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:15:11.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.625+0000 7f761bfff700 1 -- 172.21.15.130:0/270233274 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559400ce2920 con 0x559400e02960 2023-09-12T16:15:11.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.625+0000 7f761bfff700 10 client.16131 handle_command_reply: tid=0 2023-09-12T16:15:11.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.625+0000 7f764f26eb80 1 client.16131 shutdown 2023-09-12T16:15:11.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.626+0000 7f764f26eb80 1 -- 172.21.15.130:0/270233274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d98c00 msgr2=0x559400da5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:11.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.626+0000 7f764f26eb80 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d98c00 0x559400da5390 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7628004790 tx=0x7f7628011f50 comp rx=0 tx=0).stop 2023-09-12T16:15:11.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.632+0000 7f764f26eb80 1 -- 172.21.15.130:0/270233274 shutdown_connections 2023-09-12T16:15:11.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.632+0000 7f764f26eb80 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x559400e02960 0x559400e04d80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.632+0000 7f764f26eb80 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559400d995a0 0x559400da58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.632+0000 7f764f26eb80 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559400d97e00 0x559400da88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.632+0000 7f764f26eb80 1 --2- 172.21.15.130:0/270233274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559400d98c00 0x559400da5390 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.632+0000 7f764f26eb80 1 -- 172.21.15.130:0/270233274 >> 172.21.15.130:0/270233274 conn(0x559400d0c090 msgr2=0x559400da1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:11.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.632+0000 7f761bfff700 0 client.16131 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:15:11.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.632+0000 7f764f26eb80 1 -- 172.21.15.130:0/270233274 shutdown_connections 2023-09-12T16:15:11.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.632+0000 7f764f26eb80 1 -- 172.21.15.130:0/270233274 wait complete. 2023-09-12T16:15:11.635 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.633+0000 7f764f26eb80 20 client.16131 trim_cache size 0 max 16384 2023-09-12T16:15:11.635 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:15:11.636 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:15:11.636 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:15:11.636 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:15:11.636 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:15:11.636 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:15:11.637 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:15:11.637 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:15:11.637 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:15:11.637 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:15:11.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 -- 172.21.15.130:0/2157389241 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f762c0a06c0 msgr2=0x7f762c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:11.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f762c0a06c0 0x7f762c0a2b80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7634009fd0 tx=0x7f7634004ad0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.638 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 -- 172.21.15.130:0/2157389241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f763c06a2d0 msgr2=0x7f763c1a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:11.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f763c06a2d0 0x7f763c1a5220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f76300098e0 tx=0x7f7630009580 comp rx=0 tx=0).stop 2023-09-12T16:15:11.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 -- 172.21.15.130:0/2157389241 shutdown_connections 2023-09-12T16:15:11.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f762c0a06c0 0x7f762c0a2b80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f763c115410 0x7f763c1ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.639 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f763c06a2d0 0x7f763c1a5220 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 --2- 172.21.15.130:0/2157389241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f763c06b8b0 0x7f763c1a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:11.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 -- 172.21.15.130:0/2157389241 >> 172.21.15.130:0/2157389241 conn(0x7f763c06f5b0 msgr2=0x7f763c0719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:11.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.634+0000 7f76430a8700 1 -- 172.21.15.130:0/2157389241 shutdown_connections 2023-09-12T16:15:11.640 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:11.635+0000 7f76430a8700 1 -- 172.21.15.130:0/2157389241 wait complete. 2023-09-12T16:15:11.658 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:15:12.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:12 smithi159 ceph-mon[102522]: pgmap v660: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:15:12.993 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:12 smithi130 ceph-mon[98670]: pgmap v660: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:15:12.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:12 smithi203 ceph-mon[103431]: pgmap v660: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:15:14.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:14 smithi130 ceph-mon[98670]: pgmap v661: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 29 MiB/s wr, 2.67k op/s 2023-09-12T16:15:14.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:14 smithi159 ceph-mon[102522]: pgmap v661: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 29 MiB/s wr, 2.67k op/s 2023-09-12T16:15:14.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:14 smithi203 ceph-mon[103431]: pgmap v661: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 29 MiB/s wr, 2.67k op/s 2023-09-12T16:15:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 565.0 s, 421.1 tps, lat 276.229 ms stddev 579.844 2023-09-12T16:15:16.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:16 smithi130 ceph-mon[98670]: pgmap v662: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 35 MiB/s wr, 3.38k op/s 2023-09-12T16:15:16.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:16 smithi159 ceph-mon[102522]: pgmap v662: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 35 MiB/s wr, 3.38k op/s 2023-09-12T16:15:16.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:16 smithi203 ceph-mon[103431]: pgmap v662: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 35 MiB/s wr, 3.38k op/s 2023-09-12T16:15:18.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:18 smithi130 ceph-mon[98670]: pgmap v663: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 34 MiB/s wr, 3.57k op/s 2023-09-12T16:15:18.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:18 smithi159 ceph-mon[102522]: pgmap v663: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 34 MiB/s wr, 3.57k op/s 2023-09-12T16:15:18.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:18 smithi203 ceph-mon[103431]: pgmap v663: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 34 MiB/s wr, 3.57k op/s 2023-09-12T16:15:20.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 570.0 s, 1812.2 tps, lat 54.123 ms stddev 45.024 2023-09-12T16:15:20.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:20 smithi159 ceph-mon[102522]: pgmap v664: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 32 MiB/s wr, 3.37k op/s 2023-09-12T16:15:20.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:20 smithi130 ceph-mon[98670]: pgmap v664: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 32 MiB/s wr, 3.37k op/s 2023-09-12T16:15:20.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:20 smithi203 ceph-mon[103431]: pgmap v664: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 32 MiB/s wr, 3.37k op/s 2023-09-12T16:15:22.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:22 smithi130 ceph-mon[98670]: pgmap v665: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 36 MiB/s wr, 3.45k op/s 2023-09-12T16:15:22.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:22 smithi159 ceph-mon[102522]: pgmap v665: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 36 MiB/s wr, 3.45k op/s 2023-09-12T16:15:22.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:22 smithi203 ceph-mon[103431]: pgmap v665: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 36 MiB/s wr, 3.45k op/s 2023-09-12T16:15:24.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:24 smithi130 ceph-mon[98670]: pgmap v666: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 29 MiB/s wr, 2.50k op/s 2023-09-12T16:15:24.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:24 smithi159 ceph-mon[102522]: pgmap v666: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 29 MiB/s wr, 2.50k op/s 2023-09-12T16:15:24.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:24 smithi203 ceph-mon[103431]: pgmap v666: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 29 MiB/s wr, 2.50k op/s 2023-09-12T16:15:25.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 575.0 s, 1508.7 tps, lat 67.158 ms stddev 58.862 2023-09-12T16:15:26.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:26 smithi159 ceph-mon[102522]: pgmap v667: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 25 MiB/s wr, 2.01k op/s 2023-09-12T16:15:26.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:26 smithi130 ceph-mon[98670]: pgmap v667: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 25 MiB/s wr, 2.01k op/s 2023-09-12T16:15:26.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:26 smithi203 ceph-mon[103431]: pgmap v667: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 25 MiB/s wr, 2.01k op/s 2023-09-12T16:15:28.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:28 smithi130 ceph-mon[98670]: pgmap v668: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 22 MiB/s wr, 1.36k op/s 2023-09-12T16:15:28.727 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:28 smithi159 ceph-mon[102522]: pgmap v668: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 22 MiB/s wr, 1.36k op/s 2023-09-12T16:15:28.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:28 smithi203 ceph-mon[103431]: pgmap v668: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 22 MiB/s wr, 1.36k op/s 2023-09-12T16:15:30.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 580.0 s, 740.5 tps, lat 131.756 ms stddev 195.700 2023-09-12T16:15:30.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:30 smithi130 ceph-mon[98670]: pgmap v669: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 MiB/s wr, 906 op/s 2023-09-12T16:15:30.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:30 smithi159 ceph-mon[102522]: pgmap v669: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 MiB/s wr, 906 op/s 2023-09-12T16:15:30.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:30 smithi203 ceph-mon[103431]: pgmap v669: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 MiB/s wr, 906 op/s 2023-09-12T16:15:32.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:32 smithi159 ceph-mon[102522]: pgmap v670: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 20 MiB/s wr, 985 op/s 2023-09-12T16:15:32.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:32 smithi130 ceph-mon[98670]: pgmap v670: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 20 MiB/s wr, 985 op/s 2023-09-12T16:15:32.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:32 smithi203 ceph-mon[103431]: pgmap v670: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 20 MiB/s wr, 985 op/s 2023-09-12T16:15:34.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:34 smithi130 ceph-mon[98670]: pgmap v671: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 15 MiB/s wr, 833 op/s 2023-09-12T16:15:34.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:34 smithi159 ceph-mon[102522]: pgmap v671: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 15 MiB/s wr, 833 op/s 2023-09-12T16:15:34.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:34 smithi203 ceph-mon[103431]: pgmap v671: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 15 MiB/s wr, 833 op/s 2023-09-12T16:15:35.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 585.0 s, 1263.7 tps, lat 80.198 ms stddev 78.237 2023-09-12T16:15:36.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:36 smithi130 ceph-mon[98670]: pgmap v672: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 16 MiB/s wr, 938 op/s 2023-09-12T16:15:36.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:36 smithi159 ceph-mon[102522]: pgmap v672: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 16 MiB/s wr, 938 op/s 2023-09-12T16:15:36.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:36 smithi203 ceph-mon[103431]: pgmap v672: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 16 MiB/s wr, 938 op/s 2023-09-12T16:15:38.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:38 smithi130 ceph-mon[98670]: pgmap v673: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 1k op/s 2023-09-12T16:15:38.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:38 smithi159 ceph-mon[102522]: pgmap v673: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 1k op/s 2023-09-12T16:15:38.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:38 smithi203 ceph-mon[103431]: pgmap v673: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 1k op/s 2023-09-12T16:15:40.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 590.0 s, 1348.4 tps, lat 64.803 ms stddev 52.835 2023-09-12T16:15:40.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:40 smithi159 ceph-mon[102522]: pgmap v674: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 870 op/s 2023-09-12T16:15:40.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:40 smithi130 ceph-mon[98670]: pgmap v674: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 870 op/s 2023-09-12T16:15:40.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:40 smithi203 ceph-mon[103431]: pgmap v674: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 870 op/s 2023-09-12T16:15:41.659 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:15:41.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:41 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:15:41.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:41 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:15:41.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:41 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:15:42.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.042+0000 7f963c384700 1 -- 172.21.15.130:0/4250992831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f963410c940 msgr2=0x7f963410cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:42.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.042+0000 7f963c384700 1 --2- 172.21.15.130:0/4250992831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f963410c940 0x7f963410cdc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f962c009f90 tx=0x7f962c00b330 comp rx=0 tx=0).stop 2023-09-12T16:15:42.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.051+0000 7f963c384700 1 -- 172.21.15.130:0/4250992831 shutdown_connections 2023-09-12T16:15:42.050 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.051+0000 7f963c384700 1 --2- 172.21.15.130:0/4250992831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f963410c940 0x7f963410cdc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.051+0000 7f963c384700 1 --2- 172.21.15.130:0/4250992831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634107600 0x7f9634109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.051+0000 7f963c384700 1 --2- 172.21.15.130:0/4250992831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9634109f70 0x7f963410c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.051 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.051+0000 7f963c384700 1 -- 172.21.15.130:0/4250992831 >> 172.21.15.130:0/4250992831 conn(0x7f963406d3b0 msgr2=0x7f963406f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:42.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.071+0000 7f963c384700 1 -- 172.21.15.130:0/4250992831 shutdown_connections 2023-09-12T16:15:42.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.077+0000 7f963c384700 1 -- 172.21.15.130:0/4250992831 wait complete. 2023-09-12T16:15:42.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.078+0000 7f963c384700 1 Processor -- start 2023-09-12T16:15:42.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.078+0000 7f963c384700 1 -- start start 2023-09-12T16:15:42.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.079+0000 7f963c384700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634107600 0x7f9634113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.079+0000 7f963c384700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9634109f70 0x7f9634114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.079+0000 7f963c384700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f963410c940 0x7f96341b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.079+0000 7f963c384700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9634117150 con 0x7f9634109f70 2023-09-12T16:15:42.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.079+0000 7f963c384700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96341172c0 con 0x7f963410c940 2023-09-12T16:15:42.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.079+0000 7f963c384700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96341b5870 con 0x7f9634107600 2023-09-12T16:15:42.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.080+0000 7f963a120700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634107600 0x7f9634113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.080+0000 7f963a120700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634107600 0x7f9634113b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36156/0 (socket says 172.21.15.130:36156) 2023-09-12T16:15:42.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.080+0000 7f963a120700 1 -- 172.21.15.130:0/869457921 learned_addr learned my addr 172.21.15.130:0/869457921 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:42.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.081+0000 7f963991f700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9634109f70 0x7f9634114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.083+0000 7f963a120700 1 -- 172.21.15.130:0/869457921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f963410c940 msgr2=0x7f96341b5330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-12T16:15:42.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.083+0000 7f963a120700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f963410c940 0x7f96341b5330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.083 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.083+0000 7f963a120700 1 -- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9634109f70 msgr2=0x7f9634114070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:42.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.083+0000 7f963a120700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9634109f70 0x7f9634114070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.083+0000 7f963a120700 1 -- 172.21.15.130:0/869457921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c009c40 con 0x7f9634107600 2023-09-12T16:15:42.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.084+0000 7f963a120700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634107600 0x7f9634113b30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f962400e9c0 tx=0x7f962400ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:42.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.084+0000 7f962b7fe700 1 -- 172.21.15.130:0/869457921 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962400cbd0 con 0x7f9634107600 2023-09-12T16:15:42.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.084+0000 7f962b7fe700 1 -- 172.21.15.130:0/869457921 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f962400cd30 con 0x7f9634107600 2023-09-12T16:15:42.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.084+0000 7f962b7fe700 1 -- 172.21.15.130:0/869457921 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240055a0 con 0x7f9634107600 2023-09-12T16:15:42.085 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.084+0000 7f963c384700 1 -- 172.21.15.130:0/869457921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96341b5a10 con 0x7f9634107600 2023-09-12T16:15:42.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.085+0000 7f963c384700 1 -- 172.21.15.130:0/869457921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96341b5f20 con 0x7f9634107600 2023-09-12T16:15:42.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.090+0000 7f964854ab80 1 Processor -- start 2023-09-12T16:15:42.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.090+0000 7f964854ab80 1 -- start start 2023-09-12T16:15:42.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.090+0000 7f964854ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c60c00 0x562863c61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.090+0000 7f964854ab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c5f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.092 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.090+0000 7f964854ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.090+0000 7f964854ab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863bb8730 con 0x562863c615a0 2023-09-12T16:15:42.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.090+0000 7f964854ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863bb6050 con 0x562863c5fe00 2023-09-12T16:15:42.093 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.090+0000 7f964854ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863bb1940 con 0x562863c60c00 2023-09-12T16:15:42.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.095+0000 7f963a921700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c60c00 0x562863c61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.094 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.095+0000 7f963991f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.095+0000 7f963991f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39810/0 (socket says 172.21.15.130:39810) 2023-09-12T16:15:42.095 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.095+0000 7f963991f700 1 -- 172.21.15.130:0/3241693323 learned_addr learned my addr 172.21.15.130:0/3241693323 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:42.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.101+0000 7f963a120700 1 --2- 172.21.15.130:0/3241693323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c5f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.101+0000 7f962b7fe700 1 -- 172.21.15.130:0/869457921 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f9624004750 con 0x7f9634107600 2023-09-12T16:15:42.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.102+0000 7f962b7fe700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f96200a0840 0x7f96200a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.103 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.102+0000 7f962b7fe700 1 -- 172.21.15.130:0/869457921 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9624014070 con 0x7f9634107600 2023-09-12T16:15:42.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.103+0000 7f963991f700 1 -- 172.21.15.130:0/3241693323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c60c00 msgr2=0x562863c61020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:42.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.103+0000 7f963991f700 1 --2- 172.21.15.130:0/3241693323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c60c00 0x562863c61020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.103+0000 7f963991f700 1 -- 172.21.15.130:0/3241693323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 msgr2=0x562863c5f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:42.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.103+0000 7f963991f700 1 --2- 172.21.15.130:0/3241693323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c5f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.103+0000 7f963991f700 1 -- 172.21.15.130:0/3241693323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628639ddde0 con 0x562863c5fe00 2023-09-12T16:15:42.104 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.103+0000 7f963a120700 1 --2- 172.21.15.130:0/3241693323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c5f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:15:42.105 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.103+0000 7f963a921700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f96200a0840 0x7f96200a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.104+0000 7f963a921700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f96200a0840 0x7f96200a2d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f962c007a70 tx=0x7f962c00b330 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:42.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.111+0000 7f963991f700 1 --2- 172.21.15.130:0/3241693323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c602b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9630007810 tx=0x7f9630007b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=654928d3b70bb7c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:42.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.111+0000 7f9628d89700 1 -- 172.21.15.130:0/3241693323 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300125f0 con 0x562863c5fe00 2023-09-12T16:15:42.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.112+0000 7f9628d89700 1 -- 172.21.15.130:0/3241693323 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9630012c10 con 0x562863c5fe00 2023-09-12T16:15:42.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.112+0000 7f9628d89700 1 -- 172.21.15.130:0/3241693323 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630013e40 con 0x562863c5fe00 2023-09-12T16:15:42.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.113+0000 7f964854ab80 1 -- 172.21.15.130:0/3241693323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 msgr2=0x562863c602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:42.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.113+0000 7f964854ab80 1 --2- 172.21.15.130:0/3241693323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c602b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9630007810 tx=0x7f9630007b20 comp rx=0 tx=0).stop 2023-09-12T16:15:42.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.115+0000 7f964854ab80 1 -- 172.21.15.130:0/3241693323 shutdown_connections 2023-09-12T16:15:42.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.115+0000 7f964854ab80 1 --2- 172.21.15.130:0/3241693323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c602b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.115+0000 7f964854ab80 1 --2- 172.21.15.130:0/3241693323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c60c00 0x562863c61020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.115+0000 7f964854ab80 1 --2- 172.21.15.130:0/3241693323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c5f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.115+0000 7f964854ab80 1 -- 172.21.15.130:0/3241693323 >> 172.21.15.130:0/3241693323 conn(0x562863bd4090 msgr2=0x562863c5a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:42.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.117+0000 7f964854ab80 1 -- 172.21.15.130:0/3241693323 shutdown_connections 2023-09-12T16:15:42.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.117+0000 7f964854ab80 1 -- 172.21.15.130:0/3241693323 wait complete. 2023-09-12T16:15:42.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.118+0000 7f964854ab80 1 -- start start 2023-09-12T16:15:42.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.118+0000 7f964854ab80 1 Processor -- start 2023-09-12T16:15:42.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.118+0000 7f964854ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c695d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.118+0000 7f964854ab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c60c00 0x562863c69b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.118+0000 7f964854ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c615a0 0x562863c70b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.118+0000 7f964854ab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863b4fb70 con 0x562863c60c00 2023-09-12T16:15:42.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.118+0000 7f964854ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863b4f870 con 0x562863c5fe00 2023-09-12T16:15:42.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.118+0000 7f964854ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863a33920 con 0x562863c615a0 2023-09-12T16:15:42.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.118+0000 7f964854ab80 10 client.0 _ll_register_callbacks cb 0x562863a99a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:15:42.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.119+0000 7f963991f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c60c00 0x562863c69b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.119+0000 7f963991f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c60c00 0x562863c69b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56560/0 (socket says 172.21.15.130:56560) 2023-09-12T16:15:42.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.119+0000 7f963991f700 1 -- 172.21.15.130:0/3255771942 learned_addr learned my addr 172.21.15.130:0/3255771942 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:42.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.120+0000 7f963991f700 1 -- 172.21.15.130:0/3255771942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c615a0 msgr2=0x562863c70b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:15:42.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.120+0000 7f963991f700 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c615a0 0x562863c70b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.120+0000 7f963991f700 1 -- 172.21.15.130:0/3255771942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 msgr2=0x562863c695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:42.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.120+0000 7f963991f700 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.120+0000 7f963991f700 1 -- 172.21.15.130:0/3255771942 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963000e040 con 0x562863c60c00 2023-09-12T16:15:42.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.120+0000 7f963991f700 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c60c00 0x562863c69b10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9630004780 tx=0x7f96300070f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81366e8b344f6543 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:42.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.120+0000 7f961dffb700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:15:42.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.121+0000 7f961dffb700 1 -- 172.21.15.130:0/3255771942 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630012ec0 con 0x562863c60c00 2023-09-12T16:15:42.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.121+0000 7f961dffb700 1 -- 172.21.15.130:0/3255771942 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9630013d50 con 0x562863c60c00 2023-09-12T16:15:42.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.121+0000 7f964854ab80 1 -- 172.21.15.130:0/3255771942 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562863b094f0 con 0x562863c60c00 2023-09-12T16:15:42.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.121+0000 7f961dffb700 1 -- 172.21.15.130:0/3255771942 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96300125f0 con 0x562863c60c00 2023-09-12T16:15:42.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.125+0000 7f964854ab80 10 client.16140 fetch_fsmap learned FSMap version 32 2023-09-12T16:15:42.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.125+0000 7f964854ab80 1 -- 172.21.15.130:0/3255771942 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5628639c6c60 con 0x562863c60c00 2023-09-12T16:15:42.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.129+0000 7f961dffb700 1 -- 172.21.15.130:0/3255771942 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562863b094f0 con 0x562863c60c00 2023-09-12T16:15:42.130 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.129+0000 7f961dffb700 1 -- 172.21.15.130:0/3255771942 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9630031c70 con 0x562863c60c00 2023-09-12T16:15:42.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.130+0000 7f964854ab80 10 client.16140 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:15:42.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.130+0000 7f964854ab80 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:15:42.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.130+0000 7f964854ab80 20 client.16140 populate_metadata read hostname 'smithi130' 2023-09-12T16:15:42.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.131+0000 7f964854ab80 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x562863c75300 0x562863c77720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.131 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.131+0000 7f964854ab80 4 client.16140 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:15:42.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.131+0000 7f964854ab80 1 -- 172.21.15.130:0/3255771942 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562863bc68a0 con 0x562863c75300 2023-09-12T16:15:42.132 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.133+0000 7f963a120700 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x562863c75300 0x562863c77720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.133+0000 7f963a120700 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x562863c75300 0x562863c77720 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd0fe7f04a6ecc6 server_cookie=e3869fd928b9fb3f in_seq=0 out_seq=0 2023-09-12T16:15:42.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.133+0000 7f961dffb700 10 client.16140 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:15:42.133 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.134+0000 7f961dffb700 1 -- 172.21.15.130:0/3255771942 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562863bc68a0 con 0x562863c75300 2023-09-12T16:15:42.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.134+0000 7f961dffb700 10 client.16140 handle_command_reply: tid=0 2023-09-12T16:15:42.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.134+0000 7f964854ab80 1 client.16140 shutdown 2023-09-12T16:15:42.134 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.134+0000 7f964854ab80 1 -- 172.21.15.130:0/3255771942 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c60c00 msgr2=0x562863c69b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:42.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.134+0000 7f964854ab80 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c60c00 0x562863c69b10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9630004780 tx=0x7f96300070f0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.135+0000 7f964854ab80 1 -- 172.21.15.130:0/3255771942 shutdown_connections 2023-09-12T16:15:42.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.135+0000 7f964854ab80 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x562863c75300 0x562863c77720 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.135+0000 7f964854ab80 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c615a0 0x562863c70b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.135 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.135+0000 7f964854ab80 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.135+0000 7f964854ab80 1 --2- 172.21.15.130:0/3255771942 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c60c00 0x562863c69b10 secure :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9630004780 tx=0x7f96300070f0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.135+0000 7f964854ab80 1 -- 172.21.15.130:0/3255771942 >> 172.21.15.130:0/3255771942 conn(0x562863bd4090 msgr2=0x562863c59dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:42.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.136+0000 7f961dffb700 0 client.16140 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:15:42.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.136+0000 7f964854ab80 1 -- 172.21.15.130:0/3255771942 shutdown_connections 2023-09-12T16:15:42.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.136+0000 7f964854ab80 1 -- 172.21.15.130:0/3255771942 wait complete. 2023-09-12T16:15:42.138 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.139+0000 7f964854ab80 20 client.16140 trim_cache size 0 max 16384 2023-09-12T16:15:42.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.167+0000 7f964854ab80 1 Processor -- start 2023-09-12T16:15:42.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.169+0000 7f964854ab80 1 -- start start 2023-09-12T16:15:42.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.169+0000 7f964854ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c5fe00 0x562863c5f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.169+0000 7f964854ab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c70a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.169+0000 7f964854ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c67350 0x562863c677e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.169+0000 7f964854ab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863c68120 con 0x562863c615a0 2023-09-12T16:15:42.170 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.169+0000 7f964854ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863c5fc50 con 0x562863c67350 2023-09-12T16:15:42.171 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.169+0000 7f964854ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863c5b660 con 0x562863c5fe00 2023-09-12T16:15:42.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.176+0000 7f963a921700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c67350 0x562863c677e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.176+0000 7f963991f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c70a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.176+0000 7f963a921700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c67350 0x562863c677e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39824/0 (socket says 172.21.15.130:39824) 2023-09-12T16:15:42.176 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.176+0000 7f963a921700 1 -- 172.21.15.130:0/1443980498 learned_addr learned my addr 172.21.15.130:0/1443980498 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:42.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.176+0000 7f963a120700 1 --2- 172.21.15.130:0/1443980498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c5fe00 0x562863c5f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.177+0000 7f963a921700 1 -- 172.21.15.130:0/1443980498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c5fe00 msgr2=0x562863c5f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:42.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.177+0000 7f963a921700 1 --2- 172.21.15.130:0/1443980498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c5fe00 0x562863c5f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.177+0000 7f963a921700 1 -- 172.21.15.130:0/1443980498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 msgr2=0x562863c70a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:42.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.177+0000 7f963a921700 1 --2- 172.21.15.130:0/1443980498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c70a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.177+0000 7f963a921700 1 -- 172.21.15.130:0/1443980498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562863c7c350 con 0x562863c67350 2023-09-12T16:15:42.178 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.177+0000 7f963a921700 1 --2- 172.21.15.130:0/1443980498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c67350 0x562863c677e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f962c007890 tx=0x7f962c01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e57de585c2f4883 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:42.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.177+0000 7f962958a700 1 -- 172.21.15.130:0/1443980498 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c020040 con 0x562863c67350 2023-09-12T16:15:42.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.178+0000 7f962958a700 1 -- 172.21.15.130:0/1443980498 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f962c026470 con 0x562863c67350 2023-09-12T16:15:42.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.178+0000 7f962958a700 1 -- 172.21.15.130:0/1443980498 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c00e040 con 0x562863c67350 2023-09-12T16:15:42.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.178+0000 7f964854ab80 1 -- 172.21.15.130:0/1443980498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c67350 msgr2=0x562863c677e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:42.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.178+0000 7f964854ab80 1 --2- 172.21.15.130:0/1443980498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c67350 0x562863c677e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f962c007890 tx=0x7f962c01a000 comp rx=0 tx=0).stop 2023-09-12T16:15:42.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.179+0000 7f964854ab80 1 -- 172.21.15.130:0/1443980498 shutdown_connections 2023-09-12T16:15:42.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.179+0000 7f964854ab80 1 --2- 172.21.15.130:0/1443980498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c67350 0x562863c677e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.179+0000 7f964854ab80 1 --2- 172.21.15.130:0/1443980498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c5fe00 0x562863c5f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.179+0000 7f964854ab80 1 --2- 172.21.15.130:0/1443980498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c70a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.179+0000 7f964854ab80 1 -- 172.21.15.130:0/1443980498 >> 172.21.15.130:0/1443980498 conn(0x562863bd4090 msgr2=0x562863c77310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:42.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.179+0000 7f964854ab80 1 -- 172.21.15.130:0/1443980498 shutdown_connections 2023-09-12T16:15:42.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.190+0000 7f964854ab80 1 -- 172.21.15.130:0/1443980498 wait complete. 2023-09-12T16:15:42.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.190+0000 7f964854ab80 1 -- start start 2023-09-12T16:15:42.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.190+0000 7f964854ab80 1 Processor -- start 2023-09-12T16:15:42.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.195+0000 7f964854ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c6d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.195+0000 7f964854ab80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c6dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.195+0000 7f964854ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c67350 0x562863cc9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.195+0000 7f964854ab80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863c6e270 con 0x562863c615a0 2023-09-12T16:15:42.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.195+0000 7f964854ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863c6e3e0 con 0x562863c5fe00 2023-09-12T16:15:42.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.195+0000 7f964854ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562863cc9690 con 0x562863c67350 2023-09-12T16:15:42.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.195+0000 7f964854ab80 10 client.0 _ll_register_callbacks cb 0x7f9630007690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:15:42.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.196+0000 7f963a921700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c67350 0x562863cc9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.196+0000 7f963a921700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c67350 0x562863cc9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36186/0 (socket says 172.21.15.130:36186) 2023-09-12T16:15:42.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.196+0000 7f963a921700 1 -- 172.21.15.130:0/2363423226 learned_addr learned my addr 172.21.15.130:0/2363423226 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:15:42.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.201+0000 7f963a120700 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c6d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.202+0000 7f963a120700 1 -- 172.21.15.130:0/2363423226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c67350 msgr2=0x562863cc9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:42.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.202+0000 7f963a120700 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c67350 0x562863cc9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.202+0000 7f963a120700 1 -- 172.21.15.130:0/2363423226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 msgr2=0x562863c6dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:15:42.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.202+0000 7f963a120700 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c6dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.202+0000 7f963a120700 1 -- 172.21.15.130:0/2363423226 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963410d940 con 0x562863c5fe00 2023-09-12T16:15:42.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.202+0000 7f963a120700 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c6d7b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f962400c5b0 tx=0x7f96240097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dad5e196b45f89e server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:15:42.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.202+0000 7f961dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:15:42.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.202+0000 7f961dffb700 1 -- 172.21.15.130:0/2363423226 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240b2070 con 0x562863c5fe00 2023-09-12T16:15:42.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.202+0000 7f961dffb700 1 -- 172.21.15.130:0/2363423226 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f962409bdc0 con 0x562863c5fe00 2023-09-12T16:15:42.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.207+0000 7f964854ab80 1 -- 172.21.15.130:0/2363423226 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9630012c10 con 0x562863c5fe00 2023-09-12T16:15:42.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.208+0000 7f961dffb700 1 -- 172.21.15.130:0/2363423226 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f962409b200 con 0x562863c5fe00 2023-09-12T16:15:42.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.209+0000 7f964854ab80 10 client.25619 fetch_fsmap learned FSMap version 32 2023-09-12T16:15:42.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.209+0000 7f964854ab80 1 -- 172.21.15.130:0/2363423226 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562863a04250 con 0x562863c5fe00 2023-09-12T16:15:42.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.214+0000 7f961dffb700 1 -- 172.21.15.130:0/2363423226 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9630012c10 con 0x562863c5fe00 2023-09-12T16:15:42.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.214+0000 7f961dffb700 1 -- 172.21.15.130:0/2363423226 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f962409f160 con 0x562863c5fe00 2023-09-12T16:15:42.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.214+0000 7f964854ab80 10 client.25619 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:15:42.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.214+0000 7f964854ab80 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:15:42.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.214+0000 7f964854ab80 20 client.25619 populate_metadata read hostname 'smithi130' 2023-09-12T16:15:42.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.214+0000 7f964854ab80 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x562863ccb0a0 0x562863ccd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:15:42.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.214+0000 7f964854ab80 4 client.25619 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:15:42.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.214+0000 7f964854ab80 1 -- 172.21.15.130:0/2363423226 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562863baa920 con 0x562863ccb0a0 2023-09-12T16:15:42.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.221+0000 7f963991f700 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x562863ccb0a0 0x562863ccd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:15:42.230 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.224+0000 7f963991f700 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x562863ccb0a0 0x562863ccd4c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564d8be8a103c9d4 server_cookie=b47ff734a8801903 in_seq=0 out_seq=0 2023-09-12T16:15:42.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.231+0000 7f961dffb700 10 client.25619 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:15:42.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.231+0000 7f961dffb700 1 -- 172.21.15.130:0/2363423226 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562863baa920 con 0x562863ccb0a0 2023-09-12T16:15:42.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.231+0000 7f961dffb700 10 client.25619 handle_command_reply: tid=0 2023-09-12T16:15:42.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.231+0000 7f964854ab80 1 client.25619 shutdown 2023-09-12T16:15:42.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.237+0000 7f964854ab80 1 -- 172.21.15.130:0/2363423226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 msgr2=0x562863c6d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:42.237 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.237+0000 7f964854ab80 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c6d7b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f962400c5b0 tx=0x7f96240097d0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.238+0000 7f964854ab80 1 -- 172.21.15.130:0/2363423226 shutdown_connections 2023-09-12T16:15:42.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.238+0000 7f964854ab80 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x562863ccb0a0 0x562863ccd4c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.238+0000 7f964854ab80 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562863c67350 0x562863cc9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.238+0000 7f964854ab80 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562863c5fe00 0x562863c6d7b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.238 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.238+0000 7f964854ab80 1 --2- 172.21.15.130:0/2363423226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562863c615a0 0x562863c6dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.238+0000 7f964854ab80 1 -- 172.21.15.130:0/2363423226 >> 172.21.15.130:0/2363423226 conn(0x562863bd4090 msgr2=0x562863c63e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:42.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.238+0000 7f961dffb700 0 client.25619 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:15:42.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.241+0000 7f964854ab80 1 -- 172.21.15.130:0/2363423226 shutdown_connections 2023-09-12T16:15:42.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.241+0000 7f964854ab80 1 -- 172.21.15.130:0/2363423226 wait complete. 2023-09-12T16:15:42.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.244+0000 7f964854ab80 20 client.25619 trim_cache size 0 max 16384 2023-09-12T16:15:42.244 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:15:42.245 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:15:42.245 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:15:42.245 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:15:42.245 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:15:42.246 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:15:42.246 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:15:42.246 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:15:42.246 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:15:42.246 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:15:42.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.250+0000 7f963c384700 1 -- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f96200a0840 msgr2=0x7f96200a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:42.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.250+0000 7f963c384700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f96200a0840 0x7f96200a2d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f962c007a70 tx=0x7f962c00b330 comp rx=0 tx=0).stop 2023-09-12T16:15:42.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.250+0000 7f963c384700 1 -- 172.21.15.130:0/869457921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634107600 msgr2=0x7f9634113b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:15:42.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.250+0000 7f963c384700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634107600 0x7f9634113b30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f962400e9c0 tx=0x7f962400ed80 comp rx=0 tx=0).stop 2023-09-12T16:15:42.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.251+0000 7f963c384700 1 -- 172.21.15.130:0/869457921 shutdown_connections 2023-09-12T16:15:42.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.251+0000 7f963c384700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f96200a0840 0x7f96200a2d00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.262 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.251+0000 7f963c384700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f963410c940 0x7f96341b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.251+0000 7f963c384700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634107600 0x7f9634113b30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.251+0000 7f963c384700 1 --2- 172.21.15.130:0/869457921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9634109f70 0x7f9634114070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:15:42.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.251+0000 7f963c384700 1 -- 172.21.15.130:0/869457921 >> 172.21.15.130:0/869457921 conn(0x7f963406d3b0 msgr2=0x7f96341082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:15:42.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.262+0000 7f963c384700 1 -- 172.21.15.130:0/869457921 shutdown_connections 2023-09-12T16:15:42.263 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:15:42.262+0000 7f963c384700 1 -- 172.21.15.130:0/869457921 wait complete. 2023-09-12T16:15:42.311 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:15:42.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:42 smithi159 ceph-mon[102522]: pgmap v675: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 21 MiB/s wr, 1.36k op/s 2023-09-12T16:15:42.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:42 smithi130 ceph-mon[98670]: pgmap v675: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 21 MiB/s wr, 1.36k op/s 2023-09-12T16:15:43.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:42 smithi203 ceph-mon[103431]: pgmap v675: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 21 MiB/s wr, 1.36k op/s 2023-09-12T16:15:43.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:15:43.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:15:43.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:15:43.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:15:43.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:15:43.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:15:44.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:15:44.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:15:44.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:15:44.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:44 smithi130 ceph-mon[98670]: pgmap v676: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 22 MiB/s wr, 1.64k op/s 2023-09-12T16:15:44.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:44 smithi159 ceph-mon[102522]: pgmap v676: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 22 MiB/s wr, 1.64k op/s 2023-09-12T16:15:44.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:44 smithi203 ceph-mon[103431]: pgmap v676: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 22 MiB/s wr, 1.64k op/s 2023-09-12T16:15:45.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 595.0 s, 205.1 tps, lat 440.202 ms stddev 584.712 2023-09-12T16:15:46.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:46 smithi159 ceph-mon[102522]: pgmap v677: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 29 MiB/s wr, 2.49k op/s 2023-09-12T16:15:46.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:46 smithi130 ceph-mon[98670]: pgmap v677: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 29 MiB/s wr, 2.49k op/s 2023-09-12T16:15:46.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:46 smithi203 ceph-mon[103431]: pgmap v677: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 29 MiB/s wr, 2.49k op/s 2023-09-12T16:15:48.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:48 smithi159 ceph-mon[102522]: pgmap v678: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 33 MiB/s wr, 3.02k op/s 2023-09-12T16:15:48.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:48 smithi130 ceph-mon[98670]: pgmap v678: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 33 MiB/s wr, 3.02k op/s 2023-09-12T16:15:48.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:48 smithi203 ceph-mon[103431]: pgmap v678: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 33 MiB/s wr, 3.02k op/s 2023-09-12T16:15:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 600.0 s, 824.6 tps, lat 150.134 ms stddev 380.968 2023-09-12T16:15:50.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:50 smithi130 ceph-mon[98670]: pgmap v679: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 32 MiB/s wr, 3.12k op/s 2023-09-12T16:15:50.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:50 smithi159 ceph-mon[102522]: pgmap v679: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 32 MiB/s wr, 3.12k op/s 2023-09-12T16:15:50.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:50 smithi203 ceph-mon[103431]: pgmap v679: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 32 MiB/s wr, 3.12k op/s 2023-09-12T16:15:52.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:52 smithi130 ceph-mon[98670]: pgmap v680: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 37 MiB/s wr, 3.68k op/s 2023-09-12T16:15:52.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:52 smithi159 ceph-mon[102522]: pgmap v680: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 37 MiB/s wr, 3.68k op/s 2023-09-12T16:15:52.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:52 smithi203 ceph-mon[103431]: pgmap v680: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 37 MiB/s wr, 3.68k op/s 2023-09-12T16:15:54.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:54 smithi159 ceph-mon[102522]: pgmap v681: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 32 MiB/s wr, 3.30k op/s 2023-09-12T16:15:54.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:54 smithi130 ceph-mon[98670]: pgmap v681: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 32 MiB/s wr, 3.30k op/s 2023-09-12T16:15:54.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:54 smithi203 ceph-mon[103431]: pgmap v681: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 32 MiB/s wr, 3.30k op/s 2023-09-12T16:15:55.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 605.0 s, 1636.8 tps, lat 61.287 ms stddev 47.808 2023-09-12T16:15:56.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:56 smithi130 ceph-mon[98670]: pgmap v682: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 31 MiB/s wr, 2.99k op/s 2023-09-12T16:15:56.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:56 smithi159 ceph-mon[102522]: pgmap v682: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 31 MiB/s wr, 2.99k op/s 2023-09-12T16:15:56.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:56 smithi203 ceph-mon[103431]: pgmap v682: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 31 MiB/s wr, 2.99k op/s 2023-09-12T16:15:58.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:15:58 smithi159 ceph-mon[102522]: pgmap v683: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 26 MiB/s wr, 2.21k op/s 2023-09-12T16:15:58.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:15:58 smithi130 ceph-mon[98670]: pgmap v683: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 26 MiB/s wr, 2.21k op/s 2023-09-12T16:15:58.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:15:58 smithi203 ceph-mon[103431]: pgmap v683: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 26 MiB/s wr, 2.21k op/s 2023-09-12T16:16:00.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 610.0 s, 1639.3 tps, lat 56.466 ms stddev 64.695 2023-09-12T16:16:00.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:00 smithi130 ceph-mon[98670]: pgmap v684: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 1.54k op/s 2023-09-12T16:16:00.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:00 smithi159 ceph-mon[102522]: pgmap v684: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 1.54k op/s 2023-09-12T16:16:01.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:00 smithi203 ceph-mon[103431]: pgmap v684: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 1.54k op/s 2023-09-12T16:16:02.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:02 smithi130 ceph-mon[98670]: pgmap v685: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 24 MiB/s wr, 1.46k op/s 2023-09-12T16:16:02.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:02 smithi159 ceph-mon[102522]: pgmap v685: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 24 MiB/s wr, 1.46k op/s 2023-09-12T16:16:02.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:02 smithi203 ceph-mon[103431]: pgmap v685: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 24 MiB/s wr, 1.46k op/s 2023-09-12T16:16:04.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:04 smithi130 ceph-mon[98670]: pgmap v686: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 985 op/s 2023-09-12T16:16:04.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:04 smithi159 ceph-mon[102522]: pgmap v686: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 985 op/s 2023-09-12T16:16:04.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:04 smithi203 ceph-mon[103431]: pgmap v686: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 985 op/s 2023-09-12T16:16:05.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 615.0 s, 497.9 tps, lat 164.725 ms stddev 152.666 2023-09-12T16:16:06.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:06 smithi130 ceph-mon[98670]: pgmap v687: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2023-09-12T16:16:06.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:06 smithi159 ceph-mon[102522]: pgmap v687: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2023-09-12T16:16:06.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:06 smithi203 ceph-mon[103431]: pgmap v687: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2023-09-12T16:16:08.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:08 smithi159 ceph-mon[102522]: pgmap v688: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:16:08.733 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:08 smithi130 ceph-mon[98670]: pgmap v688: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:16:08.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:08 smithi203 ceph-mon[103431]: pgmap v688: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:16:10.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 620.0 s, 694.6 tps, lat 181.553 ms stddev 596.969 2023-09-12T16:16:10.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:10 smithi130 ceph-mon[98670]: pgmap v689: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 24 MiB/s wr, 2.16k op/s 2023-09-12T16:16:10.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:10 smithi159 ceph-mon[102522]: pgmap v689: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 24 MiB/s wr, 2.16k op/s 2023-09-12T16:16:10.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:10 smithi203 ceph-mon[103431]: pgmap v689: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 24 MiB/s wr, 2.16k op/s 2023-09-12T16:16:12.312 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:16:12.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.618+0000 7fa123177700 1 -- 172.21.15.130:0/311441994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c075370 msgr2=0x7fa11c077800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:12.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.618+0000 7fa123177700 1 --2- 172.21.15.130:0/311441994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c075370 0x7fa11c077800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa11400b0a0 tx=0x7fa11400b3b0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.618+0000 7fa123177700 1 -- 172.21.15.130:0/311441994 shutdown_connections 2023-09-12T16:16:12.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.618+0000 7fa123177700 1 --2- 172.21.15.130:0/311441994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa11c077d40 0x7fa11c07a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.618+0000 7fa123177700 1 --2- 172.21.15.130:0/311441994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa11c072490 0x7fa11c0728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.618+0000 7fa123177700 1 --2- 172.21.15.130:0/311441994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c075370 0x7fa11c077800 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.618+0000 7fa123177700 1 -- 172.21.15.130:0/311441994 >> 172.21.15.130:0/311441994 conn(0x7fa11c06d3b0 msgr2=0x7fa11c06f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:12.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.618+0000 7fa123177700 1 -- 172.21.15.130:0/311441994 shutdown_connections 2023-09-12T16:16:12.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.618+0000 7fa123177700 1 -- 172.21.15.130:0/311441994 wait complete. 2023-09-12T16:16:12.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.619+0000 7fa123177700 1 Processor -- start 2023-09-12T16:16:12.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.619+0000 7fa123177700 1 -- start start 2023-09-12T16:16:12.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.619+0000 7fa123177700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa11c072490 0x7fa11c1b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.619+0000 7fa123177700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa11c075370 0x7fa11c1bb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.619+0000 7fa123177700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c077d40 0x7fa11c1bd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.619+0000 7fa123177700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1be030 con 0x7fa11c072490 2023-09-12T16:16:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.619+0000 7fa123177700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1be1a0 con 0x7fa11c075370 2023-09-12T16:16:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.619+0000 7fa123177700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1be360 con 0x7fa11c077d40 2023-09-12T16:16:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa121714700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c077d40 0x7fa11c1bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa121714700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c077d40 0x7fa11c1bd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36196/0 (socket says 172.21.15.130:36196) 2023-09-12T16:16:12.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa121714700 1 -- 172.21.15.130:0/3155135119 learned_addr learned my addr 172.21.15.130:0/3155135119 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:12.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa120f13700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa11c072490 0x7fa11c1b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa11bfff700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa11c075370 0x7fa11c1bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa121714700 1 -- 172.21.15.130:0/3155135119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa11c075370 msgr2=0x7fa11c1bb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa121714700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa11c075370 0x7fa11c1bb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa121714700 1 -- 172.21.15.130:0/3155135119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa11c072490 msgr2=0x7fa11c1b8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa121714700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa11c072490 0x7fa11c1b8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa121714700 1 -- 172.21.15.130:0/3155135119 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa114009d00 con 0x7fa11c077d40 2023-09-12T16:16:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa121714700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c077d40 0x7fa11c1bd860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b4a0 tx=0x7fa11000b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:12.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa119ffb700 1 -- 172.21.15.130:0/3155135119 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110011830 con 0x7fa11c077d40 2023-09-12T16:16:12.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa119ffb700 1 -- 172.21.15.130:0/3155135119 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa110011e50 con 0x7fa11c077d40 2023-09-12T16:16:12.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa123177700 1 -- 172.21.15.130:0/3155135119 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c1be640 con 0x7fa11c077d40 2023-09-12T16:16:12.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa119ffb700 1 -- 172.21.15.130:0/3155135119 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110010e80 con 0x7fa11c077d40 2023-09-12T16:16:12.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.620+0000 7fa123177700 1 -- 172.21.15.130:0/3155135119 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1beb20 con 0x7fa11c077d40 2023-09-12T16:16:12.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.623+0000 7fa119ffb700 1 -- 172.21.15.130:0/3155135119 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fa110010380 con 0x7fa11c077d40 2023-09-12T16:16:12.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.624+0000 7fa119ffb700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa1040a0690 0x7fa1040a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.624+0000 7fa119ffb700 1 -- 172.21.15.130:0/3155135119 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa1100cb890 con 0x7fa11c077d40 2023-09-12T16:16:12.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.624+0000 7fa120f13700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa1040a0690 0x7fa1040a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.625+0000 7fa120f13700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa1040a0690 0x7fa1040a2b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0836e0 tx=0x7fa10c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:12.684 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.661+0000 7fa12f33db80 1 Processor -- start 2023-09-12T16:16:12.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.662+0000 7fa12f33db80 1 -- start start 2023-09-12T16:16:12.685 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.663+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 0x55f02de6f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.663+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de6d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.663+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de6e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.663+0000 7fa12f33db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02ddc6730 con 0x55f02de6ec00 2023-09-12T16:16:12.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.663+0000 7fa12f33db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02ddc4050 con 0x55f02de6de00 2023-09-12T16:16:12.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.663+0000 7fa12f33db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02ddbf940 con 0x55f02de6f5a0 2023-09-12T16:16:12.687 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.674+0000 7fa121714700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de6e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.674+0000 7fa121714700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de6e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39850/0 (socket says 172.21.15.130:39850) 2023-09-12T16:16:12.688 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.674+0000 7fa121714700 1 -- 172.21.15.130:0/2130382285 learned_addr learned my addr 172.21.15.130:0/2130382285 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:12.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.678+0000 7fa120f13700 1 --2- 172.21.15.130:0/2130382285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 0x55f02de6f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.679+0000 7fa120f13700 1 -- 172.21.15.130:0/2130382285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 msgr2=0x55f02de6d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:12.689 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.679+0000 7fa120f13700 1 --2- 172.21.15.130:0/2130382285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de6d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.679+0000 7fa120f13700 1 -- 172.21.15.130:0/2130382285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 msgr2=0x55f02de6e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:12.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.679+0000 7fa120f13700 1 --2- 172.21.15.130:0/2130382285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de6e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.690 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.679+0000 7fa120f13700 1 -- 172.21.15.130:0/2130382285 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f02dbebde0 con 0x55f02de6ec00 2023-09-12T16:16:12.691 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.680+0000 7fa120f13700 1 --2- 172.21.15.130:0/2130382285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 0x55f02de6f020 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa10c006fd0 tx=0x7fa10c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f711c82684935e32 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:12.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.681+0000 7fa102ffd700 1 -- 172.21.15.130:0/2130382285 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c00ee70 con 0x55f02de6ec00 2023-09-12T16:16:12.695 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.681+0000 7fa102ffd700 1 -- 172.21.15.130:0/2130382285 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10c015940 con 0x55f02de6ec00 2023-09-12T16:16:12.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.681+0000 7fa102ffd700 1 -- 172.21.15.130:0/2130382285 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0139d0 con 0x55f02de6ec00 2023-09-12T16:16:12.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.682+0000 7fa12f33db80 1 -- 172.21.15.130:0/2130382285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 msgr2=0x55f02de6f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:12.696 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.682+0000 7fa12f33db80 1 --2- 172.21.15.130:0/2130382285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 0x55f02de6f020 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa10c006fd0 tx=0x7fa10c011000 comp rx=0 tx=0).stop 2023-09-12T16:16:12.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.685+0000 7fa12f33db80 1 -- 172.21.15.130:0/2130382285 shutdown_connections 2023-09-12T16:16:12.697 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.685+0000 7fa12f33db80 1 --2- 172.21.15.130:0/2130382285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de6e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.698 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.685+0000 7fa12f33db80 1 --2- 172.21.15.130:0/2130382285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 0x55f02de6f020 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.698 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.685+0000 7fa12f33db80 1 --2- 172.21.15.130:0/2130382285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de6d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.698 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.685+0000 7fa12f33db80 1 -- 172.21.15.130:0/2130382285 >> 172.21.15.130:0/2130382285 conn(0x55f02dde2090 msgr2=0x55f02de68720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:12.709 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.697+0000 7fa12f33db80 1 -- 172.21.15.130:0/2130382285 shutdown_connections 2023-09-12T16:16:12.712 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.709+0000 7fa12f33db80 1 -- 172.21.15.130:0/2130382285 wait complete. 2023-09-12T16:16:12.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.709+0000 7fa12f33db80 1 -- start start 2023-09-12T16:16:12.713 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.709+0000 7fa12f33db80 1 Processor -- start 2023-09-12T16:16:12.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.715+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de77630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.715+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 0x55f02de77b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.715+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de7ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.715+0000 7fa12f33db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02dd5db70 con 0x55f02de6ec00 2023-09-12T16:16:12.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.715+0000 7fa12f33db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02dd5d870 con 0x55f02de6de00 2023-09-12T16:16:12.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.715+0000 7fa12f33db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02dc41920 con 0x55f02de6f5a0 2023-09-12T16:16:12.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.715+0000 7fa12f33db80 10 client.0 _ll_register_callbacks cb 0x55f02dca7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:16:12.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.716+0000 7fa120f13700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de77630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.716+0000 7fa120f13700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de77630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39854/0 (socket says 172.21.15.130:39854) 2023-09-12T16:16:12.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.716+0000 7fa120f13700 1 -- 172.21.15.130:0/1217148319 learned_addr learned my addr 172.21.15.130:0/1217148319 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:12.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.716+0000 7fa120f13700 1 -- 172.21.15.130:0/1217148319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 msgr2=0x55f02de7ebb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:16:12.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.716+0000 7fa120f13700 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de7ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.716+0000 7fa120f13700 1 -- 172.21.15.130:0/1217148319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 msgr2=0x55f02de77b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:12.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.716+0000 7fa120f13700 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 0x55f02de77b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.716+0000 7fa120f13700 1 -- 172.21.15.130:0/1217148319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c00e690 con 0x55f02de6de00 2023-09-12T16:16:12.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.716+0000 7fa120f13700 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de77630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0134b0 tx=0x7fa10c015760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e17af9449b1dd4b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:12.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.717+0000 7fa0fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:16:12.720 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.717+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1217148319 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c004610 con 0x55f02de6de00 2023-09-12T16:16:12.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.717+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1217148319 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa10c003740 con 0x55f02de6de00 2023-09-12T16:16:12.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.717+0000 7fa12f33db80 1 -- 172.21.15.130:0/1217148319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f02dd174f0 con 0x55f02de6de00 2023-09-12T16:16:12.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.717+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1217148319 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa10c003900 con 0x55f02de6de00 2023-09-12T16:16:12.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.717+0000 7fa12f33db80 10 client.25628 fetch_fsmap learned FSMap version 32 2023-09-12T16:16:12.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.717+0000 7fa12f33db80 1 -- 172.21.15.130:0/1217148319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f02dbd4c60 con 0x55f02de6de00 2023-09-12T16:16:12.730 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:12 smithi130 ceph-mon[98670]: pgmap v690: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 27 MiB/s wr, 2.53k op/s 2023-09-12T16:16:12.730 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.718+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1217148319 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f02dd174f0 con 0x55f02de6de00 2023-09-12T16:16:12.731 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.718+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1217148319 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa10c031460 con 0x55f02de6de00 2023-09-12T16:16:12.731 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.719+0000 7fa12f33db80 10 client.25628 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:16:12.731 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.719+0000 7fa12f33db80 10 client.25628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:16:12.731 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.719+0000 7fa12f33db80 20 client.25628 populate_metadata read hostname 'smithi130' 2023-09-12T16:16:12.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.719+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f02de832f0 0x55f02de85730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.719+0000 7fa12f33db80 4 client.25628 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:16:12.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.719+0000 7fa12f33db80 1 -- 172.21.15.130:0/1217148319 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f02ddd48a0 con 0x55f02de832f0 2023-09-12T16:16:12.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.719+0000 7fa11bfff700 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f02de832f0 0x55f02de85730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.724+0000 7fa0fbfff700 10 client.25628 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:16:12.732 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.724+0000 7fa11bfff700 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f02de832f0 0x55f02de85730 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca1cb3f11c8ab11 server_cookie=16af8549dff1be50 in_seq=0 out_seq=0 2023-09-12T16:16:12.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.724+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1217148319 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f02ddd48a0 con 0x55f02de832f0 2023-09-12T16:16:12.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.724+0000 7fa0fbfff700 10 client.25628 handle_command_reply: tid=0 2023-09-12T16:16:12.733 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.724+0000 7fa12f33db80 1 client.25628 shutdown 2023-09-12T16:16:12.736 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.732+0000 7fa12f33db80 1 -- 172.21.15.130:0/1217148319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 msgr2=0x55f02de77630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:12.737 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.732+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de77630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0134b0 tx=0x7fa10c015760 comp rx=0 tx=0).stop 2023-09-12T16:16:12.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.739+0000 7fa12f33db80 1 -- 172.21.15.130:0/1217148319 shutdown_connections 2023-09-12T16:16:12.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.739+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f02de832f0 0x55f02de85730 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.739+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de7ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.739+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6de00 0x55f02de77630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.739+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1217148319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6ec00 0x55f02de77b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.739+0000 7fa12f33db80 1 -- 172.21.15.130:0/1217148319 >> 172.21.15.130:0/1217148319 conn(0x55f02dde2090 msgr2=0x55f02de67dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:12.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.743+0000 7fa0fbfff700 0 client.25628 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:16:12.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.754+0000 7fa12f33db80 1 -- 172.21.15.130:0/1217148319 shutdown_connections 2023-09-12T16:16:12.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.757+0000 7fa12f33db80 1 -- 172.21.15.130:0/1217148319 wait complete. 2023-09-12T16:16:12.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.761+0000 7fa12f33db80 20 client.25628 trim_cache size 0 max 16384 2023-09-12T16:16:12.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.787+0000 7fa12f33db80 1 Processor -- start 2023-09-12T16:16:12.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa12f33db80 1 -- start start 2023-09-12T16:16:12.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6de00 0x55f02de75430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de71f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa12f33db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02de698c0 con 0x55f02de6de00 2023-09-12T16:16:12.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa12f33db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02ddd48a0 con 0x55f02de6ec00 2023-09-12T16:16:12.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa12f33db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02dd174f0 con 0x55f02de6f5a0 2023-09-12T16:16:12.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa11bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa11bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39862/0 (socket says 172.21.15.130:39862) 2023-09-12T16:16:12.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa11bfff700 1 -- 172.21.15.130:0/340727431 learned_addr learned my addr 172.21.15.130:0/340727431 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:12.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.788+0000 7fa121714700 1 --2- 172.21.15.130:0/340727431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de71f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.789+0000 7fa11bfff700 1 -- 172.21.15.130:0/340727431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 msgr2=0x55f02de71f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:12.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.789+0000 7fa11bfff700 1 --2- 172.21.15.130:0/340727431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de71f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.789+0000 7fa11bfff700 1 -- 172.21.15.130:0/340727431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6de00 msgr2=0x55f02de75430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:12.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.789+0000 7fa11bfff700 1 --2- 172.21.15.130:0/340727431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6de00 0x55f02de75430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.789+0000 7fa11bfff700 1 -- 172.21.15.130:0/340727431 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f02dc12250 con 0x55f02de6ec00 2023-09-12T16:16:12.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.789+0000 7fa11bfff700 1 --2- 172.21.15.130:0/340727431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa114009fd0 tx=0x7fa11400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=522a4ead667ee188 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:12.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.789+0000 7fa1037fe700 1 -- 172.21.15.130:0/340727431 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11400e040 con 0x55f02de6ec00 2023-09-12T16:16:12.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.789+0000 7fa1037fe700 1 -- 172.21.15.130:0/340727431 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa114007960 con 0x55f02de6ec00 2023-09-12T16:16:12.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.789+0000 7fa1037fe700 1 -- 172.21.15.130:0/340727431 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114005670 con 0x55f02de6ec00 2023-09-12T16:16:12.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.790+0000 7fa12f33db80 1 -- 172.21.15.130:0/340727431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 msgr2=0x55f02de7eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:12.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.790+0000 7fa12f33db80 1 --2- 172.21.15.130:0/340727431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa114009fd0 tx=0x7fa11400b960 comp rx=0 tx=0).stop 2023-09-12T16:16:12.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.790+0000 7fa12f33db80 1 -- 172.21.15.130:0/340727431 shutdown_connections 2023-09-12T16:16:12.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.790+0000 7fa12f33db80 1 --2- 172.21.15.130:0/340727431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6f5a0 0x55f02de71f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.790+0000 7fa12f33db80 1 --2- 172.21.15.130:0/340727431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6de00 0x55f02de75430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.790+0000 7fa12f33db80 1 --2- 172.21.15.130:0/340727431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.790+0000 7fa12f33db80 1 -- 172.21.15.130:0/340727431 >> 172.21.15.130:0/340727431 conn(0x55f02dde2090 msgr2=0x55f02de85300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:12.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.791+0000 7fa12f33db80 1 -- 172.21.15.130:0/340727431 shutdown_connections 2023-09-12T16:16:12.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.792+0000 7fa12f33db80 1 -- 172.21.15.130:0/340727431 wait complete. 2023-09-12T16:16:12.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.792+0000 7fa12f33db80 1 -- start start 2023-09-12T16:16:12.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa12f33db80 1 Processor -- start 2023-09-12T16:16:12.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6de00 0x55f02de7e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa12f33db80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6f5a0 0x55f02de7b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa12f33db80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02de7b8b0 con 0x55f02de6f5a0 2023-09-12T16:16:12.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa12f33db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02de7ba20 con 0x55f02de6ec00 2023-09-12T16:16:12.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa12f33db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02de7bbe0 con 0x55f02de6de00 2023-09-12T16:16:12.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa12f33db80 10 client.0 _ll_register_callbacks cb 0x7fa10c00e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:16:12.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa11bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa11bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39868/0 (socket says 172.21.15.130:39868) 2023-09-12T16:16:12.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa121714700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6f5a0 0x55f02de7b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa11bfff700 1 -- 172.21.15.130:0/1081074075 learned_addr learned my addr 172.21.15.130:0/1081074075 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:12.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.793+0000 7fa120f13700 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6de00 0x55f02de7e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa11bfff700 1 -- 172.21.15.130:0/1081074075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6de00 msgr2=0x55f02de7e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:12.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa11bfff700 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6de00 0x55f02de7e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa11bfff700 1 -- 172.21.15.130:0/1081074075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6f5a0 msgr2=0x55f02de7b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:12.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa11bfff700 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6f5a0 0x55f02de7b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa11bfff700 1 -- 172.21.15.130:0/1081074075 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c07ac00 con 0x55f02de6ec00 2023-09-12T16:16:12.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa120f13700 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6de00 0x55f02de7e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:16:12.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa11bfff700 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa114005280 tx=0x7fa1140050d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9af06e39d212381 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:12.802 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa0fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:16:12.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1081074075 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11400e040 con 0x55f02de6ec00 2023-09-12T16:16:12.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1081074075 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa114003a00 con 0x55f02de6ec00 2023-09-12T16:16:12.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.794+0000 7fa12f33db80 1 -- 172.21.15.130:0/1081074075 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa10c015940 con 0x55f02de6ec00 2023-09-12T16:16:12.803 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.795+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1081074075 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa114003bc0 con 0x55f02de6ec00 2023-09-12T16:16:12.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa12f33db80 10 client.25634 fetch_fsmap learned FSMap version 32 2023-09-12T16:16:12.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa12f33db80 1 -- 172.21.15.130:0/1081074075 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f02dbea2b0 con 0x55f02de6ec00 2023-09-12T16:16:12.804 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1081074075 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10c015940 con 0x55f02de6ec00 2023-09-12T16:16:12.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1081074075 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa11400b0d0 con 0x55f02de6ec00 2023-09-12T16:16:12.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa12f33db80 10 client.25634 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:16:12.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa12f33db80 10 client.25634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:16:12.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa12f33db80 20 client.25634 populate_metadata read hostname 'smithi130' 2023-09-12T16:16:12.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f02dee9510 0x55f02deeb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:12.805 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa12f33db80 4 client.25634 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:16:12.806 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.799+0000 7fa12f33db80 1 -- 172.21.15.130:0/1081074075 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f02ddb8920 con 0x55f02dee9510 2023-09-12T16:16:12.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.802+0000 7fa120f13700 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f02dee9510 0x55f02deeb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:12.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.803+0000 7fa0fbfff700 10 client.25634 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:16:12.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.803+0000 7fa120f13700 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f02dee9510 0x55f02deeb930 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f00d0fde12ec4e0 server_cookie=96c27fd71ba65f55 in_seq=0 out_seq=0 2023-09-12T16:16:12.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa0fbfff700 1 -- 172.21.15.130:0/1081074075 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f02ddb8920 con 0x55f02dee9510 2023-09-12T16:16:12.808 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa0fbfff700 10 client.25634 handle_command_reply: tid=0 2023-09-12T16:16:12.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 client.25634 shutdown 2023-09-12T16:16:12.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 -- 172.21.15.130:0/1081074075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 msgr2=0x55f02de7eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:12.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa114005280 tx=0x7fa1140050d0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 -- 172.21.15.130:0/1081074075 shutdown_connections 2023-09-12T16:16:12.809 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55f02dee9510 0x55f02deeb930 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f02de6f5a0 0x55f02de7b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f02de6de00 0x55f02de7e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa0fbfff700 0 client.25634 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:16:12.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 --2- 172.21.15.130:0/1081074075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f02de6ec00 0x55f02de7eb50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 -- 172.21.15.130:0/1081074075 >> 172.21.15.130:0/1081074075 conn(0x55f02dde2090 msgr2=0x55f02de75430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:12.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.804+0000 7fa12f33db80 1 -- 172.21.15.130:0/1081074075 shutdown_connections 2023-09-12T16:16:12.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.805+0000 7fa12f33db80 1 -- 172.21.15.130:0/1081074075 wait complete. 2023-09-12T16:16:12.811 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:16:12.812 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:16:12.812 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:16:12.812 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:16:12.812 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:16:12.812 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:16:12.813 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:16:12.813 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:16:12.813 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:16:12.813 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:16:12.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa12f33db80 20 client.25634 trim_cache size 0 max 16384 2023-09-12T16:16:12.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 -- 172.21.15.130:0/3155135119 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa1040a0690 msgr2=0x7fa1040a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:12.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa1040a0690 0x7fa1040a2b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0836e0 tx=0x7fa10c009190 comp rx=0 tx=0).stop 2023-09-12T16:16:12.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 -- 172.21.15.130:0/3155135119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c077d40 msgr2=0x7fa11c1bd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:12.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c077d40 0x7fa11c1bd860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b4a0 tx=0x7fa11000b7b0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 -- 172.21.15.130:0/3155135119 shutdown_connections 2023-09-12T16:16:12.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa1040a0690 0x7fa1040a2b50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa11c077d40 0x7fa11c1bd860 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa11c072490 0x7fa11c1b8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 --2- 172.21.15.130:0/3155135119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa11c075370 0x7fa11c1bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:12.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.808+0000 7fa123177700 1 -- 172.21.15.130:0/3155135119 >> 172.21.15.130:0/3155135119 conn(0x7fa11c06d3b0 msgr2=0x7fa11c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:12.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.809+0000 7fa123177700 1 -- 172.21.15.130:0/3155135119 shutdown_connections 2023-09-12T16:16:12.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:12.809+0000 7fa123177700 1 -- 172.21.15.130:0/3155135119 wait complete. 2023-09-12T16:16:12.827 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:16:12.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:12 smithi159 ceph-mon[102522]: pgmap v690: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 27 MiB/s wr, 2.53k op/s 2023-09-12T16:16:12.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:12 smithi203 ceph-mon[103431]: pgmap v690: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 27 MiB/s wr, 2.53k op/s 2023-09-12T16:16:14.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:14 smithi130 ceph-mon[98670]: pgmap v691: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 2.46k op/s 2023-09-12T16:16:14.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:14 smithi159 ceph-mon[102522]: pgmap v691: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 2.46k op/s 2023-09-12T16:16:15.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:14 smithi203 ceph-mon[103431]: pgmap v691: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 2.46k op/s 2023-09-12T16:16:15.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 625.0 s, 1914.2 tps, lat 51.410 ms stddev 41.355 2023-09-12T16:16:16.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:16 smithi159 ceph-mon[102522]: pgmap v692: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 24 MiB/s wr, 2.37k op/s 2023-09-12T16:16:16.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:16 smithi130 ceph-mon[98670]: pgmap v692: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 24 MiB/s wr, 2.37k op/s 2023-09-12T16:16:16.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:16 smithi203 ceph-mon[103431]: pgmap v692: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 24 MiB/s wr, 2.37k op/s 2023-09-12T16:16:18.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:18 smithi130 ceph-mon[98670]: pgmap v693: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 31 MiB/s wr, 2.80k op/s 2023-09-12T16:16:18.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:18 smithi159 ceph-mon[102522]: pgmap v693: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 31 MiB/s wr, 2.80k op/s 2023-09-12T16:16:18.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:18 smithi203 ceph-mon[103431]: pgmap v693: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 31 MiB/s wr, 2.80k op/s 2023-09-12T16:16:20.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 630.0 s, 309.6 tps, lat 291.594 ms stddev 394.745 2023-09-12T16:16:20.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:20 smithi130 ceph-mon[98670]: pgmap v694: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 24 MiB/s wr, 1.85k op/s 2023-09-12T16:16:20.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:20 smithi159 ceph-mon[102522]: pgmap v694: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 24 MiB/s wr, 1.85k op/s 2023-09-12T16:16:20.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:20 smithi203 ceph-mon[103431]: pgmap v694: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 24 MiB/s wr, 1.85k op/s 2023-09-12T16:16:22.981 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:22 smithi130 ceph-mon[98670]: pgmap v695: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 30 MiB/s wr, 2.39k op/s 2023-09-12T16:16:22.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:22 smithi159 ceph-mon[102522]: pgmap v695: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 30 MiB/s wr, 2.39k op/s 2023-09-12T16:16:22.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:22 smithi203 ceph-mon[103431]: pgmap v695: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 30 MiB/s wr, 2.39k op/s 2023-09-12T16:16:24.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:24 smithi130 ceph-mon[98670]: pgmap v696: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 29 MiB/s wr, 2.41k op/s 2023-09-12T16:16:24.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:24 smithi159 ceph-mon[102522]: pgmap v696: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 29 MiB/s wr, 2.41k op/s 2023-09-12T16:16:24.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:24 smithi203 ceph-mon[103431]: pgmap v696: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 29 MiB/s wr, 2.41k op/s 2023-09-12T16:16:25.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 635.0 s, 1626.2 tps, lat 68.420 ms stddev 110.494 2023-09-12T16:16:26.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:26 smithi130 ceph-mon[98670]: pgmap v697: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 27 MiB/s wr, 2.30k op/s 2023-09-12T16:16:26.989 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:26 smithi159 ceph-mon[102522]: pgmap v697: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 27 MiB/s wr, 2.30k op/s 2023-09-12T16:16:26.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:26 smithi203 ceph-mon[103431]: pgmap v697: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 27 MiB/s wr, 2.30k op/s 2023-09-12T16:16:28.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:28 smithi159 ceph-mon[102522]: pgmap v698: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 31 MiB/s wr, 2.51k op/s 2023-09-12T16:16:28.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:28 smithi130 ceph-mon[98670]: pgmap v698: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 31 MiB/s wr, 2.51k op/s 2023-09-12T16:16:28.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:28 smithi203 ceph-mon[103431]: pgmap v698: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 31 MiB/s wr, 2.51k op/s 2023-09-12T16:16:30.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 640.0 s, 1413.0 tps, lat 70.278 ms stddev 55.075 2023-09-12T16:16:30.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:30 smithi159 ceph-mon[102522]: pgmap v699: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 24 MiB/s wr, 1.98k op/s 2023-09-12T16:16:30.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:30 smithi130 ceph-mon[98670]: pgmap v699: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 24 MiB/s wr, 1.98k op/s 2023-09-12T16:16:30.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:30 smithi203 ceph-mon[103431]: pgmap v699: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 24 MiB/s wr, 1.98k op/s 2023-09-12T16:16:32.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:32 smithi130 ceph-mon[98670]: pgmap v700: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 25 MiB/s wr, 1.90k op/s 2023-09-12T16:16:32.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:32 smithi159 ceph-mon[102522]: pgmap v700: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 25 MiB/s wr, 1.90k op/s 2023-09-12T16:16:32.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:32 smithi203 ceph-mon[103431]: pgmap v700: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 25 MiB/s wr, 1.90k op/s 2023-09-12T16:16:34.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:34 smithi159 ceph-mon[102522]: pgmap v701: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 1.32k op/s 2023-09-12T16:16:34.980 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:34 smithi130 ceph-mon[98670]: pgmap v701: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 1.32k op/s 2023-09-12T16:16:35.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:34 smithi203 ceph-mon[103431]: pgmap v701: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 1.32k op/s 2023-09-12T16:16:35.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 645.0 s, 1030.3 tps, lat 88.917 ms stddev 84.837 2023-09-12T16:16:36.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:36 smithi130 ceph-mon[98670]: pgmap v702: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 938 op/s 2023-09-12T16:16:36.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:36 smithi203 ceph-mon[103431]: pgmap v702: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 938 op/s 2023-09-12T16:16:37.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:36 smithi159 ceph-mon[102522]: pgmap v702: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 19 MiB/s wr, 938 op/s 2023-09-12T16:16:38.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:38 smithi130 ceph-mon[98670]: pgmap v703: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 21 MiB/s wr, 1.35k op/s 2023-09-12T16:16:38.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:38 smithi159 ceph-mon[102522]: pgmap v703: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 21 MiB/s wr, 1.35k op/s 2023-09-12T16:16:38.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:38 smithi203 ceph-mon[103431]: pgmap v703: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 21 MiB/s wr, 1.35k op/s 2023-09-12T16:16:40.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 650.0 s, 612.2 tps, lat 166.041 ms stddev 286.977 2023-09-12T16:16:40.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:40 smithi130 ceph-mon[98670]: pgmap v704: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-09-12T16:16:40.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:40 smithi159 ceph-mon[102522]: pgmap v704: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-09-12T16:16:41.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:40 smithi203 ceph-mon[103431]: pgmap v704: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-09-12T16:16:42.828 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:16:42.893 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:42 smithi130 ceph-mon[98670]: pgmap v705: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-09-12T16:16:42.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:42 smithi159 ceph-mon[102522]: pgmap v705: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-09-12T16:16:42.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:42 smithi203 ceph-mon[103431]: pgmap v705: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-09-12T16:16:43.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.100+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/1829172891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e5810c940 msgr2=0x7f0e5810cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:43.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.100+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/1829172891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e5810c940 0x7f0e5810cdc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e5001c0d0 tx=0x7f0e5001c3e0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.103+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/1829172891 shutdown_connections 2023-09-12T16:16:43.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.103+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/1829172891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e5810c940 0x7f0e5810cdc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.103+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/1829172891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e58107600 0x7f0e58109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.103+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/1829172891 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e58109f70 0x7f0e5810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.103+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/1829172891 >> 172.21.15.130:0/1829172891 conn(0x7f0e5806d3b0 msgr2=0x7f0e5806f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:43.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.109+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/1829172891 shutdown_connections 2023-09-12T16:16:43.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.114+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/1829172891 wait complete. 2023-09-12T16:16:43.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.115+0000 7f0e5f3e5700 1 Processor -- start 2023-09-12T16:16:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.118+0000 7f0e5f3e5700 1 -- start start 2023-09-12T16:16:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.118+0000 7f0e5f3e5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e58107600 0x7f0e5819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.118+0000 7f0e5f3e5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e58109f70 0x7f0e5819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.118+0000 7f0e5f3e5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e5810c940 0x7f0e581a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.118+0000 7f0e5f3e5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e581a44f0 con 0x7f0e58109f70 2023-09-12T16:16:43.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.118+0000 7f0e5f3e5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e581a4630 con 0x7f0e5810c940 2023-09-12T16:16:43.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.118+0000 7f0e5f3e5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e581a47c0 con 0x7f0e58107600 2023-09-12T16:16:43.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.119+0000 7f0e5c980700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e58109f70 0x7f0e5819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.119+0000 7f0e5c980700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e58109f70 0x7f0e5819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56626/0 (socket says 172.21.15.130:56626) 2023-09-12T16:16:43.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.119+0000 7f0e5c980700 1 -- 172.21.15.130:0/2979975940 learned_addr learned my addr 172.21.15.130:0/2979975940 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:43.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.119+0000 7f0e5c980700 1 -- 172.21.15.130:0/2979975940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e58107600 msgr2=0x7f0e5819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:43.121 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.119+0000 7f0e5c980700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e58107600 0x7f0e5819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.119+0000 7f0e5c980700 1 -- 172.21.15.130:0/2979975940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e5810c940 msgr2=0x7f0e581a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:43.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.119+0000 7f0e5c980700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e5810c940 0x7f0e581a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.122 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.119+0000 7f0e5c980700 1 -- 172.21.15.130:0/2979975940 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5001ad10 con 0x7f0e58109f70 2023-09-12T16:16:43.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.124+0000 7f0e5c980700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e58109f70 0x7f0e5819ceb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800b530 tx=0x7f0e4800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:43.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.136+0000 7f0e4e7fc700 1 -- 172.21.15.130:0/2979975940 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48011830 con 0x7f0e58109f70 2023-09-12T16:16:43.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.136+0000 7f0e4e7fc700 1 -- 172.21.15.130:0/2979975940 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e48011e50 con 0x7f0e58109f70 2023-09-12T16:16:43.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.136+0000 7f0e4e7fc700 1 -- 172.21.15.130:0/2979975940 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48010e70 con 0x7f0e58109f70 2023-09-12T16:16:43.136 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.136+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/2979975940 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e581a4aa0 con 0x7f0e58109f70 2023-09-12T16:16:43.137 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.136+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/2979975940 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e581a4f60 con 0x7f0e58109f70 2023-09-12T16:16:43.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.142+0000 7f0e6b5abb80 1 Processor -- start 2023-09-12T16:16:43.142 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.142+0000 7f0e4e7fc700 1 -- 172.21.15.130:0/2979975940 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f0e48010510 con 0x7f0e58109f70 2023-09-12T16:16:43.147 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.142+0000 7f0e6b5abb80 1 -- start start 2023-09-12T16:16:43.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.148+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f2955020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.148+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f2953800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.148+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2953e00 0x5614f29542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.148+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f28ac730 con 0x5614f2954c00 2023-09-12T16:16:43.148 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.148+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f28aa050 con 0x5614f29555a0 2023-09-12T16:16:43.149 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.148+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f28a5940 con 0x5614f2953e00 2023-09-12T16:16:43.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.148+0000 7f0e5d982700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f2955020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.148+0000 7f0e5d982700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f2955020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56632/0 (socket says 172.21.15.130:56632) 2023-09-12T16:16:43.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.148+0000 7f0e5d982700 1 -- 172.21.15.130:0/4091411228 learned_addr learned my addr 172.21.15.130:0/4091411228 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:43.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e5d982700 1 -- 172.21.15.130:0/4091411228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2953e00 msgr2=0x5614f29542b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:16:43.150 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e5d982700 1 --2- 172.21.15.130:0/4091411228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2953e00 0x5614f29542b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e5d982700 1 -- 172.21.15.130:0/4091411228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 msgr2=0x5614f2953800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:43.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e5d982700 1 --2- 172.21.15.130:0/4091411228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f2953800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e5d982700 1 -- 172.21.15.130:0/4091411228 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614f26d1de0 con 0x5614f2954c00 2023-09-12T16:16:43.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e4e7fc700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0e440a0840 0x7f0e440a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.151 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e5d982700 1 --2- 172.21.15.130:0/4091411228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f2955020 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0e50019dc0 tx=0x7f0e50019df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5015b3bdc14187ab server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:43.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e5d181700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0e440a0840 0x7f0e440a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e4e7fc700 1 -- 172.21.15.130:0/2979975940 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0e480cba00 con 0x7f0e58109f70 2023-09-12T16:16:43.152 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e437fe700 1 -- 172.21.15.130:0/4091411228 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50030070 con 0x5614f2954c00 2023-09-12T16:16:43.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e437fe700 1 -- 172.21.15.130:0/4091411228 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e50003a00 con 0x5614f2954c00 2023-09-12T16:16:43.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.149+0000 7f0e437fe700 1 -- 172.21.15.130:0/4091411228 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5001f040 con 0x5614f2954c00 2023-09-12T16:16:43.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.151+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/4091411228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 msgr2=0x5614f2955020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:43.153 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.151+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/4091411228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f2955020 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0e50019dc0 tx=0x7f0e50019df0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.151+0000 7f0e5d181700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0e440a0840 0x7f0e440a2d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009770 tx=0x7f0e54006d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:43.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.151+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/4091411228 shutdown_connections 2023-09-12T16:16:43.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.151+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/4091411228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2953e00 0x5614f29542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.151+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/4091411228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f2955020 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.154 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.151+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/4091411228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f2953800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.151+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/4091411228 >> 172.21.15.130:0/4091411228 conn(0x5614f28c8090 msgr2=0x5614f294e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:43.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.153+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/4091411228 shutdown_connections 2023-09-12T16:16:43.155 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.153+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/4091411228 wait complete. 2023-09-12T16:16:43.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.153+0000 7f0e6b5abb80 1 -- start start 2023-09-12T16:16:43.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.154+0000 7f0e6b5abb80 1 Processor -- start 2023-09-12T16:16:43.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.154+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2953e00 0x5614f295d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.154+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f295db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.154+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f2964b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.156 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.154+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f2843b70 con 0x5614f2954c00 2023-09-12T16:16:43.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.154+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f2843870 con 0x5614f29555a0 2023-09-12T16:16:43.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.154+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f2727920 con 0x5614f2953e00 2023-09-12T16:16:43.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.154+0000 7f0e6b5abb80 10 client.0 _ll_register_callbacks cb 0x5614f278da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:16:43.157 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.155+0000 7f0e5c980700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f295db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.155+0000 7f0e5c980700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f295db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56638/0 (socket says 172.21.15.130:56638) 2023-09-12T16:16:43.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.155+0000 7f0e5c980700 1 -- 172.21.15.130:0/425741222 learned_addr learned my addr 172.21.15.130:0/425741222 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:43.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.156+0000 7f0e5c980700 1 -- 172.21.15.130:0/425741222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2953e00 msgr2=0x5614f295d5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:16:43.159 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.156+0000 7f0e5c980700 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2953e00 0x5614f295d5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.156+0000 7f0e5c980700 1 -- 172.21.15.130:0/425741222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 msgr2=0x5614f2964b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:16:43.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.156+0000 7f0e5c980700 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f2964b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.156+0000 7f0e5c980700 1 -- 172.21.15.130:0/425741222 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5810d940 con 0x5614f2954c00 2023-09-12T16:16:43.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.157+0000 7f0e5c980700 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f295db20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0e48098db0 tx=0x7f0e48098880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d689ccce8919e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:43.160 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.157+0000 7f0e40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2023-09-12T16:16:43.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.157+0000 7f0e40ff9700 1 -- 172.21.15.130:0/425741222 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48098510 con 0x5614f2954c00 2023-09-12T16:16:43.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.157+0000 7f0e40ff9700 1 -- 172.21.15.130:0/425741222 <== mon.0 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0e480b1200 con 0x5614f2954c00 2023-09-12T16:16:43.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.157+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/425741222 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614f27fd4f0 con 0x5614f2954c00 2023-09-12T16:16:43.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.157+0000 7f0e40ff9700 1 -- 172.21.15.130:0/425741222 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e480a0e90 con 0x5614f2954c00 2023-09-12T16:16:43.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.157+0000 7f0e6b5abb80 10 client.16161 fetch_fsmap learned FSMap version 32 2023-09-12T16:16:43.161 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.157+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/425741222 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5614f26bac60 con 0x5614f2954c00 2023-09-12T16:16:43.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.158+0000 7f0e40ff9700 1 -- 172.21.15.130:0/425741222 <== mon.0 v2:172.21.15.130:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614f27fd4f0 con 0x5614f2954c00 2023-09-12T16:16:43.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.158+0000 7f0e40ff9700 1 -- 172.21.15.130:0/425741222 <== mon.0 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0e48094db0 con 0x5614f2954c00 2023-09-12T16:16:43.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.159+0000 7f0e6b5abb80 10 client.16161 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:16:43.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.159+0000 7f0e6b5abb80 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:16:43.162 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.159+0000 7f0e6b5abb80 20 client.16161 populate_metadata read hostname 'smithi130' 2023-09-12T16:16:43.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.159+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5614f2969350 0x5614f296b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.159+0000 7f0e6b5abb80 4 client.16161 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:16:43.163 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.159+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/425741222 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614f28ba8a0 con 0x5614f2969350 2023-09-12T16:16:43.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.160+0000 7f0e5d181700 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5614f2969350 0x5614f296b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.164+0000 7f0e40ff9700 10 client.16161 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:16:43.164 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.164+0000 7f0e5d181700 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5614f2969350 0x5614f296b770 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceac60a0f349b606 server_cookie=bfe0e3bd0b8e9f4a in_seq=0 out_seq=0 2023-09-12T16:16:43.165 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e40ff9700 1 -- 172.21.15.130:0/425741222 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5614f28ba8a0 con 0x5614f2969350 2023-09-12T16:16:43.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e40ff9700 10 client.16161 handle_command_reply: tid=0 2023-09-12T16:16:43.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 client.16161 shutdown 2023-09-12T16:16:43.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/425741222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 msgr2=0x5614f295db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:43.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f295db20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0e48098db0 tx=0x7f0e48098880 comp rx=0 tx=0).stop 2023-09-12T16:16:43.166 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/425741222 shutdown_connections 2023-09-12T16:16:43.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5614f2969350 0x5614f296b770 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f2964b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e40ff9700 0 client.16161 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:16:43.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2953e00 0x5614f295d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/425741222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2954c00 0x5614f295db20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.167 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/425741222 >> 172.21.15.130:0/425741222 conn(0x5614f28c8090 msgr2=0x5614f294ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:43.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/425741222 shutdown_connections 2023-09-12T16:16:43.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/425741222 wait complete. 2023-09-12T16:16:43.168 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.165+0000 7f0e6b5abb80 20 client.16161 trim_cache size 0 max 16384 2023-09-12T16:16:43.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.194+0000 7f0e6b5abb80 1 Processor -- start 2023-09-12T16:16:43.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e6b5abb80 1 -- start start 2023-09-12T16:16:43.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f2953e00 0x5614f2953710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 0x5614f2964b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f29555a0 0x5614f295b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f295ca90 con 0x5614f29555a0 2023-09-12T16:16:43.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f2954710 con 0x5614f2953e00 2023-09-12T16:16:43.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f2966820 con 0x5614f2954c00 2023-09-12T16:16:43.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d181700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f2953e00 0x5614f2953710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d181700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f2953e00 0x5614f2953710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39904/0 (socket says 172.21.15.130:39904) 2023-09-12T16:16:43.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d181700 1 -- 172.21.15.130:0/3050649289 learned_addr learned my addr 172.21.15.130:0/3050649289 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:43.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d982700 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f29555a0 0x5614f295b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5c980700 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 0x5614f2964b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d181700 1 -- 172.21.15.130:0/3050649289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 msgr2=0x5614f2964b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:43.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d181700 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 0x5614f2964b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d181700 1 -- 172.21.15.130:0/3050649289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f29555a0 msgr2=0x5614f295b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:43.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d181700 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f29555a0 0x5614f295b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d181700 1 -- 172.21.15.130:0/3050649289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614f26f8250 con 0x5614f2953e00 2023-09-12T16:16:43.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5d982700 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f29555a0 0x5614f295b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:16:43.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.195+0000 7f0e5c980700 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 0x5614f2964b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T16:16:43.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.196+0000 7f0e5d181700 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f2953e00 0x5614f2953710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006fd0 tx=0x7f0e5400e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da1da422b792cb57 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:43.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.197+0000 7f0e43fff700 1 -- 172.21.15.130:0/3050649289 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5401a040 con 0x5614f2953e00 2023-09-12T16:16:43.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.197+0000 7f0e43fff700 1 -- 172.21.15.130:0/3050649289 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e5401fdc0 con 0x5614f2953e00 2023-09-12T16:16:43.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.197+0000 7f0e43fff700 1 -- 172.21.15.130:0/3050649289 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54013e70 con 0x5614f2953e00 2023-09-12T16:16:43.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.198+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/3050649289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f2953e00 msgr2=0x5614f2953710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:43.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.198+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f2953e00 0x5614f2953710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006fd0 tx=0x7f0e5400e000 comp rx=0 tx=0).stop 2023-09-12T16:16:43.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.198+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/3050649289 shutdown_connections 2023-09-12T16:16:43.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.198+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f29555a0 0x5614f295b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.198+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f2953e00 0x5614f2953710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.198+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/3050649289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 0x5614f2964b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.198+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/3050649289 >> 172.21.15.130:0/3050649289 conn(0x5614f28c8090 msgr2=0x5614f2965330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:43.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.198+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/3050649289 shutdown_connections 2023-09-12T16:16:43.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.198+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/3050649289 wait complete. 2023-09-12T16:16:43.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e6b5abb80 1 -- start start 2023-09-12T16:16:43.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e6b5abb80 1 Processor -- start 2023-09-12T16:16:43.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2953e00 0x5614f2964690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 0x5614f2962ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e6b5abb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f29631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f2964eb0 con 0x5614f2953e00 2023-09-12T16:16:43.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f2963720 con 0x5614f29555a0 2023-09-12T16:16:43.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e6b5abb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f29638e0 con 0x5614f2954c00 2023-09-12T16:16:43.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e6b5abb80 10 client.0 _ll_register_callbacks cb 0x7f0e48098c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:16:43.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5d181700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2953e00 0x5614f2964690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.207 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5d982700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f29631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5d181700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2953e00 0x5614f2964690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56652/0 (socket says 172.21.15.130:56652) 2023-09-12T16:16:43.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5c980700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 0x5614f2962ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5d181700 1 -- 172.21.15.130:0/1255878935 learned_addr learned my addr 172.21.15.130:0/1255878935 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:16:43.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5d982700 1 -- 172.21.15.130:0/1255878935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 msgr2=0x5614f2962ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:43.208 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5d982700 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 0x5614f2962ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5d982700 1 -- 172.21.15.130:0/1255878935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2953e00 msgr2=0x5614f2964690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:16:43.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5d982700 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2953e00 0x5614f2964690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.199+0000 7f0e5d982700 1 -- 172.21.15.130:0/1255878935 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e54012830 con 0x5614f29555a0 2023-09-12T16:16:43.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e5d982700 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f29631e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0e50000f80 tx=0x7f0e5001a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140ef737f4d5746a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:16:43.209 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:16:43.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e40ff9700 1 -- 172.21.15.130:0/1255878935 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50030040 con 0x5614f29555a0 2023-09-12T16:16:43.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/1255878935 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e50003a00 con 0x5614f29555a0 2023-09-12T16:16:43.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e40ff9700 1 -- 172.21.15.130:0/1255878935 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0e50018900 con 0x5614f29555a0 2023-09-12T16:16:43.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e40ff9700 1 -- 172.21.15.130:0/1255878935 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e5001f040 con 0x5614f29555a0 2023-09-12T16:16:43.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e6b5abb80 10 client.25643 fetch_fsmap learned FSMap version 32 2023-09-12T16:16:43.210 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/1255878935 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5614f26d02b0 con 0x5614f29555a0 2023-09-12T16:16:43.211 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e40ff9700 1 -- 172.21.15.130:0/1255878935 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e50003a00 con 0x5614f29555a0 2023-09-12T16:16:43.211 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e40ff9700 1 -- 172.21.15.130:0/1255878935 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0e5002a040 con 0x5614f29555a0 2023-09-12T16:16:43.211 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e6b5abb80 10 client.25643 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:16:43.211 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e6b5abb80 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:16:43.211 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e6b5abb80 20 client.25643 populate_metadata read hostname 'smithi130' 2023-09-12T16:16:43.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5614f2969350 0x5614f29c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:16:43.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e6b5abb80 4 client.25643 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:16:43.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.200+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/1255878935 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614f289e920 con 0x5614f2969350 2023-09-12T16:16:43.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.201+0000 7f0e5d181700 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5614f2969350 0x5614f29c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:16:43.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.201+0000 7f0e40ff9700 10 client.25643 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:16:43.212 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.201+0000 7f0e5d181700 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5614f2969350 0x5614f29c03f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0087c03f9ed4b09 server_cookie=eef59271bcf0308e in_seq=0 out_seq=0 2023-09-12T16:16:43.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.201+0000 7f0e40ff9700 1 -- 172.21.15.130:0/1255878935 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5614f289e920 con 0x5614f2969350 2023-09-12T16:16:43.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.201+0000 7f0e40ff9700 10 client.25643 handle_command_reply: tid=0 2023-09-12T16:16:43.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.201+0000 7f0e6b5abb80 1 client.25643 shutdown 2023-09-12T16:16:43.213 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.201+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/1255878935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 msgr2=0x5614f29631e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:43.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.201+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f29631e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0e50000f80 tx=0x7f0e5001a300 comp rx=0 tx=0).stop 2023-09-12T16:16:43.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.202+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/1255878935 shutdown_connections 2023-09-12T16:16:43.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.202+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x5614f2969350 0x5614f29c03f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.202+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f29555a0 0x5614f29631e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.214 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.202+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614f2953e00 0x5614f2964690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.202+0000 7f0e6b5abb80 1 --2- 172.21.15.130:0/1255878935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614f2954c00 0x5614f2962ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.202+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/1255878935 >> 172.21.15.130:0/1255878935 conn(0x5614f28c8090 msgr2=0x5614f2965330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:43.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.202+0000 7f0e40ff9700 0 client.25643 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:16:43.215 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.203+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/1255878935 shutdown_connections 2023-09-12T16:16:43.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.203+0000 7f0e6b5abb80 1 -- 172.21.15.130:0/1255878935 wait complete. 2023-09-12T16:16:43.216 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.203+0000 7f0e6b5abb80 20 client.25643 trim_cache size 0 max 16384 2023-09-12T16:16:43.217 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:16:43.217 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:16:43.217 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:16:43.217 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:16:43.218 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:16:43.218 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:16:43.218 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:16:43.218 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:16:43.218 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:16:43.219 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:16:43.219 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.204+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0e440a0840 msgr2=0x7f0e440a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:43.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.204+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0e440a0840 0x7f0e440a2d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009770 tx=0x7f0e54006d40 comp rx=0 tx=0).stop 2023-09-12T16:16:43.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.204+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e58109f70 msgr2=0x7f0e5819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:16:43.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.204+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e58109f70 0x7f0e5819ceb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800b530 tx=0x7f0e4800b8f0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.205+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/2979975940 shutdown_connections 2023-09-12T16:16:43.220 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.205+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f0e440a0840 0x7f0e440a2d00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.205+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e5810c940 0x7f0e581a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.205+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e58107600 0x7f0e5819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.205+0000 7f0e5f3e5700 1 --2- 172.21.15.130:0/2979975940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e58109f70 0x7f0e5819ceb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:16:43.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.205+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/2979975940 >> 172.21.15.130:0/2979975940 conn(0x7f0e5806d3b0 msgr2=0x7f0e5806f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:16:43.221 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.205+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/2979975940 shutdown_connections 2023-09-12T16:16:43.222 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:16:43.205+0000 7f0e5f3e5700 1 -- 172.21.15.130:0/2979975940 wait complete. 2023-09-12T16:16:43.223 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:16:43.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:43 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:16:43.994 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:43 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:16:44.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:43 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:16:45.092 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 655.0 s, 1564.6 tps, lat 68.077 ms stddev 78.102 2023-09-12T16:16:45.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:45 smithi130 ceph-mon[98670]: pgmap v706: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 21 MiB/s wr, 1.71k op/s 2023-09-12T16:16:45.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.476 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:45 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:45 smithi159 ceph-mon[102522]: pgmap v706: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 21 MiB/s wr, 1.71k op/s 2023-09-12T16:16:45.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:45 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.506 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:45 smithi203 ceph-mon[103431]: pgmap v706: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 21 MiB/s wr, 1.71k op/s 2023-09-12T16:16:45.506 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.507 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.507 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:45.507 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:45 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:46.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:16:46.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:16:46.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:46.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:16:46.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:16:46.481 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:46.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:16:46.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:16:46.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:16:47.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:47 smithi159 ceph-mon[102522]: pgmap v707: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2023-09-12T16:16:47.478 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:47 smithi130 ceph-mon[98670]: pgmap v707: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2023-09-12T16:16:47.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:47 smithi203 ceph-mon[103431]: pgmap v707: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2023-09-12T16:16:48.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:48 smithi130 ceph-mon[98670]: pgmap v708: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 21 MiB/s wr, 1.77k op/s 2023-09-12T16:16:48.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:48 smithi159 ceph-mon[102522]: pgmap v708: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 21 MiB/s wr, 1.77k op/s 2023-09-12T16:16:48.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:48 smithi203 ceph-mon[103431]: pgmap v708: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 21 MiB/s wr, 1.77k op/s 2023-09-12T16:16:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 660.0 s, 1180.8 tps, lat 83.389 ms stddev 213.946 2023-09-12T16:16:50.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:50 smithi159 ceph-mon[102522]: pgmap v709: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-12T16:16:50.983 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:50 smithi130 ceph-mon[98670]: pgmap v709: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-12T16:16:50.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:50 smithi203 ceph-mon[103431]: pgmap v709: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-12T16:16:52.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:52 smithi159 ceph-mon[102522]: pgmap v710: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 24 MiB/s wr, 1.81k op/s 2023-09-12T16:16:52.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:52 smithi130 ceph-mon[98670]: pgmap v710: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 24 MiB/s wr, 1.81k op/s 2023-09-12T16:16:53.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:52 smithi203 ceph-mon[103431]: pgmap v710: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 24 MiB/s wr, 1.81k op/s 2023-09-12T16:16:54.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:54 smithi130 ceph-mon[98670]: pgmap v711: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 25 MiB/s wr, 2.10k op/s 2023-09-12T16:16:54.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:54 smithi159 ceph-mon[102522]: pgmap v711: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 25 MiB/s wr, 2.10k op/s 2023-09-12T16:16:55.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:54 smithi203 ceph-mon[103431]: pgmap v711: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 25 MiB/s wr, 2.10k op/s 2023-09-12T16:16:55.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 665.0 s, 260.0 tps, lat 389.006 ms stddev 389.118 2023-09-12T16:16:56.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:56 smithi130 ceph-mon[98670]: pgmap v712: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 26 MiB/s wr, 2.28k op/s 2023-09-12T16:16:56.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:56 smithi159 ceph-mon[102522]: pgmap v712: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 26 MiB/s wr, 2.28k op/s 2023-09-12T16:16:56.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:56 smithi203 ceph-mon[103431]: pgmap v712: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 26 MiB/s wr, 2.28k op/s 2023-09-12T16:16:58.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:16:58 smithi130 ceph-mon[98670]: pgmap v713: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 32 MiB/s wr, 2.94k op/s 2023-09-12T16:16:58.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:16:58 smithi159 ceph-mon[102522]: pgmap v713: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 32 MiB/s wr, 2.94k op/s 2023-09-12T16:16:58.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:16:58 smithi203 ceph-mon[103431]: pgmap v713: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 32 MiB/s wr, 2.94k op/s 2023-09-12T16:17:00.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 670.0 s, 1712.6 tps, lat 58.894 ms stddev 46.908 2023-09-12T16:17:00.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:00 smithi130 ceph-mon[98670]: pgmap v714: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 32 MiB/s wr, 3.06k op/s 2023-09-12T16:17:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:00 smithi159 ceph-mon[102522]: pgmap v714: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 32 MiB/s wr, 3.06k op/s 2023-09-12T16:17:01.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:00 smithi203 ceph-mon[103431]: pgmap v714: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 32 MiB/s wr, 3.06k op/s 2023-09-12T16:17:02.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:02 smithi159 ceph-mon[102522]: pgmap v715: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 32 MiB/s wr, 3.03k op/s 2023-09-12T16:17:02.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:02 smithi130 ceph-mon[98670]: pgmap v715: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 32 MiB/s wr, 3.03k op/s 2023-09-12T16:17:02.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:02 smithi203 ceph-mon[103431]: pgmap v715: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 32 MiB/s wr, 3.03k op/s 2023-09-12T16:17:04.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:04 smithi159 ceph-mon[102522]: pgmap v716: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 2.51k op/s 2023-09-12T16:17:04.980 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:04 smithi130 ceph-mon[98670]: pgmap v716: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 2.51k op/s 2023-09-12T16:17:04.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:04 smithi203 ceph-mon[103431]: pgmap v716: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 2.51k op/s 2023-09-12T16:17:05.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 675.0 s, 1610.3 tps, lat 61.979 ms stddev 49.215 2023-09-12T16:17:06.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:06 smithi130 ceph-mon[98670]: pgmap v717: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 24 MiB/s wr, 2.04k op/s 2023-09-12T16:17:06.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:06 smithi159 ceph-mon[102522]: pgmap v717: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 24 MiB/s wr, 2.04k op/s 2023-09-12T16:17:06.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:06 smithi203 ceph-mon[103431]: pgmap v717: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 24 MiB/s wr, 2.04k op/s 2023-09-12T16:17:08.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:08 smithi130 ceph-mon[98670]: pgmap v718: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:17:08.735 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:08 smithi159 ceph-mon[102522]: pgmap v718: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:17:08.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:08 smithi203 ceph-mon[103431]: pgmap v718: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 2.10k op/s 2023-09-12T16:17:10.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 680.0 s, 589.6 tps, lat 165.057 ms stddev 311.266 2023-09-12T16:17:10.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:10 smithi130 ceph-mon[98670]: pgmap v719: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 24 MiB/s wr, 1.77k op/s 2023-09-12T16:17:10.984 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:10 smithi159 ceph-mon[102522]: pgmap v719: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 24 MiB/s wr, 1.77k op/s 2023-09-12T16:17:10.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:10 smithi203 ceph-mon[103431]: pgmap v719: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 24 MiB/s wr, 1.77k op/s 2023-09-12T16:17:12.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:12 smithi159 ceph-mon[102522]: pgmap v720: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 22 MiB/s wr, 1.60k op/s 2023-09-12T16:17:12.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:12 smithi130 ceph-mon[98670]: pgmap v720: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 22 MiB/s wr, 1.60k op/s 2023-09-12T16:17:12.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:12 smithi203 ceph-mon[103431]: pgmap v720: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 22 MiB/s wr, 1.60k op/s 2023-09-12T16:17:13.225 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:17:13.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.555+0000 7f895f97c700 1 -- 172.21.15.130:0/1664538641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807a630 msgr2=0x7f895807aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:13.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.555+0000 7f895f97c700 1 --2- 172.21.15.130:0/1664538641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807a630 0x7f895807aab0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f89480098f0 tx=0x7f8948009c00 comp rx=0 tx=0).stop 2023-09-12T16:17:13.563 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.562+0000 7f895f97c700 1 -- 172.21.15.130:0/1664538641 shutdown_connections 2023-09-12T16:17:13.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.562+0000 7f895f97c700 1 --2- 172.21.15.130:0/1664538641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895807aff0 0x7f895807f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.562+0000 7f895f97c700 1 --2- 172.21.15.130:0/1664538641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89580793f0 0x7f8958079810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.562+0000 7f895f97c700 1 --2- 172.21.15.130:0/1664538641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807a630 0x7f895807aab0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.564 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.562+0000 7f895f97c700 1 -- 172.21.15.130:0/1664538641 >> 172.21.15.130:0/1664538641 conn(0x7f8958075bc0 msgr2=0x7f8958078000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:13.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.579+0000 7f895f97c700 1 -- 172.21.15.130:0/1664538641 shutdown_connections 2023-09-12T16:17:13.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.583+0000 7f895f97c700 1 -- 172.21.15.130:0/1664538641 wait complete. 2023-09-12T16:17:13.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.583+0000 7f895f97c700 1 Processor -- start 2023-09-12T16:17:13.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.583+0000 7f895f97c700 1 -- start start 2023-09-12T16:17:13.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.584+0000 7f895f97c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89580793f0 0x7f895819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.584+0000 7f895f97c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895807a630 0x7f895819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.584+0000 7f895f97c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807aff0 0x7f89581a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.584+0000 7f895f97c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89581a44c0 con 0x7f895807aff0 2023-09-12T16:17:13.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.584+0000 7f895f97c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89581a4600 con 0x7f89580793f0 2023-09-12T16:17:13.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.584+0000 7f895f97c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89581a4790 con 0x7f895807a630 2023-09-12T16:17:13.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.584+0000 7f895df19700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807aff0 0x7f89581a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.584+0000 7f895df19700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807aff0 0x7f89581a3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56672/0 (socket says 172.21.15.130:56672) 2023-09-12T16:17:13.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.584+0000 7f895df19700 1 -- 172.21.15.130:0/3875795480 learned_addr learned my addr 172.21.15.130:0/3875795480 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:13.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.585+0000 7f895df19700 1 -- 172.21.15.130:0/3875795480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895807a630 msgr2=0x7f895819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:13.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.585+0000 7f895df19700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895807a630 0x7f895819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.585+0000 7f895df19700 1 -- 172.21.15.130:0/3875795480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89580793f0 msgr2=0x7f895819c940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-12T16:17:13.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.585+0000 7f895df19700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89580793f0 0x7f895819c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.585+0000 7f895df19700 1 -- 172.21.15.130:0/3875795480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8948009520 con 0x7f895807aff0 2023-09-12T16:17:13.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.590+0000 7f895df19700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807aff0 0x7f89581a3ef0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f895000c110 tx=0x7f895000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:13.600 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.600+0000 7f894e7fc700 1 -- 172.21.15.130:0/3875795480 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950017070 con 0x7f895807aff0 2023-09-12T16:17:13.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.600+0000 7f894e7fc700 1 -- 172.21.15.130:0/3875795480 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f895000e050 con 0x7f895807aff0 2023-09-12T16:17:13.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.600+0000 7f894e7fc700 1 -- 172.21.15.130:0/3875795480 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950007ac0 con 0x7f895807aff0 2023-09-12T16:17:13.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.600+0000 7f895f97c700 1 -- 172.21.15.130:0/3875795480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89581a4a10 con 0x7f895807aff0 2023-09-12T16:17:13.601 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.600+0000 7f895f97c700 1 -- 172.21.15.130:0/3875795480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89581a4ed0 con 0x7f895807aff0 2023-09-12T16:17:13.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.606+0000 7f896bb42b80 1 Processor -- start 2023-09-12T16:17:13.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.606+0000 7f894e7fc700 1 -- 172.21.15.130:0/3875795480 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f8950007c20 con 0x7f895807aff0 2023-09-12T16:17:13.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.607+0000 7f894e7fc700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89440b17d0 0x7f89440b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.607+0000 7f896bb42b80 1 -- start start 2023-09-12T16:17:13.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abdf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdf5a0 0x55875abdd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 0x55875abde2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.609 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f896bb42b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875ab36730 con 0x55875abdde00 2023-09-12T16:17:13.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f896bb42b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875ab34050 con 0x55875abdec00 2023-09-12T16:17:13.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f896bb42b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875ab2f940 con 0x55875abdf5a0 2023-09-12T16:17:13.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f895cf17700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdf5a0 0x55875abdd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f895d718700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abdf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.610 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f895cf17700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdf5a0 0x55875abdd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36292/0 (socket says 172.21.15.130:36292) 2023-09-12T16:17:13.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f895d718700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abdf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39936/0 (socket says 172.21.15.130:39936) 2023-09-12T16:17:13.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f895cf17700 1 -- 172.21.15.130:0/2316398985 learned_addr learned my addr 172.21.15.130:0/2316398985 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:13.611 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f895d718700 1 -- 172.21.15.130:0/2316398985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdf5a0 msgr2=0x55875abdd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:13.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f895d718700 1 --2- 172.21.15.130:0/2316398985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdf5a0 0x55875abdd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f895d718700 1 -- 172.21.15.130:0/2316398985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 msgr2=0x55875abde2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:13.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.608+0000 7f895d718700 1 --2- 172.21.15.130:0/2316398985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 0x55875abde2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.609+0000 7f895d718700 1 -- 172.21.15.130:0/2316398985 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55875a95bde0 con 0x55875abdec00 2023-09-12T16:17:13.612 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.609+0000 7f895d718700 1 --2- 172.21.15.130:0/2316398985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abdf020 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8954009b60 tx=0x7f8954009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7141bf5ce1efa44d server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:13.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.610+0000 7f89437fe700 1 -- 172.21.15.130:0/2316398985 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954004230 con 0x55875abdec00 2023-09-12T16:17:13.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.611+0000 7f89437fe700 1 -- 172.21.15.130:0/2316398985 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f895400edc0 con 0x55875abdec00 2023-09-12T16:17:13.613 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.611+0000 7f89437fe700 1 -- 172.21.15.130:0/2316398985 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954005280 con 0x55875abdec00 2023-09-12T16:17:13.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.614+0000 7f895cf17700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89440b17d0 0x7f89440b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.615+0000 7f896bb42b80 1 -- 172.21.15.130:0/2316398985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 msgr2=0x55875abdf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:13.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.615+0000 7f896bb42b80 1 --2- 172.21.15.130:0/2316398985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abdf020 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8954009b60 tx=0x7f8954009e70 comp rx=0 tx=0).stop 2023-09-12T16:17:13.616 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.615+0000 7f896bb42b80 1 -- 172.21.15.130:0/2316398985 shutdown_connections 2023-09-12T16:17:13.617 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.615+0000 7f896bb42b80 1 --2- 172.21.15.130:0/2316398985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 0x55875abde2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.615+0000 7f896bb42b80 1 --2- 172.21.15.130:0/2316398985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abdf020 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.615+0000 7f896bb42b80 1 --2- 172.21.15.130:0/2316398985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdf5a0 0x55875abdd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.615+0000 7f896bb42b80 1 -- 172.21.15.130:0/2316398985 >> 172.21.15.130:0/2316398985 conn(0x55875ab52090 msgr2=0x55875abd8720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:13.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.616+0000 7f894e7fc700 1 -- 172.21.15.130:0/3875795480 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f89500112c0 con 0x7f895807aff0 2023-09-12T16:17:13.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.617+0000 7f896bb42b80 1 -- 172.21.15.130:0/2316398985 shutdown_connections 2023-09-12T16:17:13.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.617+0000 7f896bb42b80 1 -- 172.21.15.130:0/2316398985 wait complete. 2023-09-12T16:17:13.621 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.617+0000 7f896bb42b80 1 -- start start 2023-09-12T16:17:13.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.617+0000 7f896bb42b80 1 Processor -- start 2023-09-12T16:17:13.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdde00 0x55875abe7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdf5a0 0x55875abeebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.622 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f896bb42b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875aacdb70 con 0x55875abdf5a0 2023-09-12T16:17:13.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f896bb42b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875aacd870 con 0x55875abdde00 2023-09-12T16:17:13.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f896bb42b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875a9b1920 con 0x55875abdec00 2023-09-12T16:17:13.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f896bb42b80 10 client.0 _ll_register_callbacks cb 0x55875aa17a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:17:13.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f895d718700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdde00 0x55875abe7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f895cf17700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.623 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f895d718700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdde00 0x55875abe7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39944/0 (socket says 172.21.15.130:39944) 2023-09-12T16:17:13.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f895cf17700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36296/0 (socket says 172.21.15.130:36296) 2023-09-12T16:17:13.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.618+0000 7f895d718700 1 -- 172.21.15.130:0/748033757 learned_addr learned my addr 172.21.15.130:0/748033757 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:13.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f895d718700 1 -- 172.21.15.130:0/748033757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 msgr2=0x55875abe7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:13.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f895d718700 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.624 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f895d718700 1 -- 172.21.15.130:0/748033757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdf5a0 msgr2=0x55875abeebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:13.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f895d718700 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdf5a0 0x55875abeebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f895d718700 1 -- 172.21.15.130:0/748033757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8954009810 con 0x55875abdde00 2023-09-12T16:17:13.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f895cf17700 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:17:13.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f895d718700 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdde00 0x55875abe7640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f895400a010 tx=0x7f8954004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146914d7ac0aa37 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:13.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f8940ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:17:13.625 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f8940ff9700 1 -- 172.21.15.130:0/748033757 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954004450 con 0x55875abdde00 2023-09-12T16:17:13.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f8940ff9700 1 -- 172.21.15.130:0/748033757 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8954014070 con 0x55875abdde00 2023-09-12T16:17:13.626 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.619+0000 7f896bb42b80 1 -- 172.21.15.130:0/748033757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55875aa874f0 con 0x55875abdde00 2023-09-12T16:17:13.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f895cf17700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89440b17d0 0x7f89440b3c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f894800fa70 tx=0x7f89480094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:13.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f8940ff9700 1 -- 172.21.15.130:0/748033757 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89540045b0 con 0x55875abdde00 2023-09-12T16:17:13.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f896bb42b80 10 client.25652 fetch_fsmap learned FSMap version 32 2023-09-12T16:17:13.627 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f896bb42b80 1 -- 172.21.15.130:0/748033757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55875a944c60 con 0x55875abdde00 2023-09-12T16:17:13.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f8940ff9700 1 -- 172.21.15.130:0/748033757 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55875aa874f0 con 0x55875abdde00 2023-09-12T16:17:13.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f8940ff9700 1 -- 172.21.15.130:0/748033757 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8954005180 con 0x55875abdde00 2023-09-12T16:17:13.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f896bb42b80 10 client.25652 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:17:13.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f896bb42b80 10 client.25652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:17:13.628 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f896bb42b80 20 client.25652 populate_metadata read hostname 'smithi130' 2023-09-12T16:17:13.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f896bb42b80 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55875abf3230 0x55875abf5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f896bb42b80 4 client.25652 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:17:13.629 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.620+0000 7f896bb42b80 1 -- 172.21.15.130:0/748033757 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55875ab448a0 con 0x55875abf3230 2023-09-12T16:17:13.630 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.621+0000 7f895cf17700 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55875abf3230 0x55875abf5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.621+0000 7f895cf17700 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55875abf3230 0x55875abf5650 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d474bb49259f16b server_cookie=54312e7d810305bc in_seq=0 out_seq=0 2023-09-12T16:17:13.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.621+0000 7f8940ff9700 10 client.25652 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:17:13.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f8940ff9700 1 -- 172.21.15.130:0/748033757 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55875ab448a0 con 0x55875abf3230 2023-09-12T16:17:13.631 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f8940ff9700 10 client.25652 handle_command_reply: tid=0 2023-09-12T16:17:13.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 client.25652 shutdown 2023-09-12T16:17:13.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 -- 172.21.15.130:0/748033757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdde00 msgr2=0x55875abe7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:13.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdde00 0x55875abe7640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f895400a010 tx=0x7f8954004980 comp rx=0 tx=0).stop 2023-09-12T16:17:13.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 -- 172.21.15.130:0/748033757 shutdown_connections 2023-09-12T16:17:13.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55875abf3230 0x55875abf5650 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.632 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdf5a0 0x55875abeebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdde00 0x55875abe7640 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 --2- 172.21.15.130:0/748033757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 -- 172.21.15.130:0/748033757 >> 172.21.15.130:0/748033757 conn(0x55875ab52090 msgr2=0x55875abd7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:13.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f8940ff9700 0 client.25652 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:17:13.633 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 -- 172.21.15.130:0/748033757 shutdown_connections 2023-09-12T16:17:13.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.623+0000 7f896bb42b80 1 -- 172.21.15.130:0/748033757 wait complete. 2023-09-12T16:17:13.634 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.624+0000 7f896bb42b80 20 client.25652 trim_cache size 0 max 16384 2023-09-12T16:17:13.654 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.652+0000 7f896bb42b80 1 Processor -- start 2023-09-12T16:17:13.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f896bb42b80 1 -- start start 2023-09-12T16:17:13.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 0x55875abe1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.655 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdf5a0 0x55875abeeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f896bb42b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875aba2850 con 0x55875abdde00 2023-09-12T16:17:13.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f896bb42b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875abd99b0 con 0x55875abdf5a0 2023-09-12T16:17:13.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f896bb42b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875abefb30 con 0x55875abdec00 2023-09-12T16:17:13.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895d718700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 0x55875abe1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.656 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895cf17700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895d718700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 0x55875abe1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56692/0 (socket says 172.21.15.130:56692) 2023-09-12T16:17:13.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895d718700 1 -- 172.21.15.130:0/462011502 learned_addr learned my addr 172.21.15.130:0/462011502 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:13.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895df19700 1 --2- 172.21.15.130:0/462011502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdf5a0 0x55875abeeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895d718700 1 -- 172.21.15.130:0/462011502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 msgr2=0x55875abe2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:13.657 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895d718700 1 --2- 172.21.15.130:0/462011502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895d718700 1 -- 172.21.15.130:0/462011502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdf5a0 msgr2=0x55875abeeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:13.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895d718700 1 --2- 172.21.15.130:0/462011502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdf5a0 0x55875abeeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895d718700 1 -- 172.21.15.130:0/462011502 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55875a982250 con 0x55875abdde00 2023-09-12T16:17:13.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.653+0000 7f895cf17700 1 --2- 172.21.15.130:0/462011502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:17:13.658 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.654+0000 7f895d718700 1 --2- 172.21.15.130:0/462011502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 0x55875abe1f50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8954000c00 tx=0x7f895400e630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83511659f3e2d642 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:13.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.654+0000 7f8943fff700 1 -- 172.21.15.130:0/462011502 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954004250 con 0x55875abdde00 2023-09-12T16:17:13.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.654+0000 7f8943fff700 1 -- 172.21.15.130:0/462011502 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f895401d070 con 0x55875abdde00 2023-09-12T16:17:13.659 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.654+0000 7f8943fff700 1 -- 172.21.15.130:0/462011502 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954003680 con 0x55875abdde00 2023-09-12T16:17:13.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.655+0000 7f896bb42b80 1 -- 172.21.15.130:0/462011502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 msgr2=0x55875abe1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:13.661 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.655+0000 7f896bb42b80 1 --2- 172.21.15.130:0/462011502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 0x55875abe1f50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8954000c00 tx=0x7f895400e630 comp rx=0 tx=0).stop 2023-09-12T16:17:13.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.655+0000 7f896bb42b80 1 -- 172.21.15.130:0/462011502 shutdown_connections 2023-09-12T16:17:13.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.655+0000 7f896bb42b80 1 --2- 172.21.15.130:0/462011502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdf5a0 0x55875abeeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.655+0000 7f896bb42b80 1 --2- 172.21.15.130:0/462011502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdde00 0x55875abe1f50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.662 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.655+0000 7f896bb42b80 1 --2- 172.21.15.130:0/462011502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdec00 0x55875abe2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.655+0000 7f896bb42b80 1 -- 172.21.15.130:0/462011502 >> 172.21.15.130:0/462011502 conn(0x55875ab52090 msgr2=0x55875abe5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:13.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.655+0000 7f896bb42b80 1 -- 172.21.15.130:0/462011502 shutdown_connections 2023-09-12T16:17:13.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.655+0000 7f896bb42b80 1 -- 172.21.15.130:0/462011502 wait complete. 2023-09-12T16:17:13.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.656+0000 7f896bb42b80 1 -- start start 2023-09-12T16:17:13.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.656+0000 7f896bb42b80 1 Processor -- start 2023-09-12T16:17:13.663 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.656+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdde00 0x55875abecd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.656+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.656+0000 7f896bb42b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdf5a0 0x55875abee150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.656+0000 7f896bb42b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875abee720 con 0x55875abdf5a0 2023-09-12T16:17:13.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.656+0000 7f896bb42b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875abee890 con 0x55875abdec00 2023-09-12T16:17:13.664 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.656+0000 7f896bb42b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875abeea50 con 0x55875abdde00 2023-09-12T16:17:13.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.656+0000 7f896bb42b80 10 client.0 _ll_register_callbacks cb 0x7f8954009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:17:13.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.657+0000 7f895cf17700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.657+0000 7f895cf17700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abed2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39956/0 (socket says 172.21.15.130:39956) 2023-09-12T16:17:13.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.657+0000 7f895cf17700 1 -- 172.21.15.130:0/3349441571 learned_addr learned my addr 172.21.15.130:0/3349441571 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:13.665 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.657+0000 7f895df19700 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdf5a0 0x55875abee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.657+0000 7f895cf17700 1 -- 172.21.15.130:0/3349441571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdde00 msgr2=0x55875abecd70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:17:13.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.657+0000 7f895cf17700 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdde00 0x55875abecd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.657+0000 7f895cf17700 1 -- 172.21.15.130:0/3349441571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdf5a0 msgr2=0x55875abee150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:13.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.657+0000 7f895cf17700 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdf5a0 0x55875abee150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.666 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.657+0000 7f895cf17700 1 -- 172.21.15.130:0/3349441571 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8948020040 con 0x55875abdec00 2023-09-12T16:17:13.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.658+0000 7f895cf17700 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abed2b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8948009230 tx=0x7f8948009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bf00fa7ab6c182 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:13.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.658+0000 7f8940ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:17:13.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.658+0000 7f8940ff9700 1 -- 172.21.15.130:0/3349441571 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948005600 con 0x55875abdec00 2023-09-12T16:17:13.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.658+0000 7f8940ff9700 1 -- 172.21.15.130:0/3349441571 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f894802f130 con 0x55875abdec00 2023-09-12T16:17:13.667 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.658+0000 7f896bb42b80 1 -- 172.21.15.130:0/3349441571 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f895400edc0 con 0x55875abdec00 2023-09-12T16:17:13.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.658+0000 7f8940ff9700 1 -- 172.21.15.130:0/3349441571 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894802f2f0 con 0x55875abdec00 2023-09-12T16:17:13.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.659+0000 7f896bb42b80 10 client.25658 fetch_fsmap learned FSMap version 32 2023-09-12T16:17:13.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.659+0000 7f896bb42b80 1 -- 172.21.15.130:0/3349441571 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55875a944c60 con 0x55875abdec00 2023-09-12T16:17:13.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.659+0000 7f8940ff9700 1 -- 172.21.15.130:0/3349441571 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f895400edc0 con 0x55875abdec00 2023-09-12T16:17:13.668 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.659+0000 7f8940ff9700 1 -- 172.21.15.130:0/3349441571 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f89480037b0 con 0x55875abdec00 2023-09-12T16:17:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.660+0000 7f896bb42b80 10 client.25658 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:17:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.660+0000 7f896bb42b80 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:17:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.660+0000 7f896bb42b80 20 client.25658 populate_metadata read hostname 'smithi130' 2023-09-12T16:17:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.660+0000 7f896bb42b80 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55875ac48370 0x55875ac4a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:13.669 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.660+0000 7f896bb42b80 4 client.25658 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:17:13.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.660+0000 7f896bb42b80 1 -- 172.21.15.130:0/3349441571 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55875ab28920 con 0x55875ac48370 2023-09-12T16:17:13.670 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.661+0000 7f895d718700 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55875ac48370 0x55875ac4a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:13.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.661+0000 7f8940ff9700 10 client.25658 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:17:13.671 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.661+0000 7f895d718700 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55875ac48370 0x55875ac4a790 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad287443c15f420 server_cookie=35b046f33943d742 in_seq=0 out_seq=0 2023-09-12T16:17:13.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f8940ff9700 1 -- 172.21.15.130:0/3349441571 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55875ab28920 con 0x55875ac48370 2023-09-12T16:17:13.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f8940ff9700 10 client.25658 handle_command_reply: tid=0 2023-09-12T16:17:13.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 client.25658 shutdown 2023-09-12T16:17:13.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 -- 172.21.15.130:0/3349441571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 msgr2=0x55875abed2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:13.672 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abed2b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8948009230 tx=0x7f8948009260 comp rx=0 tx=0).stop 2023-09-12T16:17:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 -- 172.21.15.130:0/3349441571 shutdown_connections 2023-09-12T16:17:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55875ac48370 0x55875ac4a790 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55875abdf5a0 0x55875abee150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55875abdde00 0x55875abecd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 --2- 172.21.15.130:0/3349441571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55875abdec00 0x55875abed2b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 -- 172.21.15.130:0/3349441571 >> 172.21.15.130:0/3349441571 conn(0x55875ab52090 msgr2=0x55875abe5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f8940ff9700 0 client.25658 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:17:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.662+0000 7f896bb42b80 1 -- 172.21.15.130:0/3349441571 shutdown_connections 2023-09-12T16:17:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.663+0000 7f896bb42b80 1 -- 172.21.15.130:0/3349441571 wait complete. 2023-09-12T16:17:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.663+0000 7f896bb42b80 20 client.25658 trim_cache size 0 max 16384 2023-09-12T16:17:13.675 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:17:13.675 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:17:13.675 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:17:13.675 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:17:13.675 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:17:13.676 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:17:13.676 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:17:13.676 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:17:13.676 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:17:13.676 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:17:13.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 -- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89440b17d0 msgr2=0x7f89440b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:13.677 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89440b17d0 0x7f89440b3c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f894800fa70 tx=0x7f89480094b0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 -- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807aff0 msgr2=0x7f89581a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:13.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807aff0 0x7f89581a3ef0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f895000c110 tx=0x7f895000c4d0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 -- 172.21.15.130:0/3875795480 shutdown_connections 2023-09-12T16:17:13.678 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f89440b17d0 0x7f89440b3c90 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f894800fa70 tx=0x7f89480094b0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f895807aff0 0x7f89581a3ef0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89580793f0 0x7f895819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 --2- 172.21.15.130:0/3875795480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f895807a630 0x7f895819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:13.679 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.664+0000 7f895f97c700 1 -- 172.21.15.130:0/3875795480 >> 172.21.15.130:0/3875795480 conn(0x7f8958075bc0 msgr2=0x7f8958077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:13.686 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.681+0000 7f895f97c700 1 -- 172.21.15.130:0/3875795480 shutdown_connections 2023-09-12T16:17:13.694 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:13.690+0000 7f895f97c700 1 -- 172.21.15.130:0/3875795480 wait complete. 2023-09-12T16:17:13.731 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:17:14.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:14 smithi130 ceph-mon[98670]: pgmap v721: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 22 MiB/s wr, 1.80k op/s 2023-09-12T16:17:15.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 685.0 s, 773.6 tps, lat 132.682 ms stddev 139.261 2023-09-12T16:17:15.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:14 smithi159 ceph-mon[102522]: pgmap v721: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 22 MiB/s wr, 1.80k op/s 2023-09-12T16:17:15.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:14 smithi203 ceph-mon[103431]: pgmap v721: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 22 MiB/s wr, 1.80k op/s 2023-09-12T16:17:16.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:16 smithi159 ceph-mon[102522]: pgmap v722: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 21 MiB/s wr, 1.83k op/s 2023-09-12T16:17:17.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:16 smithi130 ceph-mon[98670]: pgmap v722: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 21 MiB/s wr, 1.83k op/s 2023-09-12T16:17:17.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:16 smithi203 ceph-mon[103431]: pgmap v722: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 21 MiB/s wr, 1.83k op/s 2023-09-12T16:17:19.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:19 smithi159 ceph-mon[102522]: pgmap v723: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 24 MiB/s wr, 1.95k op/s 2023-09-12T16:17:19.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:19 smithi130 ceph-mon[98670]: pgmap v723: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 24 MiB/s wr, 1.95k op/s 2023-09-12T16:17:19.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:19 smithi203 ceph-mon[103431]: pgmap v723: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 24 MiB/s wr, 1.95k op/s 2023-09-12T16:17:20.092 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 690.0 s, 1528.0 tps, lat 65.631 ms stddev 51.847 2023-09-12T16:17:21.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:21 smithi159 ceph-mon[102522]: pgmap v724: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 1.59k op/s 2023-09-12T16:17:21.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:21 smithi130 ceph-mon[98670]: pgmap v724: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 1.59k op/s 2023-09-12T16:17:21.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:21 smithi203 ceph-mon[103431]: pgmap v724: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 1.59k op/s 2023-09-12T16:17:23.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:23 smithi130 ceph-mon[98670]: pgmap v725: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-09-12T16:17:23.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:23 smithi159 ceph-mon[102522]: pgmap v725: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-09-12T16:17:23.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:23 smithi203 ceph-mon[103431]: pgmap v725: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-09-12T16:17:24.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:24 smithi130 ceph-mon[98670]: pgmap v726: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 19 MiB/s wr, 1.33k op/s 2023-09-12T16:17:24.734 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:24 smithi159 ceph-mon[102522]: pgmap v726: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 19 MiB/s wr, 1.33k op/s 2023-09-12T16:17:24.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:24 smithi203 ceph-mon[103431]: pgmap v726: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 19 MiB/s wr, 1.33k op/s 2023-09-12T16:17:25.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 695.0 s, 1098.2 tps, lat 88.890 ms stddev 188.581 2023-09-12T16:17:26.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:26 smithi130 ceph-mon[98670]: pgmap v727: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-09-12T16:17:26.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:26 smithi159 ceph-mon[102522]: pgmap v727: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-09-12T16:17:26.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:26 smithi203 ceph-mon[103431]: pgmap v727: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-09-12T16:17:29.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:29 smithi159 ceph-mon[102522]: pgmap v728: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 25 MiB/s wr, 1.89k op/s 2023-09-12T16:17:29.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:29 smithi130 ceph-mon[98670]: pgmap v728: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 25 MiB/s wr, 1.89k op/s 2023-09-12T16:17:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:29 smithi203 ceph-mon[103431]: pgmap v728: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 25 MiB/s wr, 1.89k op/s 2023-09-12T16:17:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 700.0 s, 225.6 tps, lat 455.492 ms stddev 610.492 2023-09-12T16:17:30.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:30 smithi159 ceph-mon[102522]: pgmap v729: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 25 MiB/s wr, 2.20k op/s 2023-09-12T16:17:30.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:30 smithi130 ceph-mon[98670]: pgmap v729: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 25 MiB/s wr, 2.20k op/s 2023-09-12T16:17:30.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:30 smithi203 ceph-mon[103431]: pgmap v729: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 25 MiB/s wr, 2.20k op/s 2023-09-12T16:17:33.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:33 smithi130 ceph-mon[98670]: pgmap v730: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 26 MiB/s wr, 2.44k op/s 2023-09-12T16:17:33.728 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:33 smithi159 ceph-mon[102522]: pgmap v730: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 26 MiB/s wr, 2.44k op/s 2023-09-12T16:17:33.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:33 smithi203 ceph-mon[103431]: pgmap v730: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 26 MiB/s wr, 2.44k op/s 2023-09-12T16:17:35.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 705.0 s, 1918.0 tps, lat 52.109 ms stddev 39.319 2023-09-12T16:17:35.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:35 smithi130 ceph-mon[98670]: pgmap v731: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 30 MiB/s wr, 2.99k op/s 2023-09-12T16:17:35.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:35 smithi159 ceph-mon[102522]: pgmap v731: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 30 MiB/s wr, 2.99k op/s 2023-09-12T16:17:35.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:35 smithi203 ceph-mon[103431]: pgmap v731: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 30 MiB/s wr, 2.99k op/s 2023-09-12T16:17:37.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:37 smithi159 ceph-mon[102522]: pgmap v732: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 29 MiB/s wr, 2.87k op/s 2023-09-12T16:17:37.727 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:37 smithi130 ceph-mon[98670]: pgmap v732: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 29 MiB/s wr, 2.87k op/s 2023-09-12T16:17:37.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:37 smithi203 ceph-mon[103431]: pgmap v732: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 29 MiB/s wr, 2.87k op/s 2023-09-12T16:17:39.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:39 smithi130 ceph-mon[98670]: pgmap v733: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 33 MiB/s wr, 3.09k op/s 2023-09-12T16:17:39.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:39 smithi159 ceph-mon[102522]: pgmap v733: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 33 MiB/s wr, 3.09k op/s 2023-09-12T16:17:39.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:39 smithi203 ceph-mon[103431]: pgmap v733: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 33 MiB/s wr, 3.09k op/s 2023-09-12T16:17:40.094 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 710.0 s, 812.1 tps, lat 122.274 ms stddev 296.000 2023-09-12T16:17:40.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:40 smithi130 ceph-mon[98670]: pgmap v734: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 26 MiB/s wr, 2.45k op/s 2023-09-12T16:17:40.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:40 smithi159 ceph-mon[102522]: pgmap v734: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 26 MiB/s wr, 2.45k op/s 2023-09-12T16:17:40.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:40 smithi203 ceph-mon[103431]: pgmap v734: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 26 MiB/s wr, 2.45k op/s 2023-09-12T16:17:43.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:43 smithi130 ceph-mon[98670]: pgmap v735: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 24 MiB/s wr, 2.09k op/s 2023-09-12T16:17:43.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:43 smithi159 ceph-mon[102522]: pgmap v735: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 24 MiB/s wr, 2.09k op/s 2023-09-12T16:17:43.732 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:17:43.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:43 smithi203 ceph-mon[103431]: pgmap v735: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 24 MiB/s wr, 2.09k op/s 2023-09-12T16:17:43.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.952+0000 7f6b73368700 1 -- 172.21.15.130:0/1537800560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b6c1006f0 msgr2=0x7f6b6c100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:43.953 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.952+0000 7f6b73368700 1 --2- 172.21.15.130:0/1537800560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b6c1006f0 0x7f6b6c100b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0099c0 tx=0x7f6b5c009cd0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.953+0000 7f6b73368700 1 -- 172.21.15.130:0/1537800560 shutdown_connections 2023-09-12T16:17:43.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.953+0000 7f6b73368700 1 --2- 172.21.15.130:0/1537800560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b6c101090 0x7f6b6c107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.953+0000 7f6b73368700 1 --2- 172.21.15.130:0/1537800560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c0ff500 0x7f6b6c0ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.953+0000 7f6b73368700 1 --2- 172.21.15.130:0/1537800560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b6c1006f0 0x7f6b6c100b50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.954 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.953+0000 7f6b73368700 1 -- 172.21.15.130:0/1537800560 >> 172.21.15.130:0/1537800560 conn(0x7f6b6c0faa90 msgr2=0x7f6b6c0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:43.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.953+0000 7f6b73368700 1 -- 172.21.15.130:0/1537800560 shutdown_connections 2023-09-12T16:17:43.955 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.953+0000 7f6b73368700 1 -- 172.21.15.130:0/1537800560 wait complete. 2023-09-12T16:17:43.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.954+0000 7f6b73368700 1 Processor -- start 2023-09-12T16:17:43.956 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.954+0000 7f6b73368700 1 -- start start 2023-09-12T16:17:43.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.954+0000 7f6b73368700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b6c0ff500 0x7f6b6c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.954+0000 7f6b73368700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b6c1006f0 0x7f6b6c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.954+0000 7f6b73368700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c101090 0x7f6b6c1a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.954+0000 7f6b73368700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c1a44d0 con 0x7f6b6c0ff500 2023-09-12T16:17:43.957 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.954+0000 7f6b73368700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c1a4610 con 0x7f6b6c1006f0 2023-09-12T16:17:43.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.954+0000 7f6b73368700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c1a47a0 con 0x7f6b6c101090 2023-09-12T16:17:43.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71905700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c101090 0x7f6b6c1a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71104700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b6c0ff500 0x7f6b6c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b70903700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b6c1006f0 0x7f6b6c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.958 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71104700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b6c0ff500 0x7f6b6c19c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56710/0 (socket says 172.21.15.130:56710) 2023-09-12T16:17:43.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71104700 1 -- 172.21.15.130:0/2312561998 learned_addr learned my addr 172.21.15.130:0/2312561998 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:43.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71104700 1 -- 172.21.15.130:0/2312561998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c101090 msgr2=0x7f6b6c1a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:43.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71104700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c101090 0x7f6b6c1a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71104700 1 -- 172.21.15.130:0/2312561998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b6c1006f0 msgr2=0x7f6b6c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:43.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71104700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b6c1006f0 0x7f6b6c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71104700 1 -- 172.21.15.130:0/2312561998 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0095f0 con 0x7f6b6c0ff500 2023-09-12T16:17:43.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.955+0000 7f6b71104700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b6c0ff500 0x7f6b6c19c950 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800e970 tx=0x7f6b6800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:43.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.956+0000 7f6b627fc700 1 -- 172.21.15.130:0/2312561998 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68009790 con 0x7f6b6c0ff500 2023-09-12T16:17:43.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.956+0000 7f6b73368700 1 -- 172.21.15.130:0/2312561998 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c1a4a80 con 0x7f6b6c0ff500 2023-09-12T16:17:43.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.956+0000 7f6b73368700 1 -- 172.21.15.130:0/2312561998 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c1a4fe0 con 0x7f6b6c0ff500 2023-09-12T16:17:43.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.957+0000 7f6b627fc700 1 -- 172.21.15.130:0/2312561998 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b6800c7e0 con 0x7f6b6c0ff500 2023-09-12T16:17:43.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.957+0000 7f6b627fc700 1 -- 172.21.15.130:0/2312561998 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68003c70 con 0x7f6b6c0ff500 2023-09-12T16:17:43.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b7f52eb80 1 Processor -- start 2023-09-12T16:17:43.961 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b7f52eb80 1 -- start start 2023-09-12T16:17:43.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df94020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112df92800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112deeb730 con 0x56112df92e00 2023-09-12T16:17:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112dee9050 con 0x56112df945a0 2023-09-12T16:17:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112dee4940 con 0x56112df93c00 2023-09-12T16:17:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b71905700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df94020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b71905700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df94020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36328/0 (socket says 172.21.15.130:36328) 2023-09-12T16:17:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b71905700 1 -- 172.21.15.130:0/3698588041 learned_addr learned my addr 172.21.15.130:0/3698588041 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b70903700 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.962+0000 7f6b71104700 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112df92800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b71905700 1 -- 172.21.15.130:0/3698588041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 msgr2=0x56112df92800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b71905700 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112df92800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b71905700 1 -- 172.21.15.130:0/3698588041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 msgr2=0x56112df932b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b71905700 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df932b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b71905700 1 -- 172.21.15.130:0/3698588041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56112dd10de0 con 0x56112df93c00 2023-09-12T16:17:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b71104700 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112df92800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T16:17:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b71905700 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df94020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400b570 tx=0x7f6b6400bb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7323b9f890a934c server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:43.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b577fe700 1 -- 172.21.15.130:0/3698588041 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64010040 con 0x56112df93c00 2023-09-12T16:17:43.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b577fe700 1 -- 172.21.15.130:0/3698588041 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b6400b5a0 con 0x56112df93c00 2023-09-12T16:17:43.967 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.963+0000 7f6b577fe700 1 -- 172.21.15.130:0/3698588041 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64005280 con 0x56112df93c00 2023-09-12T16:17:43.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.964+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/3698588041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 msgr2=0x56112df94020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:43.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.964+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df94020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400b570 tx=0x7f6b6400bb30 comp rx=0 tx=0).stop 2023-09-12T16:17:43.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.964+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/3698588041 shutdown_connections 2023-09-12T16:17:43.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.964+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.964+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df94020 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.964+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/3698588041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112df92800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.964+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/3698588041 >> 172.21.15.130:0/3698588041 conn(0x56112df07090 msgr2=0x56112df8d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:43.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.964+0000 7f6b627fc700 1 -- 172.21.15.130:0/2312561998 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f6b6800c950 con 0x7f6b6c0ff500 2023-09-12T16:17:43.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.965+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/3698588041 shutdown_connections 2023-09-12T16:17:43.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.965+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/3698588041 wait complete. 2023-09-12T16:17:43.973 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.965+0000 7f6b7f52eb80 1 -- start start 2023-09-12T16:17:43.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.965+0000 7f6b7f52eb80 1 Processor -- start 2023-09-12T16:17:43.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.965+0000 7f6b627fc700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6b580a05d0 0x7f6b580a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df9c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df9ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112dfa3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.974 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112de82b70 con 0x56112df92e00 2023-09-12T16:17:43.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112de82870 con 0x56112df945a0 2023-09-12T16:17:43.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112dd66920 con 0x56112df93c00 2023-09-12T16:17:43.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b7f52eb80 10 client.0 _ll_register_callbacks cb 0x56112ddcca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:17:43.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b627fc700 1 -- 172.21.15.130:0/2312561998 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6b68014070 con 0x7f6b6c0ff500 2023-09-12T16:17:43.975 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b71905700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6b580a05d0 0x7f6b580a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b71104700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df9c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b71104700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df9c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56724/0 (socket says 172.21.15.130:56724) 2023-09-12T16:17:43.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b71104700 1 -- 172.21.15.130:0/685558177 learned_addr learned my addr 172.21.15.130:0/685558177 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:43.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b70903700 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df9ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b71905700 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112dfa3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.976 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b70903700 1 -- 172.21.15.130:0/685558177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 msgr2=0x56112dfa3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:43.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b70903700 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112dfa3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b70903700 1 -- 172.21.15.130:0/685558177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 msgr2=0x56112df9c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:43.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b70903700 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df9c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b70903700 1 -- 172.21.15.130:0/685558177 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6400b220 con 0x56112df93c00 2023-09-12T16:17:43.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b71104700 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df9c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:17:43.977 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.966+0000 7f6b71905700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6b580a05d0 0x7f6b580a2a90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6b64018000 tx=0x7f6b64016640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:43.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.967+0000 7f6b70903700 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df9ca80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0098e0 tx=0x7f6b5c005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac350994eb44663 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:43.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.967+0000 7f6b54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:17:43.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.967+0000 7f6b54ff9700 1 -- 172.21.15.130:0/685558177 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c003c60 con 0x56112df93c00 2023-09-12T16:17:43.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.967+0000 7f6b54ff9700 1 -- 172.21.15.130:0/685558177 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6b5c003dc0 con 0x56112df93c00 2023-09-12T16:17:43.978 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.967+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/685558177 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56112de3c4f0 con 0x56112df93c00 2023-09-12T16:17:43.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.967+0000 7f6b54ff9700 1 -- 172.21.15.130:0/685558177 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b5c005040 con 0x56112df93c00 2023-09-12T16:17:43.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.967+0000 7f6b7f52eb80 10 client.25876 fetch_fsmap learned FSMap version 32 2023-09-12T16:17:43.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.967+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/685558177 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56112dcf9c60 con 0x56112df93c00 2023-09-12T16:17:43.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.967+0000 7f6b54ff9700 1 -- 172.21.15.130:0/685558177 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56112de3c4f0 con 0x56112df93c00 2023-09-12T16:17:43.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.968+0000 7f6b54ff9700 1 -- 172.21.15.130:0/685558177 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b5c02c070 con 0x56112df93c00 2023-09-12T16:17:43.979 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.968+0000 7f6b7f52eb80 10 client.25876 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:17:43.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.968+0000 7f6b7f52eb80 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:17:43.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.968+0000 7f6b7f52eb80 20 client.25876 populate_metadata read hostname 'smithi130' 2023-09-12T16:17:43.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.968+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x56112dfa82b0 0x56112dfaa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:43.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.968+0000 7f6b7f52eb80 4 client.25876 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:17:43.980 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.968+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/685558177 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56112def98a0 con 0x56112dfa82b0 2023-09-12T16:17:43.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.968+0000 7f6b71104700 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x56112dfa82b0 0x56112dfaa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:43.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.969+0000 7f6b71104700 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x56112dfa82b0 0x56112dfaa6d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5974795075c3775 server_cookie=83712af967b62958 in_seq=0 out_seq=0 2023-09-12T16:17:43.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.969+0000 7f6b54ff9700 10 client.25876 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:17:43.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b54ff9700 1 -- 172.21.15.130:0/685558177 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56112def98a0 con 0x56112dfa82b0 2023-09-12T16:17:43.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b54ff9700 10 client.25876 handle_command_reply: tid=0 2023-09-12T16:17:43.981 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 client.25876 shutdown 2023-09-12T16:17:43.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/685558177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 msgr2=0x56112df9ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:43.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df9ca80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0098e0 tx=0x7f6b5c005a90 comp rx=0 tx=0).stop 2023-09-12T16:17:43.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/685558177 shutdown_connections 2023-09-12T16:17:43.982 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x56112dfa82b0 0x56112dfaa6d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112dfa3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b54ff9700 0 client.25876 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:17:43.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df9c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/685558177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112df9ca80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:43.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/685558177 >> 172.21.15.130:0/685558177 conn(0x56112df07090 msgr2=0x56112df8cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:43.983 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/685558177 shutdown_connections 2023-09-12T16:17:43.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/685558177 wait complete. 2023-09-12T16:17:43.984 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:43.971+0000 7f6b7f52eb80 20 client.25876 trim_cache size 0 max 16384 2023-09-12T16:17:44.000 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.000+0000 7f6b7f52eb80 1 Processor -- start 2023-09-12T16:17:44.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b7f52eb80 1 -- start start 2023-09-12T16:17:44.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df92e00 0x56112df9a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:44.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df93c00 0x56112dfa39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:44.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df945a0 0x56112df9a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:44.001 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112df9b9f0 con 0x56112df945a0 2023-09-12T16:17:44.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112df9c0b0 con 0x56112df93c00 2023-09-12T16:17:44.002 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112df9b120 con 0x56112df92e00 2023-09-12T16:17:44.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b70903700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df93c00 0x56112dfa39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:44.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b70903700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df93c00 0x56112dfa39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:39990/0 (socket says 172.21.15.130:39990) 2023-09-12T16:17:44.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b70903700 1 -- 172.21.15.130:0/2990704771 learned_addr learned my addr 172.21.15.130:0/2990704771 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:44.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b71104700 1 --2- 172.21.15.130:0/2990704771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df92e00 0x56112df9a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:44.003 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b70903700 1 -- 172.21.15.130:0/2990704771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df92e00 msgr2=0x56112df9a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:44.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b70903700 1 --2- 172.21.15.130:0/2990704771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df92e00 0x56112df9a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b70903700 1 -- 172.21.15.130:0/2990704771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df945a0 msgr2=0x56112df9a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:44.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b70903700 1 --2- 172.21.15.130:0/2990704771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df945a0 0x56112df9a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.001+0000 7f6b70903700 1 -- 172.21.15.130:0/2990704771 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56112dd37250 con 0x56112df93c00 2023-09-12T16:17:44.004 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.002+0000 7f6b70903700 1 --2- 172.21.15.130:0/2990704771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df93c00 0x56112dfa39e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00a010 tx=0x7f6b5c009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332030b8406160 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:44.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.002+0000 7f6b57fff700 1 -- 172.21.15.130:0/2990704771 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c009290 con 0x56112df93c00 2023-09-12T16:17:44.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.002+0000 7f6b57fff700 1 -- 172.21.15.130:0/2990704771 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b5c01e040 con 0x56112df93c00 2023-09-12T16:17:44.005 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.002+0000 7f6b57fff700 1 -- 172.21.15.130:0/2990704771 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c005280 con 0x56112df93c00 2023-09-12T16:17:44.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/2990704771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df93c00 msgr2=0x56112dfa39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:44.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/2990704771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df93c00 0x56112dfa39e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00a010 tx=0x7f6b5c009910 comp rx=0 tx=0).stop 2023-09-12T16:17:44.006 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/2990704771 shutdown_connections 2023-09-12T16:17:44.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/2990704771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df945a0 0x56112df9a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/2990704771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df92e00 0x56112df9a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/2990704771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df93c00 0x56112dfa39e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/2990704771 >> 172.21.15.130:0/2990704771 conn(0x56112df07090 msgr2=0x56112df96e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:44.007 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/2990704771 shutdown_connections 2023-09-12T16:17:44.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/2990704771 wait complete. 2023-09-12T16:17:44.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 -- start start 2023-09-12T16:17:44.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.004+0000 7f6b7f52eb80 1 Processor -- start 2023-09-12T16:17:44.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df9fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:44.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112dfa0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:44.008 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b7f52eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112dfa3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:44.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112dfa3690 con 0x56112df92e00 2023-09-12T16:17:44.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112dfa3800 con 0x56112df945a0 2023-09-12T16:17:44.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b7f52eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112dfa3970 con 0x56112df93c00 2023-09-12T16:17:44.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b7f52eb80 10 client.0 _ll_register_callbacks cb 0x7f6b5c00baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:17:44.009 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b70903700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112dfa0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:44.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b70903700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112dfa0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36350/0 (socket says 172.21.15.130:36350) 2023-09-12T16:17:44.010 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.005+0000 7f6b70903700 1 -- 172.21.15.130:0/1934076547 learned_addr learned my addr 172.21.15.130:0/1934076547 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:17:44.012 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.012+0000 7f6b70903700 1 -- 172.21.15.130:0/1934076547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 msgr2=0x56112dfa3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:44.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.012+0000 7f6b70903700 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112dfa3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.013+0000 7f6b70903700 1 -- 172.21.15.130:0/1934076547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 msgr2=0x56112df9fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:17:44.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.013+0000 7f6b70903700 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df9fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.013+0000 7f6b70903700 1 -- 172.21.15.130:0/1934076547 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56112dcf9c60 con 0x56112df93c00 2023-09-12T16:17:44.013 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.013+0000 7f6b70903700 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112dfa0130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c000c00 tx=0x7f6b5c005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40d35eb86692d247 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:17:44.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.013+0000 7f6b54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:17:44.014 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.013+0000 7f6b54ff9700 1 -- 172.21.15.130:0/1934076547 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c005220 con 0x56112df93c00 2023-09-12T16:17:44.018 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.019+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/1934076547 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b6400b5a0 con 0x56112df93c00 2023-09-12T16:17:44.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.021+0000 7f6b54ff9700 1 -- 172.21.15.130:0/1934076547 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6b5c000e40 con 0x56112df93c00 2023-09-12T16:17:44.022 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.021+0000 7f6b54ff9700 1 -- 172.21.15.130:0/1934076547 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b5c004e80 con 0x56112df93c00 2023-09-12T16:17:44.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.023+0000 7f6b7f52eb80 10 client.25882 fetch_fsmap learned FSMap version 32 2023-09-12T16:17:44.023 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.023+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/1934076547 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56112dd0f2b0 con 0x56112df93c00 2023-09-12T16:17:44.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.036+0000 7f6b54ff9700 1 -- 172.21.15.130:0/1934076547 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b6400b5a0 con 0x56112df93c00 2023-09-12T16:17:44.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.036+0000 7f6b54ff9700 1 -- 172.21.15.130:0/1934076547 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b5c02c070 con 0x56112df93c00 2023-09-12T16:17:44.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.036+0000 7f6b7f52eb80 10 client.25882 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:17:44.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.036+0000 7f6b7f52eb80 10 client.25882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:17:44.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.036+0000 7f6b7f52eb80 20 client.25882 populate_metadata read hostname 'smithi130' 2023-09-12T16:17:44.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.036+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x56112e00e080 0x56112e010470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:17:44.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.036+0000 7f6b7f52eb80 4 client.25882 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:17:44.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.036+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/1934076547 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56112dedd920 con 0x56112e00e080 2023-09-12T16:17:44.048 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.049+0000 7f6b71104700 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x56112e00e080 0x56112e010470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:17:44.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.072+0000 7f6b71104700 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x56112e00e080 0x56112e010470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2796425ed56636b8 server_cookie=c1a0138ae248dca2 in_seq=0 out_seq=0 2023-09-12T16:17:44.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.072+0000 7f6b54ff9700 10 client.25882 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:17:44.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.075+0000 7f6b54ff9700 1 -- 172.21.15.130:0/1934076547 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56112dedd920 con 0x56112e00e080 2023-09-12T16:17:44.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.075+0000 7f6b54ff9700 10 client.25882 handle_command_reply: tid=0 2023-09-12T16:17:44.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.079+0000 7f6b7f52eb80 1 client.25882 shutdown 2023-09-12T16:17:44.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.084+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/1934076547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 msgr2=0x56112dfa0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:44.084 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.084+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112dfa0130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c000c00 tx=0x7f6b5c005ab0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.086+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/1934076547 shutdown_connections 2023-09-12T16:17:44.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.086+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x56112e00e080 0x56112e010470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.086 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.086+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56112df945a0 0x56112dfa3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.086+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56112df92e00 0x56112df9fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.086+0000 7f6b7f52eb80 1 --2- 172.21.15.130:0/1934076547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112df93c00 0x56112dfa0130 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.086+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/1934076547 >> 172.21.15.130:0/1934076547 conn(0x56112df07090 msgr2=0x56112df9a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:44.087 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.086+0000 7f6b54ff9700 0 client.25882 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:17:44.090 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.090+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/1934076547 shutdown_connections 2023-09-12T16:17:44.099 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.100+0000 7f6b7f52eb80 1 -- 172.21.15.130:0/1934076547 wait complete. 2023-09-12T16:17:44.106 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.107+0000 7f6b7f52eb80 20 client.25882 trim_cache size 0 max 16384 2023-09-12T16:17:44.108 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:17:44.108 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:17:44.109 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:17:44.109 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:17:44.109 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:17:44.109 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:17:44.109 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:17:44.110 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:17:44.110 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:17:44.110 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:17:44.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.110+0000 7f6b73368700 1 -- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6b580a05d0 msgr2=0x7f6b580a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:44.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.110+0000 7f6b73368700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6b580a05d0 0x7f6b580a2a90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6b64018000 tx=0x7f6b64016640 comp rx=0 tx=0).stop 2023-09-12T16:17:44.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.110+0000 7f6b73368700 1 -- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b6c0ff500 msgr2=0x7f6b6c19c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:17:44.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.110+0000 7f6b73368700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b6c0ff500 0x7f6b6c19c950 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800e970 tx=0x7f6b6800ec80 comp rx=0 tx=0).stop 2023-09-12T16:17:44.119 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.120+0000 7f6b73368700 1 -- 172.21.15.130:0/2312561998 shutdown_connections 2023-09-12T16:17:44.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.120+0000 7f6b73368700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f6b580a05d0 0x7f6b580a2a90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.120+0000 7f6b73368700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c101090 0x7f6b6c1a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.120+0000 7f6b73368700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b6c0ff500 0x7f6b6c19c950 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.120+0000 7f6b73368700 1 --2- 172.21.15.130:0/2312561998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b6c1006f0 0x7f6b6c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:17:44.120 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.120+0000 7f6b73368700 1 -- 172.21.15.130:0/2312561998 >> 172.21.15.130:0/2312561998 conn(0x7f6b6c0faa90 msgr2=0x7f6b6c10be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:17:44.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.127+0000 7f6b73368700 1 -- 172.21.15.130:0/2312561998 shutdown_connections 2023-09-12T16:17:44.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:17:44.127+0000 7f6b73368700 1 -- 172.21.15.130:0/2312561998 wait complete. 2023-09-12T16:17:44.150 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:17:45.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 715.0 s, 1263.0 tps, lat 78.480 ms stddev 142.861 2023-09-12T16:17:45.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:45 smithi130 ceph-mon[98670]: pgmap v736: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 27 MiB/s wr, 2.24k op/s 2023-09-12T16:17:45.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:45 smithi159 ceph-mon[102522]: pgmap v736: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 27 MiB/s wr, 2.24k op/s 2023-09-12T16:17:45.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:45 smithi203 ceph-mon[103431]: pgmap v736: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 27 MiB/s wr, 2.24k op/s 2023-09-12T16:17:46.981 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:17:46.981 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:46 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:17:46.989 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:17:46.990 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:46 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:17:47.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:17:47.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:46 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:17:47.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:47 smithi130 ceph-mon[98670]: pgmap v737: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-09-12T16:17:47.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:17:47.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:17:47.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:17:47.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:47 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:17:47.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:47 smithi159 ceph-mon[102522]: pgmap v737: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-09-12T16:17:47.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:17:47.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:17:47.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:17:47.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:47 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:17:47.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:47 smithi203 ceph-mon[103431]: pgmap v737: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-09-12T16:17:47.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:17:47.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:17:47.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:17:47.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:47 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:17:48.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:48 smithi130 ceph-mon[98670]: pgmap v738: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 23 MiB/s wr, 1.89k op/s 2023-09-12T16:17:48.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:48 smithi159 ceph-mon[102522]: pgmap v738: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 23 MiB/s wr, 1.89k op/s 2023-09-12T16:17:48.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:48 smithi203 ceph-mon[103431]: pgmap v738: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 23 MiB/s wr, 1.89k op/s 2023-09-12T16:17:50.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 720.0 s, 708.4 tps, lat 141.638 ms stddev 177.485 2023-09-12T16:17:50.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:50 smithi130 ceph-mon[98670]: pgmap v739: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 1.59k op/s 2023-09-12T16:17:50.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:50 smithi159 ceph-mon[102522]: pgmap v739: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 1.59k op/s 2023-09-12T16:17:50.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:50 smithi203 ceph-mon[103431]: pgmap v739: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 1.59k op/s 2023-09-12T16:17:53.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:53 smithi130 ceph-mon[98670]: pgmap v740: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-09-12T16:17:53.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:53 smithi159 ceph-mon[102522]: pgmap v740: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-09-12T16:17:53.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:53 smithi203 ceph-mon[103431]: pgmap v740: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-09-12T16:17:55.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 725.0 s, 1496.6 tps, lat 67.482 ms stddev 52.706 2023-09-12T16:17:55.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:55 smithi159 ceph-mon[102522]: pgmap v741: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-09-12T16:17:55.729 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:55 smithi130 ceph-mon[98670]: pgmap v741: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-09-12T16:17:55.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:55 smithi203 ceph-mon[103431]: pgmap v741: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-09-12T16:17:57.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:57 smithi159 ceph-mon[102522]: pgmap v742: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-09-12T16:17:57.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:57 smithi130 ceph-mon[98670]: pgmap v742: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-09-12T16:17:57.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:57 smithi203 ceph-mon[103431]: pgmap v742: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-09-12T16:17:59.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:17:59 smithi203 ceph-mon[103431]: pgmap v743: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2023-09-12T16:17:59.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:17:59 smithi159 ceph-mon[102522]: pgmap v743: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2023-09-12T16:17:59.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:17:59 smithi130 ceph-mon[98670]: pgmap v743: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2023-09-12T16:18:00.092 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 730.0 s, 1122.5 tps, lat 78.589 ms stddev 155.750 2023-09-12T16:18:00.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:00 smithi130 ceph-mon[98670]: pgmap v744: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 1.03k op/s 2023-09-12T16:18:00.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:00 smithi159 ceph-mon[102522]: pgmap v744: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 1.03k op/s 2023-09-12T16:18:00.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:00 smithi203 ceph-mon[103431]: pgmap v744: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 1.03k op/s 2023-09-12T16:18:03.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:03 smithi130 ceph-mon[98670]: pgmap v745: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 18 MiB/s wr, 1.13k op/s 2023-09-12T16:18:03.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:03 smithi159 ceph-mon[102522]: pgmap v745: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 18 MiB/s wr, 1.13k op/s 2023-09-12T16:18:03.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:03 smithi203 ceph-mon[103431]: pgmap v745: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 18 MiB/s wr, 1.13k op/s 2023-09-12T16:18:05.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 735.0 s, 317.8 tps, lat 352.750 ms stddev 615.753 2023-09-12T16:18:05.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:05 smithi130 ceph-mon[98670]: pgmap v746: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 25 MiB/s wr, 2.10k op/s 2023-09-12T16:18:05.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:05 smithi159 ceph-mon[102522]: pgmap v746: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 25 MiB/s wr, 2.10k op/s 2023-09-12T16:18:05.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:05 smithi203 ceph-mon[103431]: pgmap v746: 129 pgs: 129 active+clean; 8.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 25 MiB/s wr, 2.10k op/s 2023-09-12T16:18:07.748 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:07 smithi203 ceph-mon[103431]: pgmap v747: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 27 MiB/s wr, 2.49k op/s 2023-09-12T16:18:07.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:07 smithi159 ceph-mon[102522]: pgmap v747: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 27 MiB/s wr, 2.49k op/s 2023-09-12T16:18:07.982 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:07 smithi130 ceph-mon[98670]: pgmap v747: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 27 MiB/s wr, 2.49k op/s 2023-09-12T16:18:09.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:09 smithi130 ceph-mon[98670]: pgmap v748: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 27 MiB/s wr, 2.65k op/s 2023-09-12T16:18:09.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:09 smithi159 ceph-mon[102522]: pgmap v748: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 27 MiB/s wr, 2.65k op/s 2023-09-12T16:18:09.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:09 smithi203 ceph-mon[103431]: pgmap v748: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 27 MiB/s wr, 2.65k op/s 2023-09-12T16:18:10.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 740.0 s, 1395.7 tps, lat 71.820 ms stddev 150.903 2023-09-12T16:18:10.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:10 smithi159 ceph-mon[102522]: pgmap v749: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 30 MiB/s wr, 3.07k op/s 2023-09-12T16:18:10.980 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:10 smithi130 ceph-mon[98670]: pgmap v749: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 30 MiB/s wr, 3.07k op/s 2023-09-12T16:18:11.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:10 smithi203 ceph-mon[103431]: pgmap v749: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 30 MiB/s wr, 3.07k op/s 2023-09-12T16:18:13.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:13 smithi130 ceph-mon[98670]: pgmap v750: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 29 MiB/s wr, 2.98k op/s 2023-09-12T16:18:13.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:13 smithi159 ceph-mon[102522]: pgmap v750: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 29 MiB/s wr, 2.98k op/s 2023-09-12T16:18:13.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:13 smithi203 ceph-mon[103431]: pgmap v750: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 29 MiB/s wr, 2.98k op/s 2023-09-12T16:18:14.151 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:18:14.393 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.393+0000 7f8598c3d700 1 -- 172.21.15.130:0/36276499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f859410c940 msgr2=0x7f859410cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:14.394 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.393+0000 7f8598c3d700 1 --2- 172.21.15.130:0/36276499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f859410c940 0x7f859410cdc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f858c01c0d0 tx=0x7f858c01c3e0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.398+0000 7f8598c3d700 1 -- 172.21.15.130:0/36276499 shutdown_connections 2023-09-12T16:18:14.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.398+0000 7f8598c3d700 1 --2- 172.21.15.130:0/36276499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f859410c940 0x7f859410cdc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.399 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.398+0000 7f8598c3d700 1 --2- 172.21.15.130:0/36276499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8594107600 0x7f8594109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.400 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.398+0000 7f8598c3d700 1 --2- 172.21.15.130:0/36276499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8594109f70 0x7f859410c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.400 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.398+0000 7f8598c3d700 1 -- 172.21.15.130:0/36276499 >> 172.21.15.130:0/36276499 conn(0x7f859406d3b0 msgr2=0x7f859406f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:14.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.406+0000 7f8598c3d700 1 -- 172.21.15.130:0/36276499 shutdown_connections 2023-09-12T16:18:14.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.406+0000 7f8598c3d700 1 -- 172.21.15.130:0/36276499 wait complete. 2023-09-12T16:18:14.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.407+0000 7f8598c3d700 1 Processor -- start 2023-09-12T16:18:14.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.407+0000 7f8598c3d700 1 -- start start 2023-09-12T16:18:14.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8598c3d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8594107600 0x7f859419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8598c3d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8594109f70 0x7f859419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8598c3d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f859410c940 0x7f85941a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8598c3d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85941a4540 con 0x7f859410c940 2023-09-12T16:18:14.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8598c3d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85941a4680 con 0x7f8594107600 2023-09-12T16:18:14.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8598c3d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85941a4810 con 0x7f8594109f70 2023-09-12T16:18:14.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f859259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8594107600 0x7f859419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8592d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f859410c940 0x7f85941a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8592d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f859410c940 0x7f85941a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56752/0 (socket says 172.21.15.130:56752) 2023-09-12T16:18:14.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8592d9d700 1 -- 172.21.15.130:0/4207774054 learned_addr learned my addr 172.21.15.130:0/4207774054 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:14.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8592d9d700 1 -- 172.21.15.130:0/4207774054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8594109f70 msgr2=0x7f859419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:14.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8592d9d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8594109f70 0x7f859419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8592d9d700 1 -- 172.21.15.130:0/4207774054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8594107600 msgr2=0x7f859419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:14.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.408+0000 7f8592d9d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8594107600 0x7f859419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.409+0000 7f8592d9d700 1 -- 172.21.15.130:0/4207774054 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c01ad10 con 0x7f859410c940 2023-09-12T16:18:14.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.409+0000 7f8592d9d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f859410c940 0x7f85941a3f70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f858c01a3c0 tx=0x7f858c01a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:14.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.409+0000 7f85837fe700 1 -- 172.21.15.130:0/4207774054 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c01f040 con 0x7f859410c940 2023-09-12T16:18:14.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.409+0000 7f85837fe700 1 -- 172.21.15.130:0/4207774054 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f858c02c920 con 0x7f859410c940 2023-09-12T16:18:14.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.409+0000 7f85837fe700 1 -- 172.21.15.130:0/4207774054 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c005560 con 0x7f859410c940 2023-09-12T16:18:14.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.409+0000 7f8598c3d700 1 -- 172.21.15.130:0/4207774054 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85941a4a90 con 0x7f859410c940 2023-09-12T16:18:14.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.409+0000 7f8598c3d700 1 -- 172.21.15.130:0/4207774054 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85941a4ec0 con 0x7f859410c940 2023-09-12T16:18:14.416 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.415+0000 7f85a4e03b80 1 Processor -- start 2023-09-12T16:18:14.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.416+0000 7f85a4e03b80 1 -- start start 2023-09-12T16:18:14.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.417+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223efc00 0x556f223f0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.417+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 0x556f223ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.417+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223eee00 0x556f223ef2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.417+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f22347730 con 0x556f223f05a0 2023-09-12T16:18:14.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.417+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f22345050 con 0x556f223eee00 2023-09-12T16:18:14.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.417+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f22340940 con 0x556f223efc00 2023-09-12T16:18:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.422+0000 7f85837fe700 1 -- 172.21.15.130:0/4207774054 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f858c0056c0 con 0x7f859410c940 2023-09-12T16:18:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.423+0000 7f859259c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 0x556f223ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.423+0000 7f859259c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 0x556f223ee800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56762/0 (socket says 172.21.15.130:56762) 2023-09-12T16:18:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.423+0000 7f859259c700 1 -- 172.21.15.130:0/3546831 learned_addr learned my addr 172.21.15.130:0/3546831 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.423+0000 7f85837fe700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f857c0a0790 0x7f857c0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.423+0000 7f85837fe700 1 -- 172.21.15.130:0/4207774054 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f858c0dde80 con 0x7f859410c940 2023-09-12T16:18:14.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.423+0000 7f859259c700 1 -- 172.21.15.130:0/3546831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223efc00 msgr2=0x556f223f0020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:18:14.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.423+0000 7f859259c700 1 --2- 172.21.15.130:0/3546831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223efc00 0x556f223f0020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.423+0000 7f859259c700 1 -- 172.21.15.130:0/3546831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223eee00 msgr2=0x556f223ef2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-12T16:18:14.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.424+0000 7f859259c700 1 --2- 172.21.15.130:0/3546831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223eee00 0x556f223ef2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.424+0000 7f859259c700 1 -- 172.21.15.130:0/3546831 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f2216cde0 con 0x556f223f05a0 2023-09-12T16:18:14.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.424+0000 7f859259c700 1 --2- 172.21.15.130:0/3546831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 0x556f223ee800 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8584009af0 tx=0x7f8584009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3190755f400dbf1 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:14.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.424+0000 7f8580ff9700 1 -- 172.21.15.130:0/3546831 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858400e990 con 0x556f223f05a0 2023-09-12T16:18:14.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.424+0000 7f8580ff9700 1 -- 172.21.15.130:0/3546831 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8584013470 con 0x556f223f05a0 2023-09-12T16:18:14.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.424+0000 7f8580ff9700 1 -- 172.21.15.130:0/3546831 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85840125d0 con 0x556f223f05a0 2023-09-12T16:18:14.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.425+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3546831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 msgr2=0x556f223ee800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:14.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.425+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/3546831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 0x556f223ee800 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8584009af0 tx=0x7f8584009e00 comp rx=0 tx=0).stop 2023-09-12T16:18:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.425+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3546831 shutdown_connections 2023-09-12T16:18:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.426+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/3546831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223eee00 0x556f223ef2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.426+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/3546831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223efc00 0x556f223f0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.426+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/3546831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 0x556f223ee800 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.426+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3546831 >> 172.21.15.130:0/3546831 conn(0x556f22363090 msgr2=0x556f223e9720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:14.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.429+0000 7f8592d9d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f857c0a0790 0x7f857c0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.429+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3546831 shutdown_connections 2023-09-12T16:18:14.431 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.430+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3546831 wait complete. 2023-09-12T16:18:14.431 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.430+0000 7f85a4e03b80 1 -- start start 2023-09-12T16:18:14.431 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.430+0000 7f85a4e03b80 1 Processor -- start 2023-09-12T16:18:14.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.432+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 0x556f223f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.432+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223fdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.437 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.432+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22400320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.432+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f222deb70 con 0x556f223f05a0 2023-09-12T16:18:14.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.432+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f222de870 con 0x556f223f8d40 2023-09-12T16:18:14.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.432+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f221c2920 con 0x556f223f91c0 2023-09-12T16:18:14.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.432+0000 7f85a4e03b80 10 client.0 _ll_register_callbacks cb 0x556f22228a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:18:14.438 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.436+0000 7f859259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223fdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.436+0000 7f859259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223fdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40024/0 (socket says 172.21.15.130:40024) 2023-09-12T16:18:14.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.436+0000 7f859259c700 1 -- 172.21.15.130:0/1197559607 learned_addr learned my addr 172.21.15.130:0/1197559607 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:14.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.436+0000 7f8591d9b700 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22400320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.436+0000 7f859259c700 1 -- 172.21.15.130:0/1197559607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 msgr2=0x556f22400320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:14.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.436+0000 7f859259c700 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22400320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.439 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.436+0000 7f859259c700 1 -- 172.21.15.130:0/1197559607 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 msgr2=0x556f223f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:14.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.436+0000 7f859259c700 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 0x556f223f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.436+0000 7f859259c700 1 -- 172.21.15.130:0/1197559607 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8584009750 con 0x556f223f8d40 2023-09-12T16:18:14.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.437+0000 7f8592d9d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f857c0a0790 0x7f857c0a2c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f858c0b32c0 tx=0x7f858c01c3e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:14.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.437+0000 7f859259c700 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223fdd70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8584004790 tx=0x7f858400ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb04d673895c841 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:14.440 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.437+0000 7f8579ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:18:14.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.437+0000 7f8579ffb700 1 -- 172.21.15.130:0/1197559607 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584013be0 con 0x556f223f8d40 2023-09-12T16:18:14.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.437+0000 7f8579ffb700 1 -- 172.21.15.130:0/1197559607 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8584006df0 con 0x556f223f8d40 2023-09-12T16:18:14.442 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.437+0000 7f85a4e03b80 1 -- 172.21.15.130:0/1197559607 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f222984f0 con 0x556f223f8d40 2023-09-12T16:18:14.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.437+0000 7f8579ffb700 1 -- 172.21.15.130:0/1197559607 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f858401c660 con 0x556f223f8d40 2023-09-12T16:18:14.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.437+0000 7f85a4e03b80 10 client.25682 fetch_fsmap learned FSMap version 32 2023-09-12T16:18:14.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.437+0000 7f85a4e03b80 1 -- 172.21.15.130:0/1197559607 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556f22155c60 con 0x556f223f8d40 2023-09-12T16:18:14.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.441+0000 7f8579ffb700 1 -- 172.21.15.130:0/1197559607 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f222984f0 con 0x556f223f8d40 2023-09-12T16:18:14.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.441+0000 7f8579ffb700 1 -- 172.21.15.130:0/1197559607 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8584011070 con 0x556f223f8d40 2023-09-12T16:18:14.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.441+0000 7f85a4e03b80 10 client.25682 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:18:14.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.441+0000 7f85a4e03b80 10 client.25682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:18:14.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.441+0000 7f85a4e03b80 20 client.25682 populate_metadata read hostname 'smithi130' 2023-09-12T16:18:14.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.441+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556f22404d70 0x556f22407190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.441+0000 7f85a4e03b80 4 client.25682 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:18:14.445 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.441+0000 7f85a4e03b80 1 -- 172.21.15.130:0/1197559607 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f223558a0 con 0x556f22404d70 2023-09-12T16:18:14.445 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.441+0000 7f8591d9b700 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556f22404d70 0x556f22407190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.443+0000 7f8579ffb700 10 client.25682 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:18:14.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.443+0000 7f8591d9b700 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556f22404d70 0x556f22407190 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6817669f9048fb9f server_cookie=f1d3834ca357e3cd in_seq=0 out_seq=0 2023-09-12T16:18:14.446 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f8579ffb700 1 -- 172.21.15.130:0/1197559607 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f223558a0 con 0x556f22404d70 2023-09-12T16:18:14.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f8579ffb700 10 client.25682 handle_command_reply: tid=0 2023-09-12T16:18:14.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 client.25682 shutdown 2023-09-12T16:18:14.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 -- 172.21.15.130:0/1197559607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 msgr2=0x556f223fdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:14.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223fdd70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8584004790 tx=0x7f858400ec70 comp rx=0 tx=0).stop 2023-09-12T16:18:14.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 -- 172.21.15.130:0/1197559607 shutdown_connections 2023-09-12T16:18:14.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556f22404d70 0x556f22407190 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22400320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f8579ffb700 0 client.25682 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:18:14.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f05a0 0x556f223f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/1197559607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223fdd70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 -- 172.21.15.130:0/1197559607 >> 172.21.15.130:0/1197559607 conn(0x556f22363090 msgr2=0x556f223e8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:14.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 -- 172.21.15.130:0/1197559607 shutdown_connections 2023-09-12T16:18:14.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 1 -- 172.21.15.130:0/1197559607 wait complete. 2023-09-12T16:18:14.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.444+0000 7f85a4e03b80 20 client.25682 trim_cache size 0 max 16384 2023-09-12T16:18:14.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.475+0000 7f85a4e03b80 1 Processor -- start 2023-09-12T16:18:14.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.475+0000 7f85a4e03b80 1 -- start start 2023-09-12T16:18:14.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f05a0 0x556f22414930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.477 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f91c0 0x556f223f3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f223f7cb0 con 0x556f223f91c0 2023-09-12T16:18:14.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f223f8370 con 0x556f223f8d40 2023-09-12T16:18:14.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f223e1400 con 0x556f223f05a0 2023-09-12T16:18:14.478 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f8591d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f05a0 0x556f22414930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f8591d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f05a0 0x556f22414930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36386/0 (socket says 172.21.15.130:36386) 2023-09-12T16:18:14.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f859259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f8591d9b700 1 -- 172.21.15.130:0/3430290439 learned_addr learned my addr 172.21.15.130:0/3430290439 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:14.479 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f8591d9b700 1 --2- 172.21.15.130:0/3430290439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f91c0 0x556f223f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f8591d9b700 1 -- 172.21.15.130:0/3430290439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 msgr2=0x556f223f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:14.480 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f8591d9b700 1 --2- 172.21.15.130:0/3430290439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f8591d9b700 1 -- 172.21.15.130:0/3430290439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f91c0 msgr2=0x556f223f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:14.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f8591d9b700 1 --2- 172.21.15.130:0/3430290439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f91c0 0x556f223f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.476+0000 7f8591d9b700 1 -- 172.21.15.130:0/3430290439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f22193250 con 0x556f223f05a0 2023-09-12T16:18:14.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.477+0000 7f8591d9b700 1 --2- 172.21.15.130:0/3430290439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f05a0 0x556f22414930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x556f223ea760 tx=0x7f8588005e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b6781aa634825bc server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:14.481 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.477+0000 7f85817fa700 1 -- 172.21.15.130:0/3430290439 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588004220 con 0x556f223f05a0 2023-09-12T16:18:14.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.477+0000 7f85817fa700 1 -- 172.21.15.130:0/3430290439 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8588004380 con 0x556f223f05a0 2023-09-12T16:18:14.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.477+0000 7f85817fa700 1 -- 172.21.15.130:0/3430290439 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588003ce0 con 0x556f223f05a0 2023-09-12T16:18:14.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.478+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3430290439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f05a0 msgr2=0x556f22414930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:14.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.478+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/3430290439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f05a0 0x556f22414930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x556f223ea760 tx=0x7f8588005e90 comp rx=0 tx=0).stop 2023-09-12T16:18:14.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.478+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3430290439 shutdown_connections 2023-09-12T16:18:14.482 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.478+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/3430290439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f91c0 0x556f223f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.478+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/3430290439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f05a0 0x556f22414930 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.478+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/3430290439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f8d40 0x556f223f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.483 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.478+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3430290439 >> 172.21.15.130:0/3430290439 conn(0x556f22363090 msgr2=0x556f223f7360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:14.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.483+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3430290439 shutdown_connections 2023-09-12T16:18:14.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.483+0000 7f85a4e03b80 1 -- 172.21.15.130:0/3430290439 wait complete. 2023-09-12T16:18:14.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.483+0000 7f85a4e03b80 1 -- start start 2023-09-12T16:18:14.484 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.483+0000 7f85a4e03b80 1 Processor -- start 2023-09-12T16:18:14.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f05a0 0x556f224146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.487 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f8d40 0x556f22412cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f85a4e03b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22413230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f22414f00 con 0x556f223f8d40 2023-09-12T16:18:14.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f22413770 con 0x556f223f05a0 2023-09-12T16:18:14.488 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f85a4e03b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f22413930 con 0x556f223f91c0 2023-09-12T16:18:14.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f85a4e03b80 10 client.0 _ll_register_callbacks cb 0x7f8584009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:18:14.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22413230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22413230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36394/0 (socket says 172.21.15.130:36394) 2023-09-12T16:18:14.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 -- 172.21.15.130:0/153498759 learned_addr learned my addr 172.21.15.130:0/153498759 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:14.489 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f859259c700 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f8d40 0x556f22412cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f05a0 0x556f224146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 -- 172.21.15.130:0/153498759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f05a0 msgr2=0x556f224146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:14.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f05a0 0x556f224146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 -- 172.21.15.130:0/153498759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f8d40 msgr2=0x556f22412cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:14.490 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f8d40 0x556f22412cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 -- 172.21.15.130:0/153498759 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85880163c0 con 0x556f223f91c0 2023-09-12T16:18:14.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8591d9b700 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22413230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8588005c40 tx=0x7f8588013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2a0ad863b417555 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:14.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.484+0000 7f8579ffb700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:18:14.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f8579ffb700 1 -- 172.21.15.130:0/153498759 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588003e70 con 0x556f223f91c0 2023-09-12T16:18:14.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f8579ffb700 1 -- 172.21.15.130:0/153498759 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8588000e40 con 0x556f223f91c0 2023-09-12T16:18:14.491 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f85a4e03b80 1 -- 172.21.15.130:0/153498759 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8584013470 con 0x556f223f91c0 2023-09-12T16:18:14.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f8579ffb700 1 -- 172.21.15.130:0/153498759 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8588016730 con 0x556f223f91c0 2023-09-12T16:18:14.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f85a4e03b80 10 client.25888 fetch_fsmap learned FSMap version 32 2023-09-12T16:18:14.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f85a4e03b80 1 -- 172.21.15.130:0/153498759 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556f2216b2b0 con 0x556f223f91c0 2023-09-12T16:18:14.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f8579ffb700 1 -- 172.21.15.130:0/153498759 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8584013470 con 0x556f223f91c0 2023-09-12T16:18:14.492 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f8579ffb700 1 -- 172.21.15.130:0/153498759 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8588024940 con 0x556f223f91c0 2023-09-12T16:18:14.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f85a4e03b80 10 client.25888 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:18:14.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f85a4e03b80 10 client.25888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:18:14.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f85a4e03b80 20 client.25888 populate_metadata read hostname 'smithi130' 2023-09-12T16:18:14.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556f2245a920 0x556f2245cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:14.493 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f85a4e03b80 4 client.25888 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:18:14.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.485+0000 7f85a4e03b80 1 -- 172.21.15.130:0/153498759 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556f22339920 con 0x556f2245a920 2023-09-12T16:18:14.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.486+0000 7f859259c700 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556f2245a920 0x556f2245cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:14.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.486+0000 7f8579ffb700 10 client.25888 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:18:14.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.486+0000 7f859259c700 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556f2245a920 0x556f2245cd40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb9aa3ce3cbe195 server_cookie=aedd0defaa0a5649 in_seq=0 out_seq=0 2023-09-12T16:18:14.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.486+0000 7f8579ffb700 1 -- 172.21.15.130:0/153498759 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556f22339920 con 0x556f2245a920 2023-09-12T16:18:14.494 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.486+0000 7f8579ffb700 10 client.25888 handle_command_reply: tid=0 2023-09-12T16:18:14.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.486+0000 7f85a4e03b80 1 client.25888 shutdown 2023-09-12T16:18:14.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 -- 172.21.15.130:0/153498759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 msgr2=0x556f22413230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:14.495 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22413230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8588005c40 tx=0x7f8588013040 comp rx=0 tx=0).stop 2023-09-12T16:18:14.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 -- 172.21.15.130:0/153498759 shutdown_connections 2023-09-12T16:18:14.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x556f2245a920 0x556f2245cd40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f223f91c0 0x556f22413230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.496 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f8579ffb700 0 client.25888 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:18:14.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f223f05a0 0x556f224146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 --2- 172.21.15.130:0/153498759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556f223f8d40 0x556f22412cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 -- 172.21.15.130:0/153498759 >> 172.21.15.130:0/153498759 conn(0x556f22363090 msgr2=0x556f223f2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:14.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 -- 172.21.15.130:0/153498759 shutdown_connections 2023-09-12T16:18:14.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.487+0000 7f85a4e03b80 1 -- 172.21.15.130:0/153498759 wait complete. 2023-09-12T16:18:14.497 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.488+0000 7f85a4e03b80 20 client.25888 trim_cache size 0 max 16384 2023-09-12T16:18:14.498 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:18:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:18:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:18:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:18:14.499 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:18:14.499 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:18:14.499 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:18:14.499 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:18:14.499 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:18:14.500 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:18:14.500 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.488+0000 7f8598c3d700 1 -- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f857c0a0790 msgr2=0x7f857c0a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:14.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.488+0000 7f8598c3d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f857c0a0790 0x7f857c0a2c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f858c0b32c0 tx=0x7f858c01c3e0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.488+0000 7f8598c3d700 1 -- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f859410c940 msgr2=0x7f85941a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:14.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.488+0000 7f8598c3d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f859410c940 0x7f85941a3f70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f858c01a3c0 tx=0x7f858c01a3f0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.489+0000 7f8598c3d700 1 -- 172.21.15.130:0/4207774054 shutdown_connections 2023-09-12T16:18:14.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.489+0000 7f8598c3d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f857c0a0790 0x7f857c0a2c50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.501 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.489+0000 7f8598c3d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f859410c940 0x7f85941a3f70 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.489+0000 7f8598c3d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8594107600 0x7f859419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.489+0000 7f8598c3d700 1 --2- 172.21.15.130:0/4207774054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8594109f70 0x7f859419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:14.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.489+0000 7f8598c3d700 1 -- 172.21.15.130:0/4207774054 >> 172.21.15.130:0/4207774054 conn(0x7f859406d3b0 msgr2=0x7f859406f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:14.502 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.489+0000 7f8598c3d700 1 -- 172.21.15.130:0/4207774054 shutdown_connections 2023-09-12T16:18:14.503 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:14.489+0000 7f8598c3d700 1 -- 172.21.15.130:0/4207774054 wait complete. 2023-09-12T16:18:14.509 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:18:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 745.0 s, 1287.9 tps, lat 77.488 ms stddev 116.536 2023-09-12T16:18:15.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:15 smithi159 ceph-mon[102522]: pgmap v751: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 31 MiB/s wr, 3.13k op/s 2023-09-12T16:18:15.735 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:15 smithi130 ceph-mon[98670]: pgmap v751: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 31 MiB/s wr, 3.13k op/s 2023-09-12T16:18:15.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:15 smithi203 ceph-mon[103431]: pgmap v751: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 31 MiB/s wr, 3.13k op/s 2023-09-12T16:18:17.726 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:17 smithi130 ceph-mon[98670]: pgmap v752: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 24 MiB/s wr, 2.33k op/s 2023-09-12T16:18:17.734 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:17 smithi159 ceph-mon[102522]: pgmap v752: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 24 MiB/s wr, 2.33k op/s 2023-09-12T16:18:17.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:17 smithi203 ceph-mon[103431]: pgmap v752: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 24 MiB/s wr, 2.33k op/s 2023-09-12T16:18:19.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:19 smithi130 ceph-mon[98670]: pgmap v753: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 21 MiB/s wr, 1.80k op/s 2023-09-12T16:18:19.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:19 smithi159 ceph-mon[102522]: pgmap v753: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 21 MiB/s wr, 1.80k op/s 2023-09-12T16:18:19.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:19 smithi203 ceph-mon[103431]: pgmap v753: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 21 MiB/s wr, 1.80k op/s 2023-09-12T16:18:20.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 750.0 s, 1232.6 tps, lat 79.829 ms stddev 104.503 2023-09-12T16:18:20.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:20 smithi130 ceph-mon[98670]: pgmap v754: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 24 MiB/s wr, 1.89k op/s 2023-09-12T16:18:20.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:20 smithi159 ceph-mon[102522]: pgmap v754: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 24 MiB/s wr, 1.89k op/s 2023-09-12T16:18:20.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:20 smithi203 ceph-mon[103431]: pgmap v754: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 24 MiB/s wr, 1.89k op/s 2023-09-12T16:18:23.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:23 smithi130 ceph-mon[98670]: pgmap v755: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 20 MiB/s wr, 1.41k op/s 2023-09-12T16:18:23.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:23 smithi159 ceph-mon[102522]: pgmap v755: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 20 MiB/s wr, 1.41k op/s 2023-09-12T16:18:23.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:23 smithi203 ceph-mon[103431]: pgmap v755: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 20 MiB/s wr, 1.41k op/s 2023-09-12T16:18:25.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 755.0 s, 1016.4 tps, lat 99.270 ms stddev 68.666 2023-09-12T16:18:25.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:25 smithi130 ceph-mon[98670]: pgmap v756: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-12T16:18:25.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:25 smithi159 ceph-mon[102522]: pgmap v756: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-12T16:18:25.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:25 smithi203 ceph-mon[103431]: pgmap v756: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-09-12T16:18:27.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:27 smithi130 ceph-mon[98670]: pgmap v757: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.24k op/s 2023-09-12T16:18:27.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:27 smithi159 ceph-mon[102522]: pgmap v757: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.24k op/s 2023-09-12T16:18:27.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:27 smithi203 ceph-mon[103431]: pgmap v757: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.24k op/s 2023-09-12T16:18:29.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:29 smithi130 ceph-mon[98670]: pgmap v758: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 1.10k op/s 2023-09-12T16:18:29.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:29 smithi159 ceph-mon[102522]: pgmap v758: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 1.10k op/s 2023-09-12T16:18:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:29 smithi203 ceph-mon[103431]: pgmap v758: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 1.10k op/s 2023-09-12T16:18:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 760.0 s, 1795.3 tps, lat 56.161 ms stddev 41.593 2023-09-12T16:18:31.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:31 smithi159 ceph-mon[102522]: pgmap v759: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 1.23k op/s 2023-09-12T16:18:31.728 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:31 smithi130 ceph-mon[98670]: pgmap v759: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 1.23k op/s 2023-09-12T16:18:31.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:31 smithi203 ceph-mon[103431]: pgmap v759: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 1.23k op/s 2023-09-12T16:18:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:33 smithi130 ceph-mon[98670]: pgmap v760: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 1.02k op/s 2023-09-12T16:18:33.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:33 smithi159 ceph-mon[102522]: pgmap v760: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 1.02k op/s 2023-09-12T16:18:33.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:33 smithi203 ceph-mon[103431]: pgmap v760: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 1.02k op/s 2023-09-12T16:18:35.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 765.0 s, 715.9 tps, lat 126.617 ms stddev 232.340 2023-09-12T16:18:35.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:35 smithi130 ceph-mon[98670]: pgmap v761: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-12T16:18:35.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:35 smithi159 ceph-mon[102522]: pgmap v761: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-12T16:18:35.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:35 smithi203 ceph-mon[103431]: pgmap v761: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-12T16:18:36.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:36 smithi130 ceph-mon[98670]: pgmap v762: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 22 MiB/s wr, 1.57k op/s 2023-09-12T16:18:36.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:36 smithi159 ceph-mon[102522]: pgmap v762: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 22 MiB/s wr, 1.57k op/s 2023-09-12T16:18:36.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:36 smithi203 ceph-mon[103431]: pgmap v762: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 22 MiB/s wr, 1.57k op/s 2023-09-12T16:18:39.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:39 smithi130 ceph-mon[98670]: pgmap v763: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 21 MiB/s wr, 1.60k op/s 2023-09-12T16:18:39.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:39 smithi159 ceph-mon[102522]: pgmap v763: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 21 MiB/s wr, 1.60k op/s 2023-09-12T16:18:39.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:39 smithi203 ceph-mon[103431]: pgmap v763: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 21 MiB/s wr, 1.60k op/s 2023-09-12T16:18:40.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 770.0 s, 351.2 tps, lat 307.305 ms stddev 606.933 2023-09-12T16:18:41.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:41 smithi159 ceph-mon[102522]: pgmap v764: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 32 MiB/s wr, 3.01k op/s 2023-09-12T16:18:41.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:41 smithi130 ceph-mon[98670]: pgmap v764: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 32 MiB/s wr, 3.01k op/s 2023-09-12T16:18:41.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:41 smithi203 ceph-mon[103431]: pgmap v764: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 32 MiB/s wr, 3.01k op/s 2023-09-12T16:18:43.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:43 smithi203 ceph-mon[103431]: pgmap v765: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 32 MiB/s wr, 3.23k op/s 2023-09-12T16:18:43.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:43 smithi159 ceph-mon[102522]: pgmap v765: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 32 MiB/s wr, 3.23k op/s 2023-09-12T16:18:43.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:43 smithi130 ceph-mon[98670]: pgmap v765: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 32 MiB/s wr, 3.23k op/s 2023-09-12T16:18:44.510 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:18:44.769 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.769+0000 7f49c3f20700 1 -- 172.21.15.130:0/3781647521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc1010f0 msgr2=0x7f49bc1079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:44.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.769+0000 7f49c3f20700 1 --2- 172.21.15.130:0/3781647521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc1010f0 0x7f49bc1079a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f49b4008830 tx=0x7f49b4008b40 comp rx=0 tx=0).stop 2023-09-12T16:18:44.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.769+0000 7f49c3f20700 1 -- 172.21.15.130:0/3781647521 shutdown_connections 2023-09-12T16:18:44.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.769+0000 7f49c3f20700 1 --2- 172.21.15.130:0/3781647521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc1010f0 0x7f49bc1079a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.770 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.769+0000 7f49c3f20700 1 --2- 172.21.15.130:0/3781647521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc0ff560 0x7f49bc0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.769+0000 7f49c3f20700 1 --2- 172.21.15.130:0/3781647521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49bc100750 0x7f49bc100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 -- 172.21.15.130:0/3781647521 >> 172.21.15.130:0/3781647521 conn(0x7f49bc0faad0 msgr2=0x7f49bc0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:44.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 -- 172.21.15.130:0/3781647521 shutdown_connections 2023-09-12T16:18:44.771 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 -- 172.21.15.130:0/3781647521 wait complete. 2023-09-12T16:18:44.773 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 Processor -- start 2023-09-12T16:18:44.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 -- start start 2023-09-12T16:18:44.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc0ff560 0x7f49bc1940b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49bc100750 0x7f49bc1945f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.774 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc1010f0 0x7f49bc19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc19bc30 con 0x7f49bc0ff560 2023-09-12T16:18:44.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc19bd70 con 0x7f49bc100750 2023-09-12T16:18:44.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.770+0000 7f49c3f20700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc19bf00 con 0x7f49bc1010f0 2023-09-12T16:18:44.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c1cbc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc0ff560 0x7f49bc1940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.775 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c1cbc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc0ff560 0x7f49bc1940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56794/0 (socket says 172.21.15.130:56794) 2023-09-12T16:18:44.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c1cbc700 1 -- 172.21.15.130:0/2623355589 learned_addr learned my addr 172.21.15.130:0/2623355589 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:44.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c14bb700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49bc100750 0x7f49bc1945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c24bd700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc1010f0 0x7f49bc19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c1cbc700 1 -- 172.21.15.130:0/2623355589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc1010f0 msgr2=0x7f49bc19b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:44.776 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c1cbc700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc1010f0 0x7f49bc19b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c1cbc700 1 -- 172.21.15.130:0/2623355589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49bc100750 msgr2=0x7f49bc1945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:44.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c1cbc700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49bc100750 0x7f49bc1945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c1cbc700 1 -- 172.21.15.130:0/2623355589 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac00b5a0 con 0x7f49bc0ff560 2023-09-12T16:18:44.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c1cbc700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc0ff560 0x7f49bc1940b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f49b800e9f0 tx=0x7f49b800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:44.777 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49b2ffd700 1 -- 172.21.15.130:0/2623355589 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8004220 con 0x7f49bc0ff560 2023-09-12T16:18:44.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c3f20700 1 -- 172.21.15.130:0/2623355589 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b4008430 con 0x7f49bc0ff560 2023-09-12T16:18:44.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49b2ffd700 1 -- 172.21.15.130:0/2623355589 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b8004380 con 0x7f49bc0ff560 2023-09-12T16:18:44.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49b2ffd700 1 -- 172.21.15.130:0/2623355589 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8003be0 con 0x7f49bc0ff560 2023-09-12T16:18:44.778 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.771+0000 7f49c3f20700 1 -- 172.21.15.130:0/2623355589 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc19c560 con 0x7f49bc0ff560 2023-09-12T16:18:44.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.774+0000 7f49b2ffd700 1 -- 172.21.15.130:0/2623355589 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f49b8003d40 con 0x7f49bc0ff560 2023-09-12T16:18:44.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.775+0000 7f49b2ffd700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f49a80a0690 0x7f49a80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.775+0000 7f49b2ffd700 1 -- 172.21.15.130:0/2623355589 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f49b8014070 con 0x7f49bc0ff560 2023-09-12T16:18:44.780 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.776+0000 7f49d00e6b80 1 Processor -- start 2023-09-12T16:18:44.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.777+0000 7f49d00e6b80 1 -- start start 2023-09-12T16:18:44.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.777+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63672090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.777+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c636725d0 0x564c63670850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.777+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c636712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.781 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.777+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c635b8730 con 0x564c63670e30 2023-09-12T16:18:44.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.777+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c635b6050 con 0x564c63671c70 2023-09-12T16:18:44.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.777+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c635b1940 con 0x564c636725d0 2023-09-12T16:18:44.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.779+0000 7f49c1cbc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63672090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.779+0000 7f49c1cbc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63672090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40058/0 (socket says 172.21.15.130:40058) 2023-09-12T16:18:44.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.779+0000 7f49c1cbc700 1 -- 172.21.15.130:0/4058009801 learned_addr learned my addr 172.21.15.130:0/4058009801 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:44.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.779+0000 7f49c14bb700 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c636725d0 0x564c63670850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.780+0000 7f49c24bd700 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c636712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.780+0000 7f49c14bb700 1 -- 172.21.15.130:0/4058009801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 msgr2=0x564c63672090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:44.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.780+0000 7f49c14bb700 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63672090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.783 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.780+0000 7f49c14bb700 1 -- 172.21.15.130:0/4058009801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 msgr2=0x564c636712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:44.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.780+0000 7f49c14bb700 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c636712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.780+0000 7f49c14bb700 1 -- 172.21.15.130:0/4058009801 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c633ddde0 con 0x564c636725d0 2023-09-12T16:18:44.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.780+0000 7f49c1cbc700 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63672090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:18:44.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.780+0000 7f49c14bb700 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c636725d0 0x564c63670850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00be50 tx=0x7f49ac014580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ccf2caf77e75e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:44.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.780+0000 7f49c14bb700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f49a80a0690 0x7f49a80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.785 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.781+0000 7f49a7fff700 1 -- 172.21.15.130:0/4058009801 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac023040 con 0x564c636725d0 2023-09-12T16:18:44.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.781+0000 7f49a7fff700 1 -- 172.21.15.130:0/4058009801 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49ac0178e0 con 0x564c636725d0 2023-09-12T16:18:44.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.781+0000 7f49a7fff700 1 -- 172.21.15.130:0/4058009801 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac010040 con 0x564c636725d0 2023-09-12T16:18:44.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.782+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4058009801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c636725d0 msgr2=0x564c63670850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:44.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.782+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c636725d0 0x564c63670850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00be50 tx=0x7f49ac014580 comp rx=0 tx=0).stop 2023-09-12T16:18:44.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.782+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4058009801 shutdown_connections 2023-09-12T16:18:44.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.782+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c636712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.782+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63672090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.782+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4058009801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c636725d0 0x564c63670850 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00be50 tx=0x7f49ac014580 comp rx=0 tx=0).stop 2023-09-12T16:18:44.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.782+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4058009801 >> 172.21.15.130:0/4058009801 conn(0x564c635d4090 msgr2=0x564c6365a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:44.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.784+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4058009801 shutdown_connections 2023-09-12T16:18:44.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.784+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4058009801 wait complete. 2023-09-12T16:18:44.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.784+0000 7f49c14bb700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f49a80a0690 0x7f49a80a2b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f49ac01e000 tx=0x7f49ac01fb10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:44.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.784+0000 7f49d00e6b80 1 -- start start 2023-09-12T16:18:44.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.784+0000 7f49d00e6b80 1 Processor -- start 2023-09-12T16:18:44.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c6367a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c63671c70 0x564c6367ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c6367fda0 0x564c63682190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6354fb70 con 0x564c63670e30 2023-09-12T16:18:44.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6354f870 con 0x564c6367fda0 2023-09-12T16:18:44.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63433920 con 0x564c63671c70 2023-09-12T16:18:44.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49d00e6b80 10 client.0 _ll_register_callbacks cb 0x564c63499a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:18:44.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49c24bd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c6367fda0 0x564c63682190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49c24bd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c6367fda0 0x564c63682190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40066/0 (socket says 172.21.15.130:40066) 2023-09-12T16:18:44.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49c24bd700 1 -- 172.21.15.130:0/4007140957 learned_addr learned my addr 172.21.15.130:0/4007140957 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:44.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.785+0000 7f49c14bb700 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c63671c70 0x564c6367ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49c24bd700 1 -- 172.21.15.130:0/4007140957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c63671c70 msgr2=0x564c6367ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:44.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49c24bd700 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c63671c70 0x564c6367ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49c24bd700 1 -- 172.21.15.130:0/4007140957 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 msgr2=0x564c6367a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:44.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49c24bd700 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c6367a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49c24bd700 1 -- 172.21.15.130:0/4007140957 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc19c560 con 0x564c6367fda0 2023-09-12T16:18:44.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49c24bd700 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c6367fda0 0x564c63682190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f49b4008ed0 tx=0x7f49b400bba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21bdf50ff596f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:44.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:18:44.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49a57fa700 1 -- 172.21.15.130:0/4007140957 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4027070 con 0x564c6367fda0 2023-09-12T16:18:44.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49a57fa700 1 -- 172.21.15.130:0/4007140957 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f49b4016b20 con 0x564c6367fda0 2023-09-12T16:18:44.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4007140957 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c635094f0 con 0x564c6367fda0 2023-09-12T16:18:44.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49a57fa700 1 -- 172.21.15.130:0/4007140957 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b4003a00 con 0x564c6367fda0 2023-09-12T16:18:44.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49d00e6b80 10 client.25700 fetch_fsmap learned FSMap version 32 2023-09-12T16:18:44.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.786+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4007140957 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f49ac00ba80 con 0x564c6367fda0 2023-09-12T16:18:44.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.787+0000 7f49a57fa700 1 -- 172.21.15.130:0/4007140957 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c635094f0 con 0x564c6367fda0 2023-09-12T16:18:44.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.787+0000 7f49a57fa700 1 -- 172.21.15.130:0/4007140957 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f49b4009580 con 0x564c6367fda0 2023-09-12T16:18:44.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.787+0000 7f49d00e6b80 10 client.25700 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:18:44.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.787+0000 7f49d00e6b80 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:18:44.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.787+0000 7f49d00e6b80 20 client.25700 populate_metadata read hostname 'smithi130' 2023-09-12T16:18:44.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.787+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x564c63686a60 0x564c63688e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.787+0000 7f49d00e6b80 4 client.25700 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:18:44.796 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.787+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4007140957 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c635c68a0 con 0x564c63686a60 2023-09-12T16:18:44.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.787+0000 7f49c1cbc700 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x564c63686a60 0x564c63688e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.788+0000 7f49a57fa700 10 client.25700 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:18:44.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.788+0000 7f49c1cbc700 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x564c63686a60 0x564c63688e80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f882e29ad003b9 server_cookie=cbf1a4041acad6d7 in_seq=0 out_seq=0 2023-09-12T16:18:44.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.788+0000 7f49a57fa700 1 -- 172.21.15.130:0/4007140957 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c635c68a0 con 0x564c63686a60 2023-09-12T16:18:44.797 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.788+0000 7f49a57fa700 10 client.25700 handle_command_reply: tid=0 2023-09-12T16:18:44.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.788+0000 7f49d00e6b80 1 client.25700 shutdown 2023-09-12T16:18:44.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.788+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4007140957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c6367fda0 msgr2=0x564c63682190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:44.798 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c6367fda0 0x564c63682190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f49b4008ed0 tx=0x7f49b400bba0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4007140957 shutdown_connections 2023-09-12T16:18:44.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x564c63686a60 0x564c63688e80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.799 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c6367fda0 0x564c63682190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49a57fa700 0 client.25700 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:18:44.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c6367a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/4007140957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c63671c70 0x564c6367ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4007140957 >> 172.21.15.130:0/4007140957 conn(0x564c635d4090 msgr2=0x564c63659dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:44.800 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4007140957 shutdown_connections 2023-09-12T16:18:44.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 1 -- 172.21.15.130:0/4007140957 wait complete. 2023-09-12T16:18:44.801 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.789+0000 7f49d00e6b80 20 client.25700 trim_cache size 0 max 16384 2023-09-12T16:18:44.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.818+0000 7f49d00e6b80 1 Processor -- start 2023-09-12T16:18:44.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49d00e6b80 1 -- start start 2023-09-12T16:18:44.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c63679ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63659dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 0x564c63696640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.820 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63671710 con 0x564c63670e30 2023-09-12T16:18:44.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63679cb0 con 0x564c63671c70 2023-09-12T16:18:44.821 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63624850 con 0x564c6367fda0 2023-09-12T16:18:44.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49c14bb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63659dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49c14bb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63659dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40074/0 (socket says 172.21.15.130:40074) 2023-09-12T16:18:44.822 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.820+0000 7f49c14bb700 1 -- 172.21.15.130:0/1300648244 learned_addr learned my addr 172.21.15.130:0/1300648244 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:44.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.821+0000 7f49c14bb700 1 -- 172.21.15.130:0/1300648244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 msgr2=0x564c63696640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:18:44.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.821+0000 7f49c14bb700 1 --2- 172.21.15.130:0/1300648244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 0x564c63696640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.821+0000 7f49c14bb700 1 -- 172.21.15.130:0/1300648244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 msgr2=0x564c63679ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:18:44.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.821+0000 7f49c14bb700 1 --2- 172.21.15.130:0/1300648244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c63679ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.823 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.821+0000 7f49c14bb700 1 -- 172.21.15.130:0/1300648244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c633c6c60 con 0x564c63671c70 2023-09-12T16:18:44.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.821+0000 7f49c14bb700 1 --2- 172.21.15.130:0/1300648244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63659dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00be50 tx=0x7f49ac00d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97eeda8aa4e2b492 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:44.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.821+0000 7f49b0ff9700 1 -- 172.21.15.130:0/1300648244 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac030070 con 0x564c63671c70 2023-09-12T16:18:44.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.821+0000 7f49b0ff9700 1 -- 172.21.15.130:0/1300648244 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49ac01fce0 con 0x564c63671c70 2023-09-12T16:18:44.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.821+0000 7f49b0ff9700 1 -- 172.21.15.130:0/1300648244 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac023040 con 0x564c63671c70 2023-09-12T16:18:44.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.823+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1300648244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 msgr2=0x564c63659dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:44.824 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.823+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1300648244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63659dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00be50 tx=0x7f49ac00d000 comp rx=0 tx=0).stop 2023-09-12T16:18:44.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.825+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1300648244 shutdown_connections 2023-09-12T16:18:44.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.825+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1300648244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 0x564c63696640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.825+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1300648244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63670e30 0x564c63679ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.826 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.825+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1300648244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63671c70 0x564c63659dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.825+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1300648244 >> 172.21.15.130:0/1300648244 conn(0x564c635d4090 msgr2=0x564c6368d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:44.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.825+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1300648244 shutdown_connections 2023-09-12T16:18:44.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.825+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1300648244 wait complete. 2023-09-12T16:18:44.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.825+0000 7f49d00e6b80 1 -- start start 2023-09-12T16:18:44.827 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.825+0000 7f49d00e6b80 1 Processor -- start 2023-09-12T16:18:44.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63670e30 0x564c636945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63671c70 0x564c63694ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49d00e6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 0x564c63695af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63696030 con 0x564c63671c70 2023-09-12T16:18:44.828 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c636961a0 con 0x564c63670e30 2023-09-12T16:18:44.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49d00e6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63696360 con 0x564c6367fda0 2023-09-12T16:18:44.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49d00e6b80 10 client.0 _ll_register_callbacks cb 0x7f49b4008650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:18:44.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49c1cbc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63670e30 0x564c636945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49c1cbc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63670e30 0x564c636945a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40076/0 (socket says 172.21.15.130:40076) 2023-09-12T16:18:44.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49c1cbc700 1 -- 172.21.15.130:0/1962737715 learned_addr learned my addr 172.21.15.130:0/1962737715 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:18:44.829 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.826+0000 7f49c24bd700 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 0x564c63695af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.827+0000 7f49c24bd700 1 -- 172.21.15.130:0/1962737715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63670e30 msgr2=0x564c636945a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:44.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.827+0000 7f49c24bd700 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63670e30 0x564c636945a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.827+0000 7f49c24bd700 1 -- 172.21.15.130:0/1962737715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63671c70 msgr2=0x564c63694ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:18:44.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.827+0000 7f49c24bd700 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63671c70 0x564c63694ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.830 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.827+0000 7f49c24bd700 1 -- 172.21.15.130:0/1962737715 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c633ddde0 con 0x564c6367fda0 2023-09-12T16:18:44.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.827+0000 7f49c24bd700 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 0x564c63695af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x564c636863e0 tx=0x7f49b400dcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec08c5bcb7e0491 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:18:44.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.828+0000 7f49a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:18:44.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.828+0000 7f49a57fa700 1 -- 172.21.15.130:0/1962737715 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4027040 con 0x564c6367fda0 2023-09-12T16:18:44.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.828+0000 7f49a57fa700 1 -- 172.21.15.130:0/1962737715 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f49b4009580 con 0x564c6367fda0 2023-09-12T16:18:44.833 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.828+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1962737715 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49ac0178e0 con 0x564c6367fda0 2023-09-12T16:18:44.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.829+0000 7f49a57fa700 1 -- 172.21.15.130:0/1962737715 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b4009740 con 0x564c6367fda0 2023-09-12T16:18:44.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.829+0000 7f49d00e6b80 10 client.25903 fetch_fsmap learned FSMap version 32 2023-09-12T16:18:44.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.829+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1962737715 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564c63404250 con 0x564c6367fda0 2023-09-12T16:18:44.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.829+0000 7f49a57fa700 1 -- 172.21.15.130:0/1962737715 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49ac0178e0 con 0x564c6367fda0 2023-09-12T16:18:44.834 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.830+0000 7f49a57fa700 1 -- 172.21.15.130:0/1962737715 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f49b400d540 con 0x564c6367fda0 2023-09-12T16:18:44.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.830+0000 7f49d00e6b80 10 client.25903 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:18:44.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.830+0000 7f49d00e6b80 10 client.25903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:18:44.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.830+0000 7f49d00e6b80 20 client.25903 populate_metadata read hostname 'smithi130' 2023-09-12T16:18:44.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.830+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x564c636dbb00 0x564c636ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:18:44.835 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.830+0000 7f49d00e6b80 4 client.25903 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:18:44.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.830+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1962737715 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c635aa920 con 0x564c636dbb00 2023-09-12T16:18:44.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.830+0000 7f49c1cbc700 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x564c636dbb00 0x564c636ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:18:44.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.831+0000 7f49a57fa700 10 client.25903 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:18:44.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.831+0000 7f49c1cbc700 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x564c636dbb00 0x564c636ddf20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2804301db5d5ad5f server_cookie=3692f9867c2cafa8 in_seq=0 out_seq=0 2023-09-12T16:18:44.836 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.832+0000 7f49a57fa700 1 -- 172.21.15.130:0/1962737715 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564c635aa920 con 0x564c636dbb00 2023-09-12T16:18:44.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.832+0000 7f49a57fa700 10 client.25903 handle_command_reply: tid=0 2023-09-12T16:18:44.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.832+0000 7f49d00e6b80 1 client.25903 shutdown 2023-09-12T16:18:44.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1962737715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 msgr2=0x564c63695af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:44.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 0x564c63695af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x564c636863e0 tx=0x7f49b400dcb0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1962737715 shutdown_connections 2023-09-12T16:18:44.837 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x564c636dbb00 0x564c636ddf20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c6367fda0 0x564c63695af0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49a57fa700 0 client.25903 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:18:44.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c63670e30 0x564c636945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 --2- 172.21.15.130:0/1962737715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c63671c70 0x564c63694ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.838 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1962737715 >> 172.21.15.130:0/1962737715 conn(0x564c635d4090 msgr2=0x564c63678640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:44.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1962737715 shutdown_connections 2023-09-12T16:18:44.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 1 -- 172.21.15.130:0/1962737715 wait complete. 2023-09-12T16:18:44.839 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.833+0000 7f49d00e6b80 20 client.25903 trim_cache size 0 max 16384 2023-09-12T16:18:44.840 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:18:44.840 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:18:44.840 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:18:44.840 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:18:44.840 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:18:44.841 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:18:44.841 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:18:44.841 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:18:44.841 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:18:44.841 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:18:44.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.834+0000 7f49c3f20700 1 -- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f49a80a0690 msgr2=0x7f49a80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:44.842 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.834+0000 7f49c3f20700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f49a80a0690 0x7f49a80a2b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f49ac01e000 tx=0x7f49ac01fb10 comp rx=0 tx=0).stop 2023-09-12T16:18:44.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 -- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc0ff560 msgr2=0x7f49bc1940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:18:44.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc0ff560 0x7f49bc1940b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f49b800e9f0 tx=0x7f49b800ed00 comp rx=0 tx=0).stop 2023-09-12T16:18:44.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 -- 172.21.15.130:0/2623355589 shutdown_connections 2023-09-12T16:18:44.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f49a80a0690 0x7f49a80a2b50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc1010f0 0x7f49bc19b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.843 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f49bc0ff560 0x7f49bc1940b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.844 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 --2- 172.21.15.130:0/2623355589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49bc100750 0x7f49bc1945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:18:44.844 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 -- 172.21.15.130:0/2623355589 >> 172.21.15.130:0/2623355589 conn(0x7f49bc0faad0 msgr2=0x7f49bc103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:18:44.844 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 -- 172.21.15.130:0/2623355589 shutdown_connections 2023-09-12T16:18:44.844 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:18:44.835+0000 7f49c3f20700 1 -- 172.21.15.130:0/2623355589 wait complete. 2023-09-12T16:18:44.853 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:18:45.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 775.0 s, 662.4 tps, lat 151.223 ms stddev 508.337 2023-09-12T16:18:45.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:45 smithi130 ceph-mon[98670]: pgmap v766: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 30 MiB/s wr, 3.27k op/s 2023-09-12T16:18:45.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:45 smithi159 ceph-mon[102522]: pgmap v766: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 30 MiB/s wr, 3.27k op/s 2023-09-12T16:18:45.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:45 smithi203 ceph-mon[103431]: pgmap v766: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 30 MiB/s wr, 3.27k op/s 2023-09-12T16:18:47.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:47 smithi130 ceph-mon[98670]: pgmap v767: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 29 MiB/s wr, 3.08k op/s 2023-09-12T16:18:47.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:47 smithi159 ceph-mon[102522]: pgmap v767: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 29 MiB/s wr, 3.08k op/s 2023-09-12T16:18:47.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:47 smithi203 ceph-mon[103431]: pgmap v767: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 29 MiB/s wr, 3.08k op/s 2023-09-12T16:18:48.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:48 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:18:48.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:48 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:18:48.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:48 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:18:49.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:49 smithi130 ceph-mon[98670]: pgmap v768: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 2.68k op/s 2023-09-12T16:18:49.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:18:49.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:49 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:18:49.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:49 smithi159 ceph-mon[102522]: pgmap v768: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 2.68k op/s 2023-09-12T16:18:49.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:18:49.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:49 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:18:49.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:49 smithi203 ceph-mon[103431]: pgmap v768: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 2.68k op/s 2023-09-12T16:18:49.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:18:49.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:49 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:18:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 780.0 s, 1208.4 tps, lat 83.522 ms stddev 117.303 2023-09-12T16:18:50.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:18:50.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:50 smithi130 ceph-mon[98670]: pgmap v769: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 29 MiB/s wr, 2.90k op/s 2023-09-12T16:18:50.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:18:50.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:50 smithi159 ceph-mon[102522]: pgmap v769: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 29 MiB/s wr, 2.90k op/s 2023-09-12T16:18:50.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:18:50.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:50 smithi203 ceph-mon[103431]: pgmap v769: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 29 MiB/s wr, 2.90k op/s 2023-09-12T16:18:53.725 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:53 smithi159 ceph-mon[102522]: pgmap v770: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-09-12T16:18:53.731 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:53 smithi130 ceph-mon[98670]: pgmap v770: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-09-12T16:18:53.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:53 smithi203 ceph-mon[103431]: pgmap v770: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-09-12T16:18:55.094 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 785.0 s, 1764.4 tps, lat 56.298 ms stddev 43.763 2023-09-12T16:18:55.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:55 smithi159 ceph-mon[102522]: pgmap v771: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-09-12T16:18:55.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:55 smithi203 ceph-mon[103431]: pgmap v771: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-09-12T16:18:55.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:55 smithi130 ceph-mon[98670]: pgmap v771: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-09-12T16:18:57.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:57 smithi130 ceph-mon[98670]: pgmap v772: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 20 MiB/s wr, 1.32k op/s 2023-09-12T16:18:57.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:57 smithi159 ceph-mon[102522]: pgmap v772: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 20 MiB/s wr, 1.32k op/s 2023-09-12T16:18:57.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:57 smithi203 ceph-mon[103431]: pgmap v772: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 20 MiB/s wr, 1.32k op/s 2023-09-12T16:18:59.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:18:59 smithi159 ceph-mon[102522]: pgmap v773: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-09-12T16:18:59.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:18:59 smithi130 ceph-mon[98670]: pgmap v773: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-09-12T16:18:59.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:18:59 smithi203 ceph-mon[103431]: pgmap v773: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-09-12T16:19:00.094 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 790.0 s, 755.8 tps, lat 132.590 ms stddev 182.186 2023-09-12T16:19:01.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:01 smithi130 ceph-mon[98670]: pgmap v774: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 1.31k op/s 2023-09-12T16:19:01.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:01 smithi159 ceph-mon[102522]: pgmap v774: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 1.31k op/s 2023-09-12T16:19:01.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:01 smithi203 ceph-mon[103431]: pgmap v774: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 1.31k op/s 2023-09-12T16:19:03.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:03 smithi159 ceph-mon[102522]: pgmap v775: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2023-09-12T16:19:03.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:03 smithi130 ceph-mon[98670]: pgmap v775: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2023-09-12T16:19:03.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:03 smithi203 ceph-mon[103431]: pgmap v775: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2023-09-12T16:19:05.095 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 795.0 s, 1384.1 tps, lat 71.827 ms stddev 51.103 2023-09-12T16:19:05.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:05 smithi130 ceph-mon[98670]: pgmap v776: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 998 op/s 2023-09-12T16:19:05.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:05 smithi159 ceph-mon[102522]: pgmap v776: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 998 op/s 2023-09-12T16:19:05.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:05 smithi203 ceph-mon[103431]: pgmap v776: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 998 op/s 2023-09-12T16:19:06.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:06 smithi159 ceph-mon[102522]: pgmap v777: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.09k op/s 2023-09-12T16:19:06.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:06 smithi130 ceph-mon[98670]: pgmap v777: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.09k op/s 2023-09-12T16:19:06.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:06 smithi203 ceph-mon[103431]: pgmap v777: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.09k op/s 2023-09-12T16:19:09.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:09 smithi159 ceph-mon[102522]: pgmap v778: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 874 op/s 2023-09-12T16:19:09.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:09 smithi130 ceph-mon[98670]: pgmap v778: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 874 op/s 2023-09-12T16:19:09.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:09 smithi203 ceph-mon[103431]: pgmap v778: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 874 op/s 2023-09-12T16:19:10.091 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 800.0 s, 1382.5 tps, lat 72.337 ms stddev 87.054 2023-09-12T16:19:11.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:11 smithi130 ceph-mon[98670]: pgmap v779: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 1.25k op/s 2023-09-12T16:19:11.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:11 smithi159 ceph-mon[102522]: pgmap v779: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 1.25k op/s 2023-09-12T16:19:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:11 smithi203 ceph-mon[103431]: pgmap v779: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 1.25k op/s 2023-09-12T16:19:13.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:13 smithi159 ceph-mon[102522]: pgmap v780: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-09-12T16:19:13.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:13 smithi130 ceph-mon[98670]: pgmap v780: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-09-12T16:19:13.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:13 smithi203 ceph-mon[103431]: pgmap v780: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-09-12T16:19:14.854 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-12T16:19:14.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:14 smithi159 ceph-mon[102522]: pgmap v781: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-09-12T16:19:14.980 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:14 smithi130 ceph-mon[98670]: pgmap v781: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-09-12T16:19:14.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:14 smithi203 ceph-mon[103431]: pgmap v781: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-09-12T16:19:15.095 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 805.0 s, 354.0 tps, lat 216.246 ms stddev 446.221 2023-09-12T16:19:15.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.238+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1501251278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b8109f70 msgr2=0x7ff4b810c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:19:15.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.238+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1501251278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b8109f70 0x7ff4b810c400 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff4a801a9a0 tx=0x7ff4a801acb0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.238+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1501251278 shutdown_connections 2023-09-12T16:19:15.239 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.238+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1501251278 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff4b810c940 0x7ff4b810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.238+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1501251278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b8107600 0x7ff4b8109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.238+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1501251278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b8109f70 0x7ff4b810c400 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.238+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1501251278 >> 172.21.15.130:0/1501251278 conn(0x7ff4b806d3b0 msgr2=0x7ff4b806f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:19:15.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.238+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1501251278 shutdown_connections 2023-09-12T16:19:15.240 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.238+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1501251278 wait complete. 2023-09-12T16:19:15.241 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.239+0000 7ff4bd5bc700 1 Processor -- start 2023-09-12T16:19:15.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.240+0000 7ff4bd5bc700 1 -- start start 2023-09-12T16:19:15.242 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4bd5bc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff4b8107600 0x7ff4b819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4bd5bc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b8109f70 0x7ff4b819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4bd5bc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b810c940 0x7ff4b81a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4bd5bc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81a4460 con 0x7ff4b8107600 2023-09-12T16:19:15.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4bd5bc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81a45a0 con 0x7ff4b8109f70 2023-09-12T16:19:15.243 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4bd5bc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81a4730 con 0x7ff4b810c940 2023-09-12T16:19:15.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b6ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff4b8107600 0x7ff4b819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b6ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff4b8107600 0x7ff4b819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56832/0 (socket says 172.21.15.130:56832) 2023-09-12T16:19:15.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b6ffd700 1 -- 172.21.15.130:0/1736127783 learned_addr learned my addr 172.21.15.130:0/1736127783 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:19:15.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b810c940 0x7ff4b81a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b67fc700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b8109f70 0x7ff4b819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.244 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b77fe700 1 -- 172.21.15.130:0/1736127783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b8109f70 msgr2=0x7ff4b819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:19:15.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b8109f70 0x7ff4b819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b77fe700 1 -- 172.21.15.130:0/1736127783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff4b8107600 msgr2=0x7ff4b819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:19:15.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff4b8107600 0x7ff4b819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.241+0000 7ff4b77fe700 1 -- 172.21.15.130:0/1736127783 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a801a600 con 0x7ff4b810c940 2023-09-12T16:19:15.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.242+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b810c940 0x7ff4b81a3e90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0007f10 tx=0x7ff4b000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:19:15.245 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.242+0000 7ff49ffff700 1 -- 172.21.15.130:0/1736127783 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b000ddd0 con 0x7ff4b810c940 2023-09-12T16:19:15.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.242+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1736127783 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b81a4a10 con 0x7ff4b810c940 2023-09-12T16:19:15.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.242+0000 7ff49ffff700 1 -- 172.21.15.130:0/1736127783 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4b000f040 con 0x7ff4b810c940 2023-09-12T16:19:15.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.242+0000 7ff49ffff700 1 -- 172.21.15.130:0/1736127783 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0015400 con 0x7ff4b810c940 2023-09-12T16:19:15.246 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.242+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1736127783 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b81a4f20 con 0x7ff4b810c940 2023-09-12T16:19:15.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.247+0000 7ff4c9782b80 1 Processor -- start 2023-09-12T16:19:15.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.247+0000 7ff49ffff700 1 -- 172.21.15.130:0/1736127783 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7ff4b0004c40 con 0x7ff4b810c940 2023-09-12T16:19:15.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.248+0000 7ff49ffff700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4a00a0790 0x7ff4a00a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.249 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.248+0000 7ff49ffff700 1 -- 172.21.15.130:0/1736127783 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff4b00cb840 con 0x7ff4b810c940 2023-09-12T16:19:15.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.253+0000 7ff4c9782b80 1 -- start start 2023-09-12T16:19:15.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.254+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c2644020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.256 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.254+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c26445a0 0x55a3c2642800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.254+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c26432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.254+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c259b730 con 0x55a3c26445a0 2023-09-12T16:19:15.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.254+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2599050 con 0x55a3c2643c00 2023-09-12T16:19:15.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.254+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2594940 con 0x55a3c2642e00 2023-09-12T16:19:15.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.254+0000 7ff4b77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c26432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.257 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.254+0000 7ff4b77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c26432b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36454/0 (socket says 172.21.15.130:36454) 2023-09-12T16:19:15.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.254+0000 7ff4b77fe700 1 -- 172.21.15.130:0/3544166972 learned_addr learned my addr 172.21.15.130:0/3544166972 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:19:15.258 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.255+0000 7ff4b6ffd700 1 --2- 172.21.15.130:0/3544166972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c2644020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.259+0000 7ff4b67fc700 1 --2- 172.21.15.130:0/3544166972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c26445a0 0x55a3c2642800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.259+0000 7ff4b67fc700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4a00a0790 0x7ff4a00a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.260 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.259+0000 7ff4b77fe700 1 -- 172.21.15.130:0/3544166972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 msgr2=0x55a3c2644020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:19:15.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.259+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/3544166972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c2644020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.259+0000 7ff4b77fe700 1 -- 172.21.15.130:0/3544166972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c26445a0 msgr2=0x55a3c2642800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:19:15.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.259+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/3544166972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c26445a0 0x55a3c2642800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.261 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.259+0000 7ff4b77fe700 1 -- 172.21.15.130:0/3544166972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3c23c0de0 con 0x55a3c2642e00 2023-09-12T16:19:15.264 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.259+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/3544166972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c26432b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0098960 tx=0x7ff4b0098870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f82f3729e6e440da server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:19:15.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.262+0000 7ff49d7fa700 1 -- 172.21.15.130:0/3544166972 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0098c60 con 0x55a3c2642e00 2023-09-12T16:19:15.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.262+0000 7ff49d7fa700 1 -- 172.21.15.130:0/3544166972 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4b0094fa0 con 0x55a3c2642e00 2023-09-12T16:19:15.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.262+0000 7ff49d7fa700 1 -- 172.21.15.130:0/3544166972 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0099100 con 0x55a3c2642e00 2023-09-12T16:19:15.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.267+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3544166972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 msgr2=0x55a3c26432b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:19:15.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.267+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3544166972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c26432b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0098960 tx=0x7ff4b0098870 comp rx=0 tx=0).stop 2023-09-12T16:19:15.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.267+0000 7ff4b67fc700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4a00a0790 0x7ff4a00a2c50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006010 tx=0x7ff4a801acb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:19:15.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.278+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3544166972 shutdown_connections 2023-09-12T16:19:15.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.278+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3544166972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c26432b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.278+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3544166972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c2644020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.278+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3544166972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c26445a0 0x55a3c2642800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.278+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3544166972 >> 172.21.15.130:0/3544166972 conn(0x55a3c25b7090 msgr2=0x55a3c263d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:19:15.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.285+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3544166972 shutdown_connections 2023-09-12T16:19:15.286 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.285+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3544166972 wait complete. 2023-09-12T16:19:15.293 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.286+0000 7ff4c9782b80 1 -- start start 2023-09-12T16:19:15.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.286+0000 7ff4c9782b80 1 Processor -- start 2023-09-12T16:19:15.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c264c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c264cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c26445a0 0x55a3c2653b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.294 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2532b70 con 0x55a3c26445a0 2023-09-12T16:19:15.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2532870 con 0x55a3c2643c00 2023-09-12T16:19:15.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2416920 con 0x55a3c2642e00 2023-09-12T16:19:15.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4c9782b80 10 client.0 _ll_register_callbacks cb 0x55a3c247ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:19:15.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4b67fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c264cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.295 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4b67fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c264cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40106/0 (socket says 172.21.15.130:40106) 2023-09-12T16:19:15.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.287+0000 7ff4b67fc700 1 -- 172.21.15.130:0/3798112642 learned_addr learned my addr 172.21.15.130:0/3798112642 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:19:15.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.288+0000 7ff4b6ffd700 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c264c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.288+0000 7ff4b67fc700 1 -- 172.21.15.130:0/3798112642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 msgr2=0x55a3c264c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:19:15.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.288+0000 7ff4b67fc700 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c264c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.296 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.288+0000 7ff4b67fc700 1 -- 172.21.15.130:0/3798112642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c26445a0 msgr2=0x55a3c2653b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:19:15.297 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.288+0000 7ff4b67fc700 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c26445a0 0x55a3c2653b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.297 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.288+0000 7ff4b67fc700 1 -- 172.21.15.130:0/3798112642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b81a4f20 con 0x55a3c2643c00 2023-09-12T16:19:15.297 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.288+0000 7ff4b67fc700 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c264cb00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4a801a9a0 tx=0x7ff4a801a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e35528ea4e1212 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:19:15.298 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.291+0000 7ff49aff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-09-12T16:19:15.298 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.291+0000 7ff49aff5700 1 -- 172.21.15.130:0/3798112642 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a802d030 con 0x55a3c2643c00 2023-09-12T16:19:15.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.291+0000 7ff49aff5700 1 -- 172.21.15.130:0/3798112642 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff4a8003e40 con 0x55a3c2643c00 2023-09-12T16:19:15.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.291+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3798112642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3c24ec4f0 con 0x55a3c2643c00 2023-09-12T16:19:15.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.291+0000 7ff49aff5700 1 -- 172.21.15.130:0/3798112642 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4a8040400 con 0x55a3c2643c00 2023-09-12T16:19:15.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.294+0000 7ff4c9782b80 10 client.25718 fetch_fsmap learned FSMap version 32 2023-09-12T16:19:15.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.294+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3798112642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a3c23a9c60 con 0x55a3c2643c00 2023-09-12T16:19:15.299 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.295+0000 7ff49aff5700 1 -- 172.21.15.130:0/3798112642 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3c24ec4f0 con 0x55a3c2643c00 2023-09-12T16:19:15.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.295+0000 7ff49aff5700 1 -- 172.21.15.130:0/3798112642 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff4a8048070 con 0x55a3c2643c00 2023-09-12T16:19:15.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.295+0000 7ff4c9782b80 10 client.25718 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:19:15.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.295+0000 7ff4c9782b80 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:19:15.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.295+0000 7ff4c9782b80 20 client.25718 populate_metadata read hostname 'smithi130' 2023-09-12T16:19:15.300 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.295+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a3c26583a0 0x55a3c265a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.295+0000 7ff4c9782b80 4 client.25718 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T16:19:15.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.295+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3798112642 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3c25a98a0 con 0x55a3c26583a0 2023-09-12T16:19:15.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.296+0000 7ff4b6ffd700 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a3c26583a0 0x55a3c265a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.298+0000 7ff4b6ffd700 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a3c26583a0 0x55a3c265a7c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349898afca3eec20 server_cookie=1b5598d333f2c554 in_seq=0 out_seq=0 2023-09-12T16:19:15.301 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.298+0000 7ff49aff5700 10 client.25718 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:19:15.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.299+0000 7ff49aff5700 1 -- 172.21.15.130:0/3798112642 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a3c25a98a0 con 0x55a3c26583a0 2023-09-12T16:19:15.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.299+0000 7ff49aff5700 10 client.25718 handle_command_reply: tid=0 2023-09-12T16:19:15.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.299+0000 7ff4c9782b80 1 client.25718 shutdown 2023-09-12T16:19:15.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.299+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3798112642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 msgr2=0x55a3c264cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:19:15.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.299+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c264cb00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4a801a9a0 tx=0x7ff4a801a320 comp rx=0 tx=0).stop 2023-09-12T16:19:15.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.299+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3798112642 shutdown_connections 2023-09-12T16:19:15.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.299+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a3c26583a0 0x55a3c265a7c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.300+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c26445a0 0x55a3c2653b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.300+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c264c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.300+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3798112642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c264cb00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.300+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3798112642 >> 172.21.15.130:0/3798112642 conn(0x55a3c25b7090 msgr2=0x55a3c263cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:19:15.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.300+0000 7ff49aff5700 0 client.25718 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:19:15.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.300+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3798112642 shutdown_connections 2023-09-12T16:19:15.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.300+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3798112642 wait complete. 2023-09-12T16:19:15.305 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.300+0000 7ff4c9782b80 20 client.25718 trim_cache size 0 max 16384 2023-09-12T16:19:15.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.331+0000 7ff4c9782b80 1 Processor -- start 2023-09-12T16:19:15.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.335+0000 7ff4c9782b80 1 -- start start 2023-09-12T16:19:15.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.335+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c2642710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.335+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c2643c00 0x55a3c2653a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.335+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c26445a0 0x55a3c265a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.335+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c25a98a0 con 0x55a3c2643c00 2023-09-12T16:19:15.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.335+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c24ec4f0 con 0x55a3c26445a0 2023-09-12T16:19:15.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.335+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c264d7b0 con 0x55a3c2642e00 2023-09-12T16:19:15.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.339+0000 7ff4b6ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c2642710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.339+0000 7ff4b6ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c2642710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36466/0 (socket says 172.21.15.130:36466) 2023-09-12T16:19:15.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.339+0000 7ff4b6ffd700 1 -- 172.21.15.130:0/3273140945 learned_addr learned my addr 172.21.15.130:0/3273140945 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:19:15.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.340+0000 7ff4b6ffd700 1 -- 172.21.15.130:0/3273140945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c26445a0 msgr2=0x55a3c265a3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:19:15.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.340+0000 7ff4b6ffd700 1 --2- 172.21.15.130:0/3273140945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c26445a0 0x55a3c265a3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.340+0000 7ff4b6ffd700 1 -- 172.21.15.130:0/3273140945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c2643c00 msgr2=0x55a3c2653a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-12T16:19:15.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.340+0000 7ff4b6ffd700 1 --2- 172.21.15.130:0/3273140945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c2643c00 0x55a3c2653a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.340+0000 7ff4b6ffd700 1 -- 172.21.15.130:0/3273140945 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3c26434b0 con 0x55a3c2642e00 2023-09-12T16:19:15.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.341+0000 7ff4b6ffd700 1 --2- 172.21.15.130:0/3273140945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c2642710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009ed0 tx=0x7ff4ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e836b7bf220b6a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:19:15.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.341+0000 7ff49dffb700 1 -- 172.21.15.130:0/3273140945 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac01f270 con 0x55a3c2642e00 2023-09-12T16:19:15.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.341+0000 7ff49dffb700 1 -- 172.21.15.130:0/3273140945 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ac018040 con 0x55a3c2642e00 2023-09-12T16:19:15.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.341+0000 7ff49dffb700 1 -- 172.21.15.130:0/3273140945 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac011860 con 0x55a3c2642e00 2023-09-12T16:19:15.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.342+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3273140945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 msgr2=0x55a3c2642710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:19:15.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.342+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3273140945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c2642710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009ed0 tx=0x7ff4ac009260 comp rx=0 tx=0).stop 2023-09-12T16:19:15.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.343+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3273140945 shutdown_connections 2023-09-12T16:19:15.348 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.343+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3273140945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c26445a0 0x55a3c265a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.343+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3273140945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c2642e00 0x55a3c2642710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.343+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/3273140945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c2643c00 0x55a3c2653a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.343+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3273140945 >> 172.21.15.130:0/3273140945 conn(0x55a3c25b7090 msgr2=0x55a3c2654430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:19:15.349 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.343+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3273140945 shutdown_connections 2023-09-12T16:19:15.350 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.343+0000 7ff4c9782b80 1 -- 172.21.15.130:0/3273140945 wait complete. 2023-09-12T16:19:15.350 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.344+0000 7ff4c9782b80 1 -- start start 2023-09-12T16:19:15.350 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.344+0000 7ff4c9782b80 1 Processor -- start 2023-09-12T16:19:15.351 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.344+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c2642e00 0x55a3c26533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.351 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.344+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c2653900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.351 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.344+0000 7ff4c9782b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c26445a0 0x55a3c264fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.351 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.344+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2650530 con 0x55a3c2642e00 2023-09-12T16:19:15.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.344+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c26506a0 con 0x55a3c2643c00 2023-09-12T16:19:15.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.344+0000 7ff4c9782b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2650860 con 0x55a3c26445a0 2023-09-12T16:19:15.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.344+0000 7ff4c9782b80 10 client.0 _ll_register_callbacks cb 0x7ff4a801ca60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T16:19:15.352 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.346+0000 7ff4b77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c26445a0 0x55a3c264fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.346+0000 7ff4b67fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c2653900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.346+0000 7ff4b77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c26445a0 0x55a3c264fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36470/0 (socket says 172.21.15.130:36470) 2023-09-12T16:19:15.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.346+0000 7ff4b77fe700 1 -- 172.21.15.130:0/2671755554 learned_addr learned my addr 172.21.15.130:0/2671755554 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:19:15.353 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.346+0000 7ff4b77fe700 1 -- 172.21.15.130:0/2671755554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 msgr2=0x55a3c2653900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:19:15.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.346+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c2653900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.346+0000 7ff4b77fe700 1 -- 172.21.15.130:0/2671755554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c2642e00 msgr2=0x55a3c26533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T16:19:15.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.346+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c2642e00 0x55a3c26533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.346+0000 7ff4b77fe700 1 -- 172.21.15.130:0/2671755554 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ac009ab0 con 0x55a3c26445a0 2023-09-12T16:19:15.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.347+0000 7ff4b77fe700 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c26445a0 0x55a3c264fe60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00b11d0 tx=0x7ff4b009ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c2bd8bb34ada5f server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:19:15.354 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.348+0000 7ff49aff5700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-09-12T16:19:15.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.348+0000 7ff49aff5700 1 -- 172.21.15.130:0/2671755554 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00a0cd0 con 0x55a3c26445a0 2023-09-12T16:19:15.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.348+0000 7ff49aff5700 1 -- 172.21.15.130:0/2671755554 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff4b0098ec0 con 0x55a3c26445a0 2023-09-12T16:19:15.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.348+0000 7ff4c9782b80 1 -- 172.21.15.130:0/2671755554 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4b0094fa0 con 0x55a3c26445a0 2023-09-12T16:19:15.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.348+0000 7ff49aff5700 1 -- 172.21.15.130:0/2671755554 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4b0099080 con 0x55a3c26445a0 2023-09-12T16:19:15.355 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.348+0000 7ff4c9782b80 10 client.25921 fetch_fsmap learned FSMap version 32 2023-09-12T16:19:15.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.348+0000 7ff4c9782b80 1 -- 172.21.15.130:0/2671755554 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a3c23e7250 con 0x55a3c26445a0 2023-09-12T16:19:15.356 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.349+0000 7ff49aff5700 1 -- 172.21.15.130:0/2671755554 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4b0094fa0 con 0x55a3c26445a0 2023-09-12T16:19:15.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.350+0000 7ff49aff5700 1 -- 172.21.15.130:0/2671755554 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff4b0077b80 con 0x55a3c26445a0 2023-09-12T16:19:15.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.350+0000 7ff4c9782b80 10 client.25921 fetch_fsmap finished waiting for FSMap version 32 2023-09-12T16:19:15.357 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.350+0000 7ff4c9782b80 10 client.25921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-12T16:19:15.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.350+0000 7ff4c9782b80 20 client.25921 populate_metadata read hostname 'smithi130' 2023-09-12T16:19:15.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.350+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a3c26cf480 0x55a3c26d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:19:15.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.350+0000 7ff4c9782b80 4 client.25921 mds_command: new command op to 24593 tid=0[{"prefix": "scrub status"}] 2023-09-12T16:19:15.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.350+0000 7ff4c9782b80 1 -- 172.21.15.130:0/2671755554 --> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a3c258d920 con 0x55a3c26cf480 2023-09-12T16:19:15.358 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.351+0000 7ff4b6ffd700 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a3c26cf480 0x55a3c26d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:19:15.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.351+0000 7ff49aff5700 10 client.25921 ms_handle_connect on v2:172.21.15.159:6838/3094783118 2023-09-12T16:19:15.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.351+0000 7ff4b6ffd700 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a3c26cf480 0x55a3c26d18a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec318fb5a2e12f60 server_cookie=82706798b9834cec in_seq=0 out_seq=0 2023-09-12T16:19:15.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.351+0000 7ff49aff5700 1 -- 172.21.15.130:0/2671755554 <== mds.0 v2:172.21.15.159:6838/3094783118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a3c258d920 con 0x55a3c26cf480 2023-09-12T16:19:15.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.351+0000 7ff49aff5700 10 client.25921 handle_command_reply: tid=0 2023-09-12T16:19:15.359 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.351+0000 7ff4c9782b80 1 client.25921 shutdown 2023-09-12T16:19:15.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.355+0000 7ff4c9782b80 1 -- 172.21.15.130:0/2671755554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c26445a0 msgr2=0x55a3c264fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:19:15.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.355+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c26445a0 0x55a3c264fe60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00b11d0 tx=0x7ff4b009ca90 comp rx=0 tx=0).stop 2023-09-12T16:19:15.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.355+0000 7ff4c9782b80 1 -- 172.21.15.130:0/2671755554 shutdown_connections 2023-09-12T16:19:15.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.355+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.159:6838/3094783118,v1:172.21.15.159:6839/3094783118] conn(0x55a3c26cf480 0x55a3c26d18a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.360 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.355+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3c26445a0 0x55a3c264fe60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.355+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a3c2642e00 0x55a3c26533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.355+0000 7ff4c9782b80 1 --2- 172.21.15.130:0/2671755554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3c2643c00 0x55a3c2653900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.361 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.355+0000 7ff4c9782b80 1 -- 172.21.15.130:0/2671755554 >> 172.21.15.130:0/2671755554 conn(0x55a3c25b7090 msgr2=0x55a3c265a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:19:15.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.356+0000 7ff49aff5700 0 client.25921 ms_handle_reset on v2:172.21.15.159:6838/3094783118 2023-09-12T16:19:15.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.356+0000 7ff4c9782b80 1 -- 172.21.15.130:0/2671755554 shutdown_connections 2023-09-12T16:19:15.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.358+0000 7ff4c9782b80 1 -- 172.21.15.130:0/2671755554 wait complete. 2023-09-12T16:19:15.362 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4c9782b80 20 client.25921 trim_cache size 0 max 16384 2023-09-12T16:19:15.363 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:19:15.363 INFO:teuthology.orchestra.run.smithi130.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-12T16:19:15.363 INFO:teuthology.orchestra.run.smithi130.stdout: "scrubs": { 2023-09-12T16:19:15.363 INFO:teuthology.orchestra.run.smithi130.stdout: "631eff48-3d69-497b-9b33-3789a94a5fca": { 2023-09-12T16:19:15.363 INFO:teuthology.orchestra.run.smithi130.stdout: "path": "/", 2023-09-12T16:19:15.364 INFO:teuthology.orchestra.run.smithi130.stdout: "tag": "631eff48-3d69-497b-9b33-3789a94a5fca", 2023-09-12T16:19:15.364 INFO:teuthology.orchestra.run.smithi130.stdout: "options": "recursive,force" 2023-09-12T16:19:15.364 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:19:15.364 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:19:15.364 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:19:15.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1736127783 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4a00a0790 msgr2=0x7ff4a00a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:19:15.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4a00a0790 0x7ff4a00a2c50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006010 tx=0x7ff4a801acb0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.365 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1736127783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b810c940 msgr2=0x7ff4b81a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:19:15.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b810c940 0x7ff4b81a3e90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0007f10 tx=0x7ff4b000d3b0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1736127783 shutdown_connections 2023-09-12T16:19:15.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7ff4a00a0790 0x7ff4a00a2c50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b810c940 0x7ff4b81a3e90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff4b8107600 0x7ff4b819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.366 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 --2- 172.21.15.130:0/1736127783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b8109f70 0x7ff4b819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:19:15.367 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.359+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1736127783 >> 172.21.15.130:0/1736127783 conn(0x7ff4b806d3b0 msgr2=0x7ff4b806f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:19:15.369 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.368+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1736127783 shutdown_connections 2023-09-12T16:19:15.390 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:19:15.378+0000 7ff4bd5bc700 1 -- 172.21.15.130:0/1736127783 wait complete. 2023-09-12T16:19:15.408 INFO:tasks.cephfs.filesystem:scrub status for tag:631eff48-3d69-497b-9b33-3789a94a5fca - {'path': '/', 'tag': '631eff48-3d69-497b-9b33-3789a94a5fca', 'options': 'recursive,force'} 2023-09-12T16:19:15.408 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/cephfs/filesystem.py", line 1730, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-09-12T16:19:17.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:16 smithi159 ceph-mon[102522]: pgmap v782: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 27 MiB/s wr, 2.42k op/s 2023-09-12T16:19:17.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:16 smithi130 ceph-mon[98670]: pgmap v782: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 27 MiB/s wr, 2.42k op/s 2023-09-12T16:19:17.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:16 smithi203 ceph-mon[103431]: pgmap v782: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 27 MiB/s wr, 2.42k op/s 2023-09-12T16:19:19.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:19 smithi130 ceph-mon[98670]: pgmap v783: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 2.51k op/s 2023-09-12T16:19:19.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:19 smithi159 ceph-mon[102522]: pgmap v783: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 2.51k op/s 2023-09-12T16:19:19.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:19 smithi203 ceph-mon[103431]: pgmap v783: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 2.51k op/s 2023-09-12T16:19:20.094 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 810.0 s, 759.7 tps, lat 163.185 ms stddev 345.035 2023-09-12T16:19:21.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:21 smithi130 ceph-mon[98670]: pgmap v784: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 33 MiB/s wr, 3.46k op/s 2023-09-12T16:19:21.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:21 smithi159 ceph-mon[102522]: pgmap v784: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 33 MiB/s wr, 3.46k op/s 2023-09-12T16:19:22.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:21 smithi203 ceph-mon[103431]: pgmap v784: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 33 MiB/s wr, 3.46k op/s 2023-09-12T16:19:23.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:23 smithi159 ceph-mon[102522]: pgmap v785: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 32 MiB/s wr, 3.40k op/s 2023-09-12T16:19:23.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:23 smithi130 ceph-mon[98670]: pgmap v785: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 32 MiB/s wr, 3.40k op/s 2023-09-12T16:19:23.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:23 smithi203 ceph-mon[103431]: pgmap v785: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 32 MiB/s wr, 3.40k op/s 2023-09-12T16:19:24.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:24 smithi130 ceph-mon[98670]: pgmap v786: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 29 MiB/s wr, 3.18k op/s 2023-09-12T16:19:24.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:24 smithi159 ceph-mon[102522]: pgmap v786: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 29 MiB/s wr, 3.18k op/s 2023-09-12T16:19:25.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:24 smithi203 ceph-mon[103431]: pgmap v786: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 29 MiB/s wr, 3.18k op/s 2023-09-12T16:19:25.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 815.0 s, 1417.0 tps, lat 69.937 ms stddev 71.646 2023-09-12T16:19:27.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:26 smithi130 ceph-mon[98670]: pgmap v787: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 33 MiB/s wr, 3.35k op/s 2023-09-12T16:19:27.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:26 smithi159 ceph-mon[102522]: pgmap v787: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 33 MiB/s wr, 3.35k op/s 2023-09-12T16:19:27.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:26 smithi203 ceph-mon[103431]: pgmap v787: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 33 MiB/s wr, 3.35k op/s 2023-09-12T16:19:29.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:29 smithi130 ceph-mon[98670]: pgmap v788: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 23 MiB/s wr, 2.20k op/s 2023-09-12T16:19:29.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:29 smithi159 ceph-mon[102522]: pgmap v788: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 23 MiB/s wr, 2.20k op/s 2023-09-12T16:19:29.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:29 smithi203 ceph-mon[103431]: pgmap v788: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 23 MiB/s wr, 2.20k op/s 2023-09-12T16:19:30.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 820.0 s, 1524.7 tps, lat 65.967 ms stddev 50.458 2023-09-12T16:19:31.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:31 smithi130 ceph-mon[98670]: pgmap v789: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 26 MiB/s wr, 2.23k op/s 2023-09-12T16:19:31.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:31 smithi159 ceph-mon[102522]: pgmap v789: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 26 MiB/s wr, 2.23k op/s 2023-09-12T16:19:31.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:31 smithi203 ceph-mon[103431]: pgmap v789: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 26 MiB/s wr, 2.23k op/s 2023-09-12T16:19:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:33 smithi130 ceph-mon[98670]: pgmap v790: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 1.35k op/s 2023-09-12T16:19:33.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:33 smithi159 ceph-mon[102522]: pgmap v790: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 1.35k op/s 2023-09-12T16:19:33.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:33 smithi203 ceph-mon[103431]: pgmap v790: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 1.35k op/s 2023-09-12T16:19:35.090 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 825.0 s, 1028.2 tps, lat 97.546 ms stddev 113.912 2023-09-12T16:19:35.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:35 smithi130 ceph-mon[98670]: pgmap v791: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-12T16:19:35.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:35 smithi159 ceph-mon[102522]: pgmap v791: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-12T16:19:36.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:35 smithi203 ceph-mon[103431]: pgmap v791: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-12T16:19:37.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:37 smithi159 ceph-mon[102522]: pgmap v792: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 19 MiB/s wr, 1.30k op/s 2023-09-12T16:19:37.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:37 smithi130 ceph-mon[98670]: pgmap v792: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 19 MiB/s wr, 1.30k op/s 2023-09-12T16:19:37.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:37 smithi203 ceph-mon[103431]: pgmap v792: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 19 MiB/s wr, 1.30k op/s 2023-09-12T16:19:39.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:39 smithi159 ceph-mon[102522]: pgmap v793: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-09-12T16:19:39.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:39 smithi130 ceph-mon[98670]: pgmap v793: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-09-12T16:19:39.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:39 smithi203 ceph-mon[103431]: pgmap v793: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-09-12T16:19:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 830.0 s, 1286.9 tps, lat 78.285 ms stddev 70.350 2023-09-12T16:19:41.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:41 smithi130 ceph-mon[98670]: pgmap v794: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 18 MiB/s wr, 1.29k op/s 2023-09-12T16:19:41.979 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:41 smithi159 ceph-mon[102522]: pgmap v794: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 18 MiB/s wr, 1.29k op/s 2023-09-12T16:19:41.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:41 smithi203 ceph-mon[103431]: pgmap v794: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 18 MiB/s wr, 1.29k op/s 2023-09-12T16:19:43.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:43 smithi130 ceph-mon[98670]: pgmap v795: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2023-09-12T16:19:43.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:43 smithi159 ceph-mon[102522]: pgmap v795: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2023-09-12T16:19:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:43 smithi203 ceph-mon[103431]: pgmap v795: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2023-09-12T16:19:45.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 835.0 s, 1405.8 tps, lat 70.509 ms stddev 105.885 2023-09-12T16:19:45.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:45 smithi130 ceph-mon[98670]: pgmap v796: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-09-12T16:19:45.978 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:45 smithi159 ceph-mon[102522]: pgmap v796: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-09-12T16:19:45.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:45 smithi203 ceph-mon[103431]: pgmap v796: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-09-12T16:19:47.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:47 smithi130 ceph-mon[98670]: pgmap v797: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.54k op/s 2023-09-12T16:19:47.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:47 smithi159 ceph-mon[102522]: pgmap v797: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.54k op/s 2023-09-12T16:19:48.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:47 smithi203 ceph-mon[103431]: pgmap v797: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.54k op/s 2023-09-12T16:19:49.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:49 smithi159 ceph-mon[102522]: pgmap v798: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 1.43k op/s 2023-09-12T16:19:49.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:49 smithi130 ceph-mon[98670]: pgmap v798: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 1.43k op/s 2023-09-12T16:19:49.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:49 smithi203 ceph-mon[103431]: pgmap v798: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 1.43k op/s 2023-09-12T16:19:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 840.0 s, 472.5 tps, lat 208.577 ms stddev 389.867 2023-09-12T16:19:50.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:50 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:19:50.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:50 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:19:50.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:50 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:19:52.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:51 smithi203 ceph-mon[103431]: pgmap v799: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 27 MiB/s wr, 2.38k op/s 2023-09-12T16:19:52.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:19:52.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:19:52.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:51 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:19:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:51 smithi130 ceph-mon[98670]: pgmap v799: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 27 MiB/s wr, 2.38k op/s 2023-09-12T16:19:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:19:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:19:52.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:51 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:19:52.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:51 smithi159 ceph-mon[102522]: pgmap v799: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 27 MiB/s wr, 2.38k op/s 2023-09-12T16:19:52.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:19:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:19:52.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:51 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:19:53.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:53 smithi159 ceph-mon[102522]: pgmap v800: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 28 MiB/s wr, 2.62k op/s 2023-09-12T16:19:53.980 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:53 smithi130 ceph-mon[98670]: pgmap v800: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 28 MiB/s wr, 2.62k op/s 2023-09-12T16:19:53.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:53 smithi203 ceph-mon[103431]: pgmap v800: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 28 MiB/s wr, 2.62k op/s 2023-09-12T16:19:55.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 845.0 s, 479.6 tps, lat 213.480 ms stddev 338.841 2023-09-12T16:19:55.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:55 smithi130 ceph-mon[98670]: pgmap v801: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 28 MiB/s wr, 2.79k op/s 2023-09-12T16:19:55.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:55 smithi159 ceph-mon[102522]: pgmap v801: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 28 MiB/s wr, 2.79k op/s 2023-09-12T16:19:55.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:55 smithi203 ceph-mon[103431]: pgmap v801: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 28 MiB/s wr, 2.79k op/s 2023-09-12T16:19:57.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:57 smithi130 ceph-mon[98670]: pgmap v802: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 34 MiB/s wr, 3.66k op/s 2023-09-12T16:19:57.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:57 smithi159 ceph-mon[102522]: pgmap v802: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 34 MiB/s wr, 3.66k op/s 2023-09-12T16:19:57.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:57 smithi203 ceph-mon[103431]: pgmap v802: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 34 MiB/s wr, 3.66k op/s 2023-09-12T16:19:59.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:19:59 smithi159 ceph-mon[102522]: pgmap v803: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 31 MiB/s wr, 3.29k op/s 2023-09-12T16:19:59.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:19:59 smithi130 ceph-mon[98670]: pgmap v803: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 31 MiB/s wr, 3.29k op/s 2023-09-12T16:19:59.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:19:59 smithi203 ceph-mon[103431]: pgmap v803: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 31 MiB/s wr, 3.29k op/s 2023-09-12T16:20:00.089 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 850.0 s, 1580.8 tps, lat 62.498 ms stddev 55.671 2023-09-12T16:20:00.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:00 smithi130 ceph-mon[98670]: overall HEALTH_OK 2023-09-12T16:20:00.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:00 smithi159 ceph-mon[102522]: overall HEALTH_OK 2023-09-12T16:20:00.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:00 smithi203 ceph-mon[103431]: overall HEALTH_OK 2023-09-12T16:20:01.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:01 smithi130 ceph-mon[98670]: pgmap v804: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 29 MiB/s wr, 3.21k op/s 2023-09-12T16:20:01.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:01 smithi159 ceph-mon[102522]: pgmap v804: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 29 MiB/s wr, 3.21k op/s 2023-09-12T16:20:01.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:01 smithi203 ceph-mon[103431]: pgmap v804: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 29 MiB/s wr, 3.21k op/s 2023-09-12T16:20:03.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:03 smithi159 ceph-mon[102522]: pgmap v805: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 26 MiB/s wr, 2.53k op/s 2023-09-12T16:20:03.983 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:03 smithi130 ceph-mon[98670]: pgmap v805: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 26 MiB/s wr, 2.53k op/s 2023-09-12T16:20:03.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:03 smithi203 ceph-mon[103431]: pgmap v805: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 26 MiB/s wr, 2.53k op/s 2023-09-12T16:20:05.088 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 855.0 s, 1518.4 tps, lat 65.977 ms stddev 53.712 2023-09-12T16:20:05.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:05 smithi130 ceph-mon[98670]: pgmap v806: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 23 MiB/s wr, 2.03k op/s 2023-09-12T16:20:05.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:05 smithi159 ceph-mon[102522]: pgmap v806: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 23 MiB/s wr, 2.03k op/s 2023-09-12T16:20:05.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:05 smithi203 ceph-mon[103431]: pgmap v806: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 23 MiB/s wr, 2.03k op/s 2023-09-12T16:20:08.053 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:07 smithi130 ceph-mon[98670]: pgmap v807: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 24 MiB/s wr, 2.01k op/s 2023-09-12T16:20:08.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:07 smithi159 ceph-mon[102522]: pgmap v807: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 24 MiB/s wr, 2.01k op/s 2023-09-12T16:20:08.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:07 smithi203 ceph-mon[103431]: pgmap v807: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 24 MiB/s wr, 2.01k op/s 2023-09-12T16:20:09.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:09 smithi159 ceph-mon[102522]: pgmap v808: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-09-12T16:20:09.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:09 smithi130 ceph-mon[98670]: pgmap v808: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-09-12T16:20:09.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:09 smithi203 ceph-mon[103431]: pgmap v808: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-09-12T16:20:10.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 860.0 s, 933.4 tps, lat 104.978 ms stddev 142.540 2023-09-12T16:20:11.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:11 smithi130 ceph-mon[98670]: pgmap v809: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 18 MiB/s wr, 1.08k op/s 2023-09-12T16:20:11.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:11 smithi159 ceph-mon[102522]: pgmap v809: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 18 MiB/s wr, 1.08k op/s 2023-09-12T16:20:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:11 smithi203 ceph-mon[103431]: pgmap v809: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 18 MiB/s wr, 1.08k op/s 2023-09-12T16:20:13.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:13 smithi159 ceph-mon[102522]: pgmap v810: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 21 MiB/s wr, 1.35k op/s 2023-09-12T16:20:13.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:13 smithi130 ceph-mon[98670]: pgmap v810: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 21 MiB/s wr, 1.35k op/s 2023-09-12T16:20:13.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:13 smithi203 ceph-mon[103431]: pgmap v810: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 21 MiB/s wr, 1.35k op/s 2023-09-12T16:20:15.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 865.0 s, 967.7 tps, lat 100.166 ms stddev 80.054 2023-09-12T16:20:15.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:15 smithi159 ceph-mon[102522]: pgmap v811: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-09-12T16:20:15.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:15 smithi130 ceph-mon[98670]: pgmap v811: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-09-12T16:20:15.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:15 smithi203 ceph-mon[103431]: pgmap v811: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-09-12T16:20:17.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:17 smithi159 ceph-mon[102522]: pgmap v812: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-09-12T16:20:17.981 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:17 smithi130 ceph-mon[98670]: pgmap v812: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-09-12T16:20:17.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:17 smithi203 ceph-mon[103431]: pgmap v812: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-09-12T16:20:19.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:19 smithi130 ceph-mon[98670]: pgmap v813: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-09-12T16:20:19.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:19 smithi159 ceph-mon[102522]: pgmap v813: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-09-12T16:20:20.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:19 smithi203 ceph-mon[103431]: pgmap v813: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-09-12T16:20:20.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 870.0 s, 1381.1 tps, lat 70.227 ms stddev 75.261 2023-09-12T16:20:21.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:21 smithi159 ceph-mon[102522]: pgmap v814: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 15 MiB/s wr, 1.25k op/s 2023-09-12T16:20:21.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:21 smithi130 ceph-mon[98670]: pgmap v814: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 15 MiB/s wr, 1.25k op/s 2023-09-12T16:20:21.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:21 smithi203 ceph-mon[103431]: pgmap v814: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 15 MiB/s wr, 1.25k op/s 2023-09-12T16:20:23.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:23 smithi130 ceph-mon[98670]: pgmap v815: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 1.59k op/s 2023-09-12T16:20:23.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:23 smithi159 ceph-mon[102522]: pgmap v815: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 1.59k op/s 2023-09-12T16:20:23.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:23 smithi203 ceph-mon[103431]: pgmap v815: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 1.59k op/s 2023-09-12T16:20:25.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 875.0 s, 1085.2 tps, lat 95.612 ms stddev 187.593 2023-09-12T16:20:25.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:25 smithi130 ceph-mon[98670]: pgmap v816: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.41k op/s 2023-09-12T16:20:25.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:25 smithi159 ceph-mon[102522]: pgmap v816: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.41k op/s 2023-09-12T16:20:25.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:25 smithi203 ceph-mon[103431]: pgmap v816: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.41k op/s 2023-09-12T16:20:28.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:27 smithi130 ceph-mon[98670]: pgmap v817: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 25 MiB/s wr, 2.22k op/s 2023-09-12T16:20:28.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:27 smithi159 ceph-mon[102522]: pgmap v817: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 25 MiB/s wr, 2.22k op/s 2023-09-12T16:20:28.253 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:27 smithi203 ceph-mon[103431]: pgmap v817: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 25 MiB/s wr, 2.22k op/s 2023-09-12T16:20:29.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:29 smithi130 ceph-mon[98670]: pgmap v818: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 25 MiB/s wr, 2.12k op/s 2023-09-12T16:20:29.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:29 smithi159 ceph-mon[102522]: pgmap v818: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 25 MiB/s wr, 2.12k op/s 2023-09-12T16:20:29.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:29 smithi203 ceph-mon[103431]: pgmap v818: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 25 MiB/s wr, 2.12k op/s 2023-09-12T16:20:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 880.0 s, 137.2 tps, lat 720.389 ms stddev 530.474 2023-09-12T16:20:31.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:31 smithi130 ceph-mon[98670]: pgmap v819: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 26 MiB/s wr, 2.44k op/s 2023-09-12T16:20:31.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:31 smithi159 ceph-mon[102522]: pgmap v819: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 26 MiB/s wr, 2.44k op/s 2023-09-12T16:20:31.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:31 smithi203 ceph-mon[103431]: pgmap v819: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 26 MiB/s wr, 2.44k op/s 2023-09-12T16:20:33.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:33 smithi130 ceph-mon[98670]: pgmap v820: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 34 MiB/s wr, 3.40k op/s 2023-09-12T16:20:33.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:33 smithi159 ceph-mon[102522]: pgmap v820: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 34 MiB/s wr, 3.40k op/s 2023-09-12T16:20:33.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:33 smithi203 ceph-mon[103431]: pgmap v820: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 34 MiB/s wr, 3.40k op/s 2023-09-12T16:20:35.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 885.0 s, 1710.2 tps, lat 62.275 ms stddev 79.362 2023-09-12T16:20:35.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:35 smithi159 ceph-mon[102522]: pgmap v821: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 32 MiB/s wr, 3.31k op/s 2023-09-12T16:20:35.980 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:35 smithi130 ceph-mon[98670]: pgmap v821: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 32 MiB/s wr, 3.31k op/s 2023-09-12T16:20:35.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:35 smithi203 ceph-mon[103431]: pgmap v821: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 32 MiB/s wr, 3.31k op/s 2023-09-12T16:20:37.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:37 smithi159 ceph-mon[102522]: pgmap v822: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 36 MiB/s wr, 3.51k op/s 2023-09-12T16:20:37.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:37 smithi130 ceph-mon[98670]: pgmap v822: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 36 MiB/s wr, 3.51k op/s 2023-09-12T16:20:38.008 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:37 smithi203 ceph-mon[103431]: pgmap v822: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 36 MiB/s wr, 3.51k op/s 2023-09-12T16:20:40.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 890.0 s, 1652.6 tps, lat 60.433 ms stddev 47.670 2023-09-12T16:20:40.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:39 smithi130 ceph-mon[98670]: pgmap v823: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 27 MiB/s wr, 2.68k op/s 2023-09-12T16:20:40.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:39 smithi159 ceph-mon[102522]: pgmap v823: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 27 MiB/s wr, 2.68k op/s 2023-09-12T16:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:39 smithi203 ceph-mon[103431]: pgmap v823: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 27 MiB/s wr, 2.68k op/s 2023-09-12T16:20:41.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:40 smithi130 ceph-mon[98670]: pgmap v824: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 25 MiB/s wr, 2.48k op/s 2023-09-12T16:20:41.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:40 smithi159 ceph-mon[102522]: pgmap v824: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 25 MiB/s wr, 2.48k op/s 2023-09-12T16:20:41.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:40 smithi203 ceph-mon[103431]: pgmap v824: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 25 MiB/s wr, 2.48k op/s 2023-09-12T16:20:43.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:43 smithi159 ceph-mon[102522]: pgmap v825: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 27 MiB/s wr, 2.43k op/s 2023-09-12T16:20:43.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:43 smithi130 ceph-mon[98670]: pgmap v825: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 27 MiB/s wr, 2.43k op/s 2023-09-12T16:20:44.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:43 smithi203 ceph-mon[103431]: pgmap v825: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 27 MiB/s wr, 2.43k op/s 2023-09-12T16:20:45.098 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 895.0 s, 981.5 tps, lat 99.958 ms stddev 145.360 2023-09-12T16:20:45.981 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:45 smithi130 ceph-mon[98670]: pgmap v826: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-12T16:20:45.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:45 smithi159 ceph-mon[102522]: pgmap v826: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-12T16:20:45.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:45 smithi203 ceph-mon[103431]: pgmap v826: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-09-12T16:20:47.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:47 smithi130 ceph-mon[98670]: pgmap v827: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.37k op/s 2023-09-12T16:20:47.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:47 smithi159 ceph-mon[102522]: pgmap v827: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.37k op/s 2023-09-12T16:20:47.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:47 smithi203 ceph-mon[103431]: pgmap v827: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.37k op/s 2023-09-12T16:20:49.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:49 smithi130 ceph-mon[98670]: pgmap v828: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-09-12T16:20:49.982 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:49 smithi159 ceph-mon[102522]: pgmap v828: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-09-12T16:20:49.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:49 smithi203 ceph-mon[103431]: pgmap v828: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-09-12T16:20:50.087 INFO:teuthology.orchestra.run.smithi130.stderr:progress: 900.0 s, 870.3 tps, lat 114.842 ms stddev 126.095 2023-09-12T16:20:50.324 INFO:teuthology.orchestra.run.smithi130.stdout:transaction type: 2023-09-12T16:20:50.324 INFO:teuthology.orchestra.run.smithi130.stdout:scaling factor: 500 2023-09-12T16:20:50.324 INFO:teuthology.orchestra.run.smithi130.stdout:query mode: simple 2023-09-12T16:20:50.324 INFO:teuthology.orchestra.run.smithi130.stdout:number of clients: 100 2023-09-12T16:20:50.325 INFO:teuthology.orchestra.run.smithi130.stdout:number of threads: 4 2023-09-12T16:20:50.325 INFO:teuthology.orchestra.run.smithi130.stdout:duration: 900 s 2023-09-12T16:20:50.325 INFO:teuthology.orchestra.run.smithi130.stdout:number of transactions actually processed: 1073914 2023-09-12T16:20:50.325 INFO:teuthology.orchestra.run.smithi130.stdout:latency average = 83.807 ms 2023-09-12T16:20:50.325 INFO:teuthology.orchestra.run.smithi130.stdout:latency stddev = 185.297 ms 2023-09-12T16:20:50.326 INFO:teuthology.orchestra.run.smithi130.stdout:tps = 1192.923620 (including connections establishing) 2023-09-12T16:20:50.326 INFO:teuthology.orchestra.run.smithi130.stdout:tps = 1192.926667 (excluding connections establishing) 2023-09-12T16:20:50.327 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-09-12T16:20:51.977 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:51 smithi130 ceph-mon[98670]: pgmap v829: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.47k op/s 2023-09-12T16:20:51.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:51 smithi159 ceph-mon[102522]: pgmap v829: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.47k op/s 2023-09-12T16:20:51.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:51 smithi203 ceph-mon[103431]: pgmap v829: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.47k op/s 2023-09-12T16:20:53.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:52 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:20:53.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:52 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:20:53.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:52 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:20:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:53 smithi130 ceph-mon[98670]: pgmap v830: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 20 MiB/s wr, 1.70k op/s 2023-09-12T16:20:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:20:54.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:20:54.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:53 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:20:54.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:53 smithi159 ceph-mon[102522]: pgmap v830: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 20 MiB/s wr, 1.70k op/s 2023-09-12T16:20:54.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:20:54.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:20:54.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:53 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:20:54.251 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:53 smithi203 ceph-mon[103431]: pgmap v830: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 20 MiB/s wr, 1.70k op/s 2023-09-12T16:20:54.252 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:20:54.252 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:20:54.252 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:53 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:20:55.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:55 smithi159 ceph-mon[102522]: pgmap v831: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-09-12T16:20:55.978 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:55 smithi130 ceph-mon[98670]: pgmap v831: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-09-12T16:20:55.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:55 smithi203 ceph-mon[103431]: pgmap v831: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-09-12T16:20:58.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:57 smithi159 ceph-mon[102522]: pgmap v832: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 1.84k op/s 2023-09-12T16:20:58.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:57 smithi130 ceph-mon[98670]: pgmap v832: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 1.84k op/s 2023-09-12T16:20:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:57 smithi203 ceph-mon[103431]: pgmap v832: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 1.84k op/s 2023-09-12T16:21:00.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:20:59 smithi159 ceph-mon[102522]: pgmap v833: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 20 MiB/s wr, 2.18k op/s 2023-09-12T16:21:00.231 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:20:59 smithi130 ceph-mon[98670]: pgmap v833: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 20 MiB/s wr, 2.18k op/s 2023-09-12T16:21:00.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:20:59 smithi203 ceph-mon[103431]: pgmap v833: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 20 MiB/s wr, 2.18k op/s 2023-09-12T16:21:01.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:00 smithi159 ceph-mon[102522]: pgmap v834: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 19 MiB/s wr, 2.10k op/s 2023-09-12T16:21:01.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:00 smithi130 ceph-mon[98670]: pgmap v834: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 19 MiB/s wr, 2.10k op/s 2023-09-12T16:21:01.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:00 smithi203 ceph-mon[103431]: pgmap v834: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 19 MiB/s wr, 2.10k op/s 2023-09-12T16:21:03.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:03 smithi159 ceph-mon[102522]: pgmap v835: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 21 MiB/s wr, 2.37k op/s 2023-09-12T16:21:03.979 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:03 smithi130 ceph-mon[98670]: pgmap v835: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 21 MiB/s wr, 2.37k op/s 2023-09-12T16:21:03.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:03 smithi203 ceph-mon[103431]: pgmap v835: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 21 MiB/s wr, 2.37k op/s 2023-09-12T16:21:04.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:04 smithi130 ceph-mon[98670]: pgmap v836: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 18 MiB/s wr, 2.17k op/s 2023-09-12T16:21:04.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:04 smithi159 ceph-mon[102522]: pgmap v836: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 18 MiB/s wr, 2.17k op/s 2023-09-12T16:21:04.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:04 smithi203 ceph-mon[103431]: pgmap v836: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 18 MiB/s wr, 2.17k op/s 2023-09-12T16:21:06.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:05 smithi159 ceph-mon[102522]: pgmap v837: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 17 MiB/s wr, 2.13k op/s 2023-09-12T16:21:06.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:05 smithi130 ceph-mon[98670]: pgmap v837: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 17 MiB/s wr, 2.13k op/s 2023-09-12T16:21:06.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:05 smithi203 ceph-mon[103431]: pgmap v837: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 17 MiB/s wr, 2.13k op/s 2023-09-12T16:21:08.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:08 smithi130 ceph-mon[98670]: pgmap v838: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 1.79k op/s 2023-09-12T16:21:08.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:08 smithi159 ceph-mon[102522]: pgmap v838: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 1.79k op/s 2023-09-12T16:21:08.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:08 smithi203 ceph-mon[103431]: pgmap v838: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 1.79k op/s 2023-09-12T16:21:10.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:09 smithi130 ceph-mon[98670]: pgmap v839: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.26k op/s 2023-09-12T16:21:10.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:09 smithi159 ceph-mon[102522]: pgmap v839: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.26k op/s 2023-09-12T16:21:10.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:09 smithi203 ceph-mon[103431]: pgmap v839: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.26k op/s 2023-09-12T16:21:12.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:11 smithi130 ceph-mon[98670]: pgmap v840: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.6 MiB/s wr, 960 op/s 2023-09-12T16:21:12.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:11 smithi159 ceph-mon[102522]: pgmap v840: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.6 MiB/s wr, 960 op/s 2023-09-12T16:21:12.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:11 smithi203 ceph-mon[103431]: pgmap v840: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.6 MiB/s wr, 960 op/s 2023-09-12T16:21:13.606 DEBUG:teuthology.orchestra.run.smithi130:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-12T16:21:13.686 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/: 2023-09-12T16:21:13.686 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.687 INFO:teuthology.orchestra.run.smithi130.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:21:13.687 INFO:teuthology.orchestra.run.smithi130.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 .. 2023-09-12T16:21:13.687 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 data 2023-09-12T16:21:13.687 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.687 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data: 2023-09-12T16:21:13.688 INFO:teuthology.orchestra.run.smithi130.stdout:total 31 2023-09-12T16:21:13.688 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 . 2023-09-12T16:21:13.688 INFO:teuthology.orchestra.run.smithi130.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 .. 2023-09-12T16:21:13.688 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 12 16:05 base 2023-09-12T16:21:13.688 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 12 16:03 current_logfiles 2023-09-12T16:21:13.688 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Sep 12 16:03 global 2023-09-12T16:21:13.689 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 log 2023-09-12T16:21:13.689 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_commit_ts 2023-09-12T16:21:13.689 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_dynshmem 2023-09-12T16:21:13.689 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Sep 12 16:03 pg_hba.conf 2023-09-12T16:21:13.689 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 12 16:03 pg_ident.conf 2023-09-12T16:21:13.689 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:21 pg_logical 2023-09-12T16:21:13.690 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 pg_multixact 2023-09-12T16:21:13.690 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 pg_notify 2023-09-12T16:21:13.690 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_replslot 2023-09-12T16:21:13.690 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_serial 2023-09-12T16:21:13.690 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_snapshots 2023-09-12T16:21:13.691 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:21 pg_stat 2023-09-12T16:21:13.691 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:21 pg_stat_tmp 2023-09-12T16:21:13.691 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:20 pg_subtrans 2023-09-12T16:21:13.691 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_tblspc 2023-09-12T16:21:13.691 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 pg_twophase 2023-09-12T16:21:13.691 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 PG_VERSION 2023-09-12T16:21:13.692 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 12 16:21 pg_wal 2023-09-12T16:21:13.692 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:20 pg_xact 2023-09-12T16:21:13.692 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 12 16:03 postgresql.auto.conf 2023-09-12T16:21:13.692 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Sep 12 16:03 postgresql.conf 2023-09-12T16:21:13.692 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 12 16:03 postmaster.opts 2023-09-12T16:21:13.693 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.693 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/base: 2023-09-12T16:21:13.693 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.693 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 12 16:05 . 2023-09-12T16:21:13.693 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.693 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 1 2023-09-12T16:21:13.694 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 12346 2023-09-12T16:21:13.694 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 315 Sep 12 16:06 12347 2023-09-12T16:21:13.694 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:05 pgsql_tmp 2023-09-12T16:21:13.694 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.694 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/base/1: 2023-09-12T16:21:13.695 INFO:teuthology.orchestra.run.smithi130.stdout:total 7209 2023-09-12T16:21:13.695 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 . 2023-09-12T16:21:13.695 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 12 16:05 .. 2023-09-12T16:21:13.695 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 112 2023-09-12T16:21:13.695 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 113 2023-09-12T16:21:13.695 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 12 16:03 12182 2023-09-12T16:21:13.696 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12182_fsm 2023-09-12T16:21:13.696 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12182_vm 2023-09-12T16:21:13.696 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12184 2023-09-12T16:21:13.696 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12186 2023-09-12T16:21:13.696 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187 2023-09-12T16:21:13.696 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12187_fsm 2023-09-12T16:21:13.697 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187_vm 2023-09-12T16:21:13.697 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12189 2023-09-12T16:21:13.697 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12191 2023-09-12T16:21:13.697 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192 2023-09-12T16:21:13.697 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12192_fsm 2023-09-12T16:21:13.697 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192_vm 2023-09-12T16:21:13.698 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12194 2023-09-12T16:21:13.698 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12196 2023-09-12T16:21:13.698 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197 2023-09-12T16:21:13.698 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12197_fsm 2023-09-12T16:21:13.698 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197_vm 2023-09-12T16:21:13.699 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12199 2023-09-12T16:21:13.699 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12201 2023-09-12T16:21:13.699 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202 2023-09-12T16:21:13.699 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12202_fsm 2023-09-12T16:21:13.699 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202_vm 2023-09-12T16:21:13.699 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12204 2023-09-12T16:21:13.700 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12206 2023-09-12T16:21:13.700 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207 2023-09-12T16:21:13.700 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12207_fsm 2023-09-12T16:21:13.700 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207_vm 2023-09-12T16:21:13.700 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12209 2023-09-12T16:21:13.701 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12211 2023-09-12T16:21:13.701 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12212 2023-09-12T16:21:13.701 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12214 2023-09-12T16:21:13.701 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12216 2023-09-12T16:21:13.701 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 12 16:03 1247 2023-09-12T16:21:13.702 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1247_fsm 2023-09-12T16:21:13.702 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1247_vm 2023-09-12T16:21:13.702 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 12 16:03 1249 2023-09-12T16:21:13.702 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1249_fsm 2023-09-12T16:21:13.702 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1249_vm 2023-09-12T16:21:13.702 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 12 16:03 1255 2023-09-12T16:21:13.703 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1255_fsm 2023-09-12T16:21:13.703 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1255_vm 2023-09-12T16:21:13.703 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 12 16:03 1259 2023-09-12T16:21:13.703 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1259_fsm 2023-09-12T16:21:13.703 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1259_vm 2023-09-12T16:21:13.704 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417 2023-09-12T16:21:13.704 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417_vm 2023-09-12T16:21:13.704 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418 2023-09-12T16:21:13.704 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418_vm 2023-09-12T16:21:13.704 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 174 2023-09-12T16:21:13.705 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 175 2023-09-12T16:21:13.705 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2187 2023-09-12T16:21:13.705 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224 2023-09-12T16:21:13.705 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224_vm 2023-09-12T16:21:13.705 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328 2023-09-12T16:21:13.705 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328_vm 2023-09-12T16:21:13.706 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336 2023-09-12T16:21:13.706 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336_vm 2023-09-12T16:21:13.706 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2337 2023-09-12T16:21:13.706 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2600 2023-09-12T16:21:13.706 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2600_fsm 2023-09-12T16:21:13.706 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2600_vm 2023-09-12T16:21:13.707 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601 2023-09-12T16:21:13.707 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2601_fsm 2023-09-12T16:21:13.707 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601_vm 2023-09-12T16:21:13.707 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2602 2023-09-12T16:21:13.707 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2602_fsm 2023-09-12T16:21:13.708 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2602_vm 2023-09-12T16:21:13.708 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2603 2023-09-12T16:21:13.708 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2603_fsm 2023-09-12T16:21:13.708 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2603_vm 2023-09-12T16:21:13.708 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604 2023-09-12T16:21:13.708 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604_vm 2023-09-12T16:21:13.709 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2605 2023-09-12T16:21:13.709 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2605_fsm 2023-09-12T16:21:13.709 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2605_vm 2023-09-12T16:21:13.709 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606 2023-09-12T16:21:13.709 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2606_fsm 2023-09-12T16:21:13.709 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606_vm 2023-09-12T16:21:13.710 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607 2023-09-12T16:21:13.710 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607_fsm 2023-09-12T16:21:13.710 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2607_vm 2023-09-12T16:21:13.710 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 12 16:03 2608 2023-09-12T16:21:13.710 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2608_fsm 2023-09-12T16:21:13.711 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2608_vm 2023-09-12T16:21:13.711 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 12 16:03 2609 2023-09-12T16:21:13.711 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2609_fsm 2023-09-12T16:21:13.711 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2609_vm 2023-09-12T16:21:13.711 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610 2023-09-12T16:21:13.711 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610_fsm 2023-09-12T16:21:13.712 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2610_vm 2023-09-12T16:21:13.712 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611 2023-09-12T16:21:13.712 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611_vm 2023-09-12T16:21:13.712 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612 2023-09-12T16:21:13.712 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2612_fsm 2023-09-12T16:21:13.713 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612_vm 2023-09-12T16:21:13.713 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613 2023-09-12T16:21:13.713 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613_vm 2023-09-12T16:21:13.713 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615 2023-09-12T16:21:13.713 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2615_fsm 2023-09-12T16:21:13.713 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615_vm 2023-09-12T16:21:13.714 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616 2023-09-12T16:21:13.714 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616_fsm 2023-09-12T16:21:13.714 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2616_vm 2023-09-12T16:21:13.714 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 12 16:03 2617 2023-09-12T16:21:13.714 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2617_fsm 2023-09-12T16:21:13.715 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2617_vm 2023-09-12T16:21:13.715 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 12 16:03 2618 2023-09-12T16:21:13.715 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2618_fsm 2023-09-12T16:21:13.715 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2618_vm 2023-09-12T16:21:13.715 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 12 16:03 2619 2023-09-12T16:21:13.716 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2619_fsm 2023-09-12T16:21:13.716 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2619_vm 2023-09-12T16:21:13.716 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620 2023-09-12T16:21:13.716 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620_vm 2023-09-12T16:21:13.716 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2650 2023-09-12T16:21:13.716 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2651 2023-09-12T16:21:13.717 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2652 2023-09-12T16:21:13.717 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2653 2023-09-12T16:21:13.717 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2654 2023-09-12T16:21:13.717 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2655 2023-09-12T16:21:13.717 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2656 2023-09-12T16:21:13.718 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2657 2023-09-12T16:21:13.718 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 12 16:03 2658 2023-09-12T16:21:13.718 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2659 2023-09-12T16:21:13.718 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2660 2023-09-12T16:21:13.718 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2661 2023-09-12T16:21:13.718 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2662 2023-09-12T16:21:13.719 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2663 2023-09-12T16:21:13.719 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2664 2023-09-12T16:21:13.719 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2665 2023-09-12T16:21:13.722 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2666 2023-09-12T16:21:13.722 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2667 2023-09-12T16:21:13.722 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2668 2023-09-12T16:21:13.722 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2669 2023-09-12T16:21:13.722 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2670 2023-09-12T16:21:13.723 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Sep 12 16:03 2673 2023-09-12T16:21:13.723 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 12 16:03 2674 2023-09-12T16:21:13.723 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 12 16:03 2675 2023-09-12T16:21:13.723 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2678 2023-09-12T16:21:13.723 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2679 2023-09-12T16:21:13.723 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2680 2023-09-12T16:21:13.724 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2681 2023-09-12T16:21:13.724 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2682 2023-09-12T16:21:13.724 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2683 2023-09-12T16:21:13.724 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2684 2023-09-12T16:21:13.724 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2685 2023-09-12T16:21:13.725 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2686 2023-09-12T16:21:13.725 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2687 2023-09-12T16:21:13.725 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2688 2023-09-12T16:21:13.725 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2689 2023-09-12T16:21:13.725 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2690 2023-09-12T16:21:13.725 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 12 16:03 2691 2023-09-12T16:21:13.726 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2692 2023-09-12T16:21:13.726 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2693 2023-09-12T16:21:13.726 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2696 2023-09-12T16:21:13.726 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2699 2023-09-12T16:21:13.726 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2701 2023-09-12T16:21:13.727 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2702 2023-09-12T16:21:13.727 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2703 2023-09-12T16:21:13.727 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2704 2023-09-12T16:21:13.727 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2753 2023-09-12T16:21:13.727 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2753_fsm 2023-09-12T16:21:13.727 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2753_vm 2023-09-12T16:21:13.728 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2754 2023-09-12T16:21:13.728 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2755 2023-09-12T16:21:13.728 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2756 2023-09-12T16:21:13.728 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2757 2023-09-12T16:21:13.728 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830 2023-09-12T16:21:13.729 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830_vm 2023-09-12T16:21:13.729 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2831 2023-09-12T16:21:13.729 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832 2023-09-12T16:21:13.729 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832_vm 2023-09-12T16:21:13.729 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2833 2023-09-12T16:21:13.729 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834 2023-09-12T16:21:13.730 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834_vm 2023-09-12T16:21:13.730 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2835 2023-09-12T16:21:13.730 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836 2023-09-12T16:21:13.730 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836_vm 2023-09-12T16:21:13.730 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2837 2023-09-12T16:21:13.730 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 12 16:03 2838 2023-09-12T16:21:13.731 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2838_fsm 2023-09-12T16:21:13.731 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2838_vm 2023-09-12T16:21:13.731 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2839 2023-09-12T16:21:13.731 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840 2023-09-12T16:21:13.731 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840_fsm 2023-09-12T16:21:13.732 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2840_vm 2023-09-12T16:21:13.732 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2841 2023-09-12T16:21:13.732 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995 2023-09-12T16:21:13.732 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995_vm 2023-09-12T16:21:13.732 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2996 2023-09-12T16:21:13.732 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079 2023-09-12T16:21:13.733 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3079_fsm 2023-09-12T16:21:13.733 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079_vm 2023-09-12T16:21:13.733 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3080 2023-09-12T16:21:13.733 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3081 2023-09-12T16:21:13.733 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3085 2023-09-12T16:21:13.734 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118 2023-09-12T16:21:13.734 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118_vm 2023-09-12T16:21:13.734 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3119 2023-09-12T16:21:13.734 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3164 2023-09-12T16:21:13.734 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256 2023-09-12T16:21:13.734 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256_vm 2023-09-12T16:21:13.735 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3257 2023-09-12T16:21:13.735 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3258 2023-09-12T16:21:13.735 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350 2023-09-12T16:21:13.735 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350_vm 2023-09-12T16:21:13.735 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3351 2023-09-12T16:21:13.736 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3379 2023-09-12T16:21:13.736 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3380 2023-09-12T16:21:13.736 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381 2023-09-12T16:21:13.736 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381_vm 2023-09-12T16:21:13.736 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3394 2023-09-12T16:21:13.736 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3394_fsm 2023-09-12T16:21:13.737 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3394_vm 2023-09-12T16:21:13.737 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3395 2023-09-12T16:21:13.737 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439 2023-09-12T16:21:13.737 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439_vm 2023-09-12T16:21:13.737 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3440 2023-09-12T16:21:13.737 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 3455 2023-09-12T16:21:13.738 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3456 2023-09-12T16:21:13.738 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3456_fsm 2023-09-12T16:21:13.738 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3456_vm 2023-09-12T16:21:13.738 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466 2023-09-12T16:21:13.738 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466_vm 2023-09-12T16:21:13.738 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3467 2023-09-12T16:21:13.739 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3468 2023-09-12T16:21:13.739 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501 2023-09-12T16:21:13.739 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501_vm 2023-09-12T16:21:13.739 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3502 2023-09-12T16:21:13.739 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3503 2023-09-12T16:21:13.740 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3534 2023-09-12T16:21:13.740 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541 2023-09-12T16:21:13.740 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3541_fsm 2023-09-12T16:21:13.740 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541_vm 2023-09-12T16:21:13.740 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3542 2023-09-12T16:21:13.740 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3574 2023-09-12T16:21:13.741 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3575 2023-09-12T16:21:13.741 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576 2023-09-12T16:21:13.741 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576_vm 2023-09-12T16:21:13.741 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596 2023-09-12T16:21:13.741 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596_vm 2023-09-12T16:21:13.741 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3597 2023-09-12T16:21:13.742 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598 2023-09-12T16:21:13.742 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598_vm 2023-09-12T16:21:13.742 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3599 2023-09-12T16:21:13.742 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600 2023-09-12T16:21:13.742 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3600_fsm 2023-09-12T16:21:13.743 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600_vm 2023-09-12T16:21:13.743 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601 2023-09-12T16:21:13.743 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3601_fsm 2023-09-12T16:21:13.743 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601_vm 2023-09-12T16:21:13.743 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602 2023-09-12T16:21:13.743 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3602_fsm 2023-09-12T16:21:13.744 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602_vm 2023-09-12T16:21:13.744 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3603 2023-09-12T16:21:13.744 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3603_fsm 2023-09-12T16:21:13.744 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3603_vm 2023-09-12T16:21:13.747 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3604 2023-09-12T16:21:13.748 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3605 2023-09-12T16:21:13.748 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3606 2023-09-12T16:21:13.748 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3607 2023-09-12T16:21:13.748 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3608 2023-09-12T16:21:13.748 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 3609 2023-09-12T16:21:13.749 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3712 2023-09-12T16:21:13.749 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764 2023-09-12T16:21:13.749 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3764_fsm 2023-09-12T16:21:13.749 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764_vm 2023-09-12T16:21:13.749 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3766 2023-09-12T16:21:13.749 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3767 2023-09-12T16:21:13.750 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3997 2023-09-12T16:21:13.750 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 5002 2023-09-12T16:21:13.750 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 548 2023-09-12T16:21:13.750 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 549 2023-09-12T16:21:13.750 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102 2023-09-12T16:21:13.751 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102_vm 2023-09-12T16:21:13.751 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104 2023-09-12T16:21:13.751 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104_vm 2023-09-12T16:21:13.751 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106 2023-09-12T16:21:13.751 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106_vm 2023-09-12T16:21:13.751 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6110 2023-09-12T16:21:13.752 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6111 2023-09-12T16:21:13.752 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6112 2023-09-12T16:21:13.752 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6113 2023-09-12T16:21:13.752 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6117 2023-09-12T16:21:13.752 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826 2023-09-12T16:21:13.752 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826_vm 2023-09-12T16:21:13.753 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 827 2023-09-12T16:21:13.753 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 828 2023-09-12T16:21:13.753 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 12 16:03 pg_filenode.map 2023-09-12T16:21:13.753 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 PG_VERSION 2023-09-12T16:21:13.753 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.753 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-09-12T16:21:13.754 INFO:teuthology.orchestra.run.smithi130.stdout:total 7209 2023-09-12T16:21:13.754 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 12 16:03 . 2023-09-12T16:21:13.754 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 12 16:05 .. 2023-09-12T16:21:13.754 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 112 2023-09-12T16:21:13.754 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 113 2023-09-12T16:21:13.755 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 12 16:03 12182 2023-09-12T16:21:13.755 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12182_fsm 2023-09-12T16:21:13.755 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12182_vm 2023-09-12T16:21:13.755 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12184 2023-09-12T16:21:13.755 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12186 2023-09-12T16:21:13.755 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187 2023-09-12T16:21:13.756 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12187_fsm 2023-09-12T16:21:13.756 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187_vm 2023-09-12T16:21:13.756 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12189 2023-09-12T16:21:13.756 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12191 2023-09-12T16:21:13.756 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192 2023-09-12T16:21:13.756 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12192_fsm 2023-09-12T16:21:13.757 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192_vm 2023-09-12T16:21:13.757 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12194 2023-09-12T16:21:13.757 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12196 2023-09-12T16:21:13.757 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197 2023-09-12T16:21:13.757 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12197_fsm 2023-09-12T16:21:13.757 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197_vm 2023-09-12T16:21:13.758 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12199 2023-09-12T16:21:13.758 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12201 2023-09-12T16:21:13.758 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202 2023-09-12T16:21:13.758 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12202_fsm 2023-09-12T16:21:13.758 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202_vm 2023-09-12T16:21:13.759 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12204 2023-09-12T16:21:13.759 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12206 2023-09-12T16:21:13.759 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207 2023-09-12T16:21:13.759 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12207_fsm 2023-09-12T16:21:13.759 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207_vm 2023-09-12T16:21:13.759 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12209 2023-09-12T16:21:13.760 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12211 2023-09-12T16:21:13.760 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12212 2023-09-12T16:21:13.760 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12214 2023-09-12T16:21:13.760 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12216 2023-09-12T16:21:13.760 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 12 16:03 1247 2023-09-12T16:21:13.760 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1247_fsm 2023-09-12T16:21:13.761 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1247_vm 2023-09-12T16:21:13.761 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 12 16:03 1249 2023-09-12T16:21:13.761 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1249_fsm 2023-09-12T16:21:13.761 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1249_vm 2023-09-12T16:21:13.761 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 12 16:03 1255 2023-09-12T16:21:13.761 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1255_fsm 2023-09-12T16:21:13.762 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1255_vm 2023-09-12T16:21:13.762 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 12 16:03 1259 2023-09-12T16:21:13.762 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1259_fsm 2023-09-12T16:21:13.762 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1259_vm 2023-09-12T16:21:13.762 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417 2023-09-12T16:21:13.763 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417_vm 2023-09-12T16:21:13.763 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418 2023-09-12T16:21:13.763 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418_vm 2023-09-12T16:21:13.763 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 174 2023-09-12T16:21:13.763 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 175 2023-09-12T16:21:13.763 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2187 2023-09-12T16:21:13.764 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224 2023-09-12T16:21:13.764 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224_vm 2023-09-12T16:21:13.764 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328 2023-09-12T16:21:13.764 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328_vm 2023-09-12T16:21:13.764 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336 2023-09-12T16:21:13.764 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336_vm 2023-09-12T16:21:13.765 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2337 2023-09-12T16:21:13.765 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2600 2023-09-12T16:21:13.765 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2600_fsm 2023-09-12T16:21:13.765 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2600_vm 2023-09-12T16:21:13.765 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601 2023-09-12T16:21:13.765 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2601_fsm 2023-09-12T16:21:13.766 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601_vm 2023-09-12T16:21:13.766 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2602 2023-09-12T16:21:13.766 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2602_fsm 2023-09-12T16:21:13.766 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2602_vm 2023-09-12T16:21:13.766 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2603 2023-09-12T16:21:13.766 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2603_fsm 2023-09-12T16:21:13.767 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2603_vm 2023-09-12T16:21:13.767 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604 2023-09-12T16:21:13.767 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604_vm 2023-09-12T16:21:13.767 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2605 2023-09-12T16:21:13.767 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2605_fsm 2023-09-12T16:21:13.767 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2605_vm 2023-09-12T16:21:13.768 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606 2023-09-12T16:21:13.768 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2606_fsm 2023-09-12T16:21:13.768 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2606_vm 2023-09-12T16:21:13.768 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607 2023-09-12T16:21:13.768 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607_fsm 2023-09-12T16:21:13.768 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2607_vm 2023-09-12T16:21:13.769 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 12 16:03 2608 2023-09-12T16:21:13.769 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2608_fsm 2023-09-12T16:21:13.769 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2608_vm 2023-09-12T16:21:13.769 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 12 16:03 2609 2023-09-12T16:21:13.769 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2609_fsm 2023-09-12T16:21:13.769 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2609_vm 2023-09-12T16:21:13.770 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610 2023-09-12T16:21:13.770 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2610_fsm 2023-09-12T16:21:13.770 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2610_vm 2023-09-12T16:21:13.770 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611 2023-09-12T16:21:13.770 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611_vm 2023-09-12T16:21:13.771 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612 2023-09-12T16:21:13.771 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2612_fsm 2023-09-12T16:21:13.771 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612_vm 2023-09-12T16:21:13.771 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613 2023-09-12T16:21:13.771 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613_vm 2023-09-12T16:21:13.771 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615 2023-09-12T16:21:13.772 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2615_fsm 2023-09-12T16:21:13.772 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615_vm 2023-09-12T16:21:13.772 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616 2023-09-12T16:21:13.772 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616_fsm 2023-09-12T16:21:13.772 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2616_vm 2023-09-12T16:21:13.772 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 12 16:03 2617 2023-09-12T16:21:13.773 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2617_fsm 2023-09-12T16:21:13.773 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2617_vm 2023-09-12T16:21:13.773 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 12 16:03 2618 2023-09-12T16:21:13.773 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2618_fsm 2023-09-12T16:21:13.773 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2618_vm 2023-09-12T16:21:13.773 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 12 16:03 2619 2023-09-12T16:21:13.774 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2619_fsm 2023-09-12T16:21:13.774 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2619_vm 2023-09-12T16:21:13.774 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620 2023-09-12T16:21:13.774 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620_vm 2023-09-12T16:21:13.774 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2650 2023-09-12T16:21:13.775 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2651 2023-09-12T16:21:13.775 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2652 2023-09-12T16:21:13.775 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2653 2023-09-12T16:21:13.775 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2654 2023-09-12T16:21:13.775 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2655 2023-09-12T16:21:13.775 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2656 2023-09-12T16:21:13.776 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2657 2023-09-12T16:21:13.776 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 12 16:03 2658 2023-09-12T16:21:13.776 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2659 2023-09-12T16:21:13.776 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2660 2023-09-12T16:21:13.776 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2661 2023-09-12T16:21:13.776 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2662 2023-09-12T16:21:13.777 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2663 2023-09-12T16:21:13.777 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2664 2023-09-12T16:21:13.777 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2665 2023-09-12T16:21:13.777 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2666 2023-09-12T16:21:13.777 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2667 2023-09-12T16:21:13.777 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2668 2023-09-12T16:21:13.778 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2669 2023-09-12T16:21:13.778 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2670 2023-09-12T16:21:13.778 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Sep 12 16:03 2673 2023-09-12T16:21:13.778 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 12 16:03 2674 2023-09-12T16:21:13.778 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 12 16:03 2675 2023-09-12T16:21:13.779 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2678 2023-09-12T16:21:13.779 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2679 2023-09-12T16:21:13.779 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2680 2023-09-12T16:21:13.779 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2681 2023-09-12T16:21:13.779 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2682 2023-09-12T16:21:13.779 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2683 2023-09-12T16:21:13.780 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2684 2023-09-12T16:21:13.780 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2685 2023-09-12T16:21:13.780 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2686 2023-09-12T16:21:13.780 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2687 2023-09-12T16:21:13.780 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2688 2023-09-12T16:21:13.780 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2689 2023-09-12T16:21:13.781 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2690 2023-09-12T16:21:13.781 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 12 16:03 2691 2023-09-12T16:21:13.781 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2692 2023-09-12T16:21:13.781 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2693 2023-09-12T16:21:13.781 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2696 2023-09-12T16:21:13.781 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2699 2023-09-12T16:21:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2701 2023-09-12T16:21:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2702 2023-09-12T16:21:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2703 2023-09-12T16:21:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2704 2023-09-12T16:21:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2753 2023-09-12T16:21:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2753_fsm 2023-09-12T16:21:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2753_vm 2023-09-12T16:21:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2754 2023-09-12T16:21:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2755 2023-09-12T16:21:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2756 2023-09-12T16:21:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2757 2023-09-12T16:21:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830 2023-09-12T16:21:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830_vm 2023-09-12T16:21:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2831 2023-09-12T16:21:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832 2023-09-12T16:21:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832_vm 2023-09-12T16:21:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2833 2023-09-12T16:21:13.785 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834 2023-09-12T16:21:13.785 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834_vm 2023-09-12T16:21:13.785 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2835 2023-09-12T16:21:13.785 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836 2023-09-12T16:21:13.785 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836_vm 2023-09-12T16:21:13.785 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2837 2023-09-12T16:21:13.786 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 12 16:03 2838 2023-09-12T16:21:13.786 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2838_fsm 2023-09-12T16:21:13.786 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2838_vm 2023-09-12T16:21:13.786 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2839 2023-09-12T16:21:13.786 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840 2023-09-12T16:21:13.786 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840_fsm 2023-09-12T16:21:13.787 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2840_vm 2023-09-12T16:21:13.787 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2841 2023-09-12T16:21:13.787 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995 2023-09-12T16:21:13.787 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995_vm 2023-09-12T16:21:13.787 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2996 2023-09-12T16:21:13.787 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079 2023-09-12T16:21:13.788 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3079_fsm 2023-09-12T16:21:13.788 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079_vm 2023-09-12T16:21:13.788 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3080 2023-09-12T16:21:13.788 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3081 2023-09-12T16:21:13.788 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3085 2023-09-12T16:21:13.788 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118 2023-09-12T16:21:13.789 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118_vm 2023-09-12T16:21:13.789 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3119 2023-09-12T16:21:13.789 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3164 2023-09-12T16:21:13.789 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256 2023-09-12T16:21:13.789 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256_vm 2023-09-12T16:21:13.790 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3257 2023-09-12T16:21:13.790 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3258 2023-09-12T16:21:13.790 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350 2023-09-12T16:21:13.790 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350_vm 2023-09-12T16:21:13.790 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3351 2023-09-12T16:21:13.790 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3379 2023-09-12T16:21:13.791 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3380 2023-09-12T16:21:13.791 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381 2023-09-12T16:21:13.791 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381_vm 2023-09-12T16:21:13.791 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3394 2023-09-12T16:21:13.791 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3394_fsm 2023-09-12T16:21:13.791 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3394_vm 2023-09-12T16:21:13.792 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3395 2023-09-12T16:21:13.792 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439 2023-09-12T16:21:13.792 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439_vm 2023-09-12T16:21:13.792 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3440 2023-09-12T16:21:13.792 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 3455 2023-09-12T16:21:13.793 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3456 2023-09-12T16:21:13.793 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3456_fsm 2023-09-12T16:21:13.793 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3456_vm 2023-09-12T16:21:13.793 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466 2023-09-12T16:21:13.793 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466_vm 2023-09-12T16:21:13.793 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3467 2023-09-12T16:21:13.794 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3468 2023-09-12T16:21:13.794 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501 2023-09-12T16:21:13.794 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501_vm 2023-09-12T16:21:13.794 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3502 2023-09-12T16:21:13.794 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3503 2023-09-12T16:21:13.794 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3534 2023-09-12T16:21:13.795 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541 2023-09-12T16:21:13.795 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3541_fsm 2023-09-12T16:21:13.795 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541_vm 2023-09-12T16:21:13.795 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3542 2023-09-12T16:21:13.795 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3574 2023-09-12T16:21:13.795 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3575 2023-09-12T16:21:13.796 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576 2023-09-12T16:21:13.796 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576_vm 2023-09-12T16:21:13.796 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596 2023-09-12T16:21:13.796 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596_vm 2023-09-12T16:21:13.796 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3597 2023-09-12T16:21:13.796 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598 2023-09-12T16:21:13.797 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598_vm 2023-09-12T16:21:13.797 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3599 2023-09-12T16:21:13.797 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600 2023-09-12T16:21:13.797 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3600_fsm 2023-09-12T16:21:13.797 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600_vm 2023-09-12T16:21:13.797 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601 2023-09-12T16:21:13.798 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3601_fsm 2023-09-12T16:21:13.798 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601_vm 2023-09-12T16:21:13.798 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602 2023-09-12T16:21:13.798 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3602_fsm 2023-09-12T16:21:13.798 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602_vm 2023-09-12T16:21:13.798 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3603 2023-09-12T16:21:13.799 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3603_fsm 2023-09-12T16:21:13.799 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3603_vm 2023-09-12T16:21:13.799 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3604 2023-09-12T16:21:13.799 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3605 2023-09-12T16:21:13.799 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3606 2023-09-12T16:21:13.799 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3607 2023-09-12T16:21:13.800 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3608 2023-09-12T16:21:13.800 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 3609 2023-09-12T16:21:13.800 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3712 2023-09-12T16:21:13.800 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764 2023-09-12T16:21:13.800 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3764_fsm 2023-09-12T16:21:13.801 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764_vm 2023-09-12T16:21:13.801 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3766 2023-09-12T16:21:13.801 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3767 2023-09-12T16:21:13.801 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3997 2023-09-12T16:21:13.801 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 5002 2023-09-12T16:21:13.801 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 548 2023-09-12T16:21:13.802 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 549 2023-09-12T16:21:13.802 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102 2023-09-12T16:21:13.802 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102_vm 2023-09-12T16:21:13.802 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104 2023-09-12T16:21:13.802 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104_vm 2023-09-12T16:21:13.803 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106 2023-09-12T16:21:13.803 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106_vm 2023-09-12T16:21:13.803 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6110 2023-09-12T16:21:13.803 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6111 2023-09-12T16:21:13.803 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6112 2023-09-12T16:21:13.804 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6113 2023-09-12T16:21:13.804 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6117 2023-09-12T16:21:13.804 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826 2023-09-12T16:21:13.804 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826_vm 2023-09-12T16:21:13.804 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 827 2023-09-12T16:21:13.804 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 828 2023-09-12T16:21:13.805 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 12 16:03 pg_filenode.map 2023-09-12T16:21:13.805 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 PG_VERSION 2023-09-12T16:21:13.805 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.805 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-09-12T16:21:13.805 INFO:teuthology.orchestra.run.smithi130.stdout:total 7802568 2023-09-12T16:21:13.805 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 315 Sep 12 16:06 . 2023-09-12T16:21:13.806 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 12 16:05 .. 2023-09-12T16:21:13.806 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 112 2023-09-12T16:21:13.806 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 113 2023-09-12T16:21:13.806 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 12 16:03 12182 2023-09-12T16:21:13.806 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12182_fsm 2023-09-12T16:21:13.806 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12182_vm 2023-09-12T16:21:13.807 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12184 2023-09-12T16:21:13.807 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12186 2023-09-12T16:21:13.807 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187 2023-09-12T16:21:13.807 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12187_fsm 2023-09-12T16:21:13.807 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12187_vm 2023-09-12T16:21:13.807 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12189 2023-09-12T16:21:13.808 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12191 2023-09-12T16:21:13.808 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192 2023-09-12T16:21:13.808 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12192_fsm 2023-09-12T16:21:13.808 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12192_vm 2023-09-12T16:21:13.808 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12194 2023-09-12T16:21:13.808 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12196 2023-09-12T16:21:13.809 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197 2023-09-12T16:21:13.809 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12197_fsm 2023-09-12T16:21:13.809 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12197_vm 2023-09-12T16:21:13.809 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12199 2023-09-12T16:21:13.809 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12201 2023-09-12T16:21:13.810 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202 2023-09-12T16:21:13.810 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12202_fsm 2023-09-12T16:21:13.810 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12202_vm 2023-09-12T16:21:13.810 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12204 2023-09-12T16:21:13.810 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12206 2023-09-12T16:21:13.810 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207 2023-09-12T16:21:13.811 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 12207_fsm 2023-09-12T16:21:13.811 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12207_vm 2023-09-12T16:21:13.811 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12209 2023-09-12T16:21:13.811 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12211 2023-09-12T16:21:13.811 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12212 2023-09-12T16:21:13.812 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 12214 2023-09-12T16:21:13.812 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 12216 2023-09-12T16:21:13.812 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 12 16:04 1247 2023-09-12T16:21:13.812 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:04 1247_fsm 2023-09-12T16:21:13.812 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:04 1247_vm 2023-09-12T16:21:13.812 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 12 16:05 1249 2023-09-12T16:21:13.813 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:04 1249_fsm 2023-09-12T16:21:13.813 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:04 1249_vm 2023-09-12T16:21:13.813 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 12 16:03 1255 2023-09-12T16:21:13.813 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1255_fsm 2023-09-12T16:21:13.813 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1255_vm 2023-09-12T16:21:13.814 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 12 16:20 1259 2023-09-12T16:21:13.814 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1259_fsm 2023-09-12T16:21:13.814 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:04 1259_vm 2023-09-12T16:21:13.814 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417 2023-09-12T16:21:13.814 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1417_vm 2023-09-12T16:21:13.815 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418 2023-09-12T16:21:13.815 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 1418_vm 2023-09-12T16:21:13.815 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1318912 Sep 12 16:20 16387 2023-09-12T16:21:13.815 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:20 16387_fsm 2023-09-12T16:21:13.815 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:20 16387_vm 2023-09-12T16:21:13.815 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2310144 Sep 12 16:20 16393 2023-09-12T16:21:13.816 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:20 16393_fsm 2023-09-12T16:21:13.816 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:20 16393_vm 2023-09-12T16:21:13.816 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 12 16:20 16396 2023-09-12T16:21:13.816 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 12 16:20 16396.1 2023-09-12T16:21:13.816 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 12 16:20 16396.2 2023-09-12T16:21:13.816 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 12 16:20 16396.3 2023-09-12T16:21:13.817 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 12 16:20 16396.4 2023-09-12T16:21:13.817 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 12 16:20 16396.5 2023-09-12T16:21:13.817 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 354156544 Sep 12 16:20 16396.6 2023-09-12T16:21:13.817 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1687552 Sep 12 16:20 16396_fsm 2023-09-12T16:21:13.817 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Sep 12 16:20 16396_vm 2023-09-12T16:21:13.818 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 12 16:20 16397 2023-09-12T16:21:13.818 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 245760 Sep 12 16:20 16399 2023-09-12T16:21:13.818 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 12 16:20 16401 2023-09-12T16:21:13.818 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49373184 Sep 12 16:20 16401.1 2023-09-12T16:21:13.818 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 56565760 Sep 12 16:20 16403 2023-09-12T16:21:13.819 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:20 16403_fsm 2023-09-12T16:21:13.819 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 174 2023-09-12T16:21:13.819 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 175 2023-09-12T16:21:13.819 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2187 2023-09-12T16:21:13.819 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224 2023-09-12T16:21:13.819 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2224_vm 2023-09-12T16:21:13.820 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328 2023-09-12T16:21:13.820 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2328_vm 2023-09-12T16:21:13.820 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336 2023-09-12T16:21:13.820 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2336_vm 2023-09-12T16:21:13.820 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2337 2023-09-12T16:21:13.820 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2600 2023-09-12T16:21:13.821 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2600_fsm 2023-09-12T16:21:13.821 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2600_vm 2023-09-12T16:21:13.821 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601 2023-09-12T16:21:13.821 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2601_fsm 2023-09-12T16:21:13.821 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2601_vm 2023-09-12T16:21:13.822 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2602 2023-09-12T16:21:13.822 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2602_fsm 2023-09-12T16:21:13.822 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2602_vm 2023-09-12T16:21:13.822 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2603 2023-09-12T16:21:13.822 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2603_fsm 2023-09-12T16:21:13.822 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2603_vm 2023-09-12T16:21:13.823 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604 2023-09-12T16:21:13.823 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2604_vm 2023-09-12T16:21:13.823 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2605 2023-09-12T16:21:13.823 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2605_fsm 2023-09-12T16:21:13.823 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2605_vm 2023-09-12T16:21:13.824 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:05 2606 2023-09-12T16:21:13.824 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2606_fsm 2023-09-12T16:21:13.824 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:05 2606_vm 2023-09-12T16:21:13.824 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607 2023-09-12T16:21:13.824 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2607_fsm 2023-09-12T16:21:13.824 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2607_vm 2023-09-12T16:21:13.825 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Sep 12 16:05 2608 2023-09-12T16:21:13.825 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:05 2608_fsm 2023-09-12T16:21:13.825 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:05 2608_vm 2023-09-12T16:21:13.825 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 12 16:03 2609 2023-09-12T16:21:13.825 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2609_fsm 2023-09-12T16:21:13.826 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2609_vm 2023-09-12T16:21:13.826 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:05 2610 2023-09-12T16:21:13.826 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:05 2610_fsm 2023-09-12T16:21:13.826 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:05 2610_vm 2023-09-12T16:21:13.826 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611 2023-09-12T16:21:13.826 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2611_vm 2023-09-12T16:21:13.827 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612 2023-09-12T16:21:13.827 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2612_fsm 2023-09-12T16:21:13.827 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2612_vm 2023-09-12T16:21:13.827 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613 2023-09-12T16:21:13.827 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2613_vm 2023-09-12T16:21:13.828 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615 2023-09-12T16:21:13.828 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2615_fsm 2023-09-12T16:21:13.828 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2615_vm 2023-09-12T16:21:13.828 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616 2023-09-12T16:21:13.828 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2616_fsm 2023-09-12T16:21:13.828 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2616_vm 2023-09-12T16:21:13.829 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 12 16:03 2617 2023-09-12T16:21:13.829 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2617_fsm 2023-09-12T16:21:13.829 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2617_vm 2023-09-12T16:21:13.829 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 12 16:03 2618 2023-09-12T16:21:13.829 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2618_fsm 2023-09-12T16:21:13.829 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2618_vm 2023-09-12T16:21:13.830 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 12 16:20 2619 2023-09-12T16:21:13.830 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:11 2619_fsm 2023-09-12T16:21:13.830 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:05 2619_vm 2023-09-12T16:21:13.830 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620 2023-09-12T16:21:13.830 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2620_vm 2023-09-12T16:21:13.831 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2650 2023-09-12T16:21:13.831 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2651 2023-09-12T16:21:13.831 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2652 2023-09-12T16:21:13.831 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2653 2023-09-12T16:21:13.831 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2654 2023-09-12T16:21:13.832 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2655 2023-09-12T16:21:13.832 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2656 2023-09-12T16:21:13.832 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2657 2023-09-12T16:21:13.832 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 12 16:05 2658 2023-09-12T16:21:13.832 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:05 2659 2023-09-12T16:21:13.832 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2660 2023-09-12T16:21:13.833 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2661 2023-09-12T16:21:13.833 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:05 2662 2023-09-12T16:21:13.833 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:05 2663 2023-09-12T16:21:13.833 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:05 2664 2023-09-12T16:21:13.833 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:05 2665 2023-09-12T16:21:13.833 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:05 2666 2023-09-12T16:21:13.834 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:05 2667 2023-09-12T16:21:13.834 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2668 2023-09-12T16:21:13.834 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2669 2023-09-12T16:21:13.834 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2670 2023-09-12T16:21:13.834 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Sep 12 16:05 2673 2023-09-12T16:21:13.835 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 12 16:05 2674 2023-09-12T16:21:13.835 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 12 16:03 2675 2023-09-12T16:21:13.835 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:05 2678 2023-09-12T16:21:13.835 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:05 2679 2023-09-12T16:21:13.835 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2680 2023-09-12T16:21:13.835 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2681 2023-09-12T16:21:13.836 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2682 2023-09-12T16:21:13.836 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2683 2023-09-12T16:21:13.836 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2684 2023-09-12T16:21:13.836 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2685 2023-09-12T16:21:13.836 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2686 2023-09-12T16:21:13.836 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2687 2023-09-12T16:21:13.837 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:03 2688 2023-09-12T16:21:13.837 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 12 16:03 2689 2023-09-12T16:21:13.837 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 12 16:03 2690 2023-09-12T16:21:13.837 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 12 16:03 2691 2023-09-12T16:21:13.837 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2692 2023-09-12T16:21:13.837 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2693 2023-09-12T16:21:13.838 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:11 2696 2023-09-12T16:21:13.838 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2699 2023-09-12T16:21:13.838 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2701 2023-09-12T16:21:13.838 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2702 2023-09-12T16:21:13.838 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:04 2703 2023-09-12T16:21:13.838 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:04 2704 2023-09-12T16:21:13.839 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2753 2023-09-12T16:21:13.839 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2753_fsm 2023-09-12T16:21:13.839 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2753_vm 2023-09-12T16:21:13.839 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2754 2023-09-12T16:21:13.839 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2755 2023-09-12T16:21:13.840 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2756 2023-09-12T16:21:13.840 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 2757 2023-09-12T16:21:13.840 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830 2023-09-12T16:21:13.840 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2830_vm 2023-09-12T16:21:13.840 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2831 2023-09-12T16:21:13.840 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832 2023-09-12T16:21:13.841 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2832_vm 2023-09-12T16:21:13.841 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2833 2023-09-12T16:21:13.841 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834 2023-09-12T16:21:13.841 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2834_vm 2023-09-12T16:21:13.841 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2835 2023-09-12T16:21:13.841 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836 2023-09-12T16:21:13.842 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2836_vm 2023-09-12T16:21:13.842 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2837 2023-09-12T16:21:13.842 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 12 16:03 2838 2023-09-12T16:21:13.842 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2838_fsm 2023-09-12T16:21:13.842 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2838_vm 2023-09-12T16:21:13.843 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2839 2023-09-12T16:21:13.843 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840 2023-09-12T16:21:13.843 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2840_fsm 2023-09-12T16:21:13.843 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2840_vm 2023-09-12T16:21:13.843 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2841 2023-09-12T16:21:13.843 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995 2023-09-12T16:21:13.844 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2995_vm 2023-09-12T16:21:13.844 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2996 2023-09-12T16:21:13.844 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079 2023-09-12T16:21:13.844 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3079_fsm 2023-09-12T16:21:13.844 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3079_vm 2023-09-12T16:21:13.844 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3080 2023-09-12T16:21:13.845 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3081 2023-09-12T16:21:13.845 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3085 2023-09-12T16:21:13.845 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118 2023-09-12T16:21:13.845 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3118_vm 2023-09-12T16:21:13.845 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3119 2023-09-12T16:21:13.845 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3164 2023-09-12T16:21:13.846 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256 2023-09-12T16:21:13.846 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3256_vm 2023-09-12T16:21:13.846 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3257 2023-09-12T16:21:13.846 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3258 2023-09-12T16:21:13.846 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350 2023-09-12T16:21:13.846 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3350_vm 2023-09-12T16:21:13.847 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3351 2023-09-12T16:21:13.847 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3379 2023-09-12T16:21:13.847 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3380 2023-09-12T16:21:13.847 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381 2023-09-12T16:21:13.847 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3381_vm 2023-09-12T16:21:13.847 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3394 2023-09-12T16:21:13.848 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3394_fsm 2023-09-12T16:21:13.848 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3394_vm 2023-09-12T16:21:13.848 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3395 2023-09-12T16:21:13.848 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439 2023-09-12T16:21:13.848 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3439_vm 2023-09-12T16:21:13.848 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3440 2023-09-12T16:21:13.849 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 16:05 3455 2023-09-12T16:21:13.849 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3456 2023-09-12T16:21:13.849 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3456_fsm 2023-09-12T16:21:13.849 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3456_vm 2023-09-12T16:21:13.849 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466 2023-09-12T16:21:13.849 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3466_vm 2023-09-12T16:21:13.850 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3467 2023-09-12T16:21:13.850 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3468 2023-09-12T16:21:13.850 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501 2023-09-12T16:21:13.850 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3501_vm 2023-09-12T16:21:13.850 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3502 2023-09-12T16:21:13.851 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3503 2023-09-12T16:21:13.851 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3534 2023-09-12T16:21:13.851 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541 2023-09-12T16:21:13.851 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3541_fsm 2023-09-12T16:21:13.851 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3541_vm 2023-09-12T16:21:13.851 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3542 2023-09-12T16:21:13.852 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3574 2023-09-12T16:21:13.852 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3575 2023-09-12T16:21:13.852 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576 2023-09-12T16:21:13.852 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3576_vm 2023-09-12T16:21:13.852 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596 2023-09-12T16:21:13.853 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3596_vm 2023-09-12T16:21:13.853 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3597 2023-09-12T16:21:13.853 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598 2023-09-12T16:21:13.853 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3598_vm 2023-09-12T16:21:13.853 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3599 2023-09-12T16:21:13.853 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600 2023-09-12T16:21:13.854 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3600_fsm 2023-09-12T16:21:13.854 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3600_vm 2023-09-12T16:21:13.854 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601 2023-09-12T16:21:13.854 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3601_fsm 2023-09-12T16:21:13.854 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3601_vm 2023-09-12T16:21:13.855 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602 2023-09-12T16:21:13.855 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3602_fsm 2023-09-12T16:21:13.855 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3602_vm 2023-09-12T16:21:13.855 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3603 2023-09-12T16:21:13.855 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3603_fsm 2023-09-12T16:21:13.855 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3603_vm 2023-09-12T16:21:13.856 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3604 2023-09-12T16:21:13.856 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3605 2023-09-12T16:21:13.856 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3606 2023-09-12T16:21:13.856 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3607 2023-09-12T16:21:13.856 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3608 2023-09-12T16:21:13.857 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 12 16:03 3609 2023-09-12T16:21:13.857 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3712 2023-09-12T16:21:13.857 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764 2023-09-12T16:21:13.857 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 3764_fsm 2023-09-12T16:21:13.857 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3764_vm 2023-09-12T16:21:13.857 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3766 2023-09-12T16:21:13.858 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 3767 2023-09-12T16:21:13.858 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3997 2023-09-12T16:21:13.858 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 5002 2023-09-12T16:21:13.858 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 548 2023-09-12T16:21:13.858 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 549 2023-09-12T16:21:13.858 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102 2023-09-12T16:21:13.859 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6102_vm 2023-09-12T16:21:13.859 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104 2023-09-12T16:21:13.859 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6104_vm 2023-09-12T16:21:13.859 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106 2023-09-12T16:21:13.859 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6106_vm 2023-09-12T16:21:13.860 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6110 2023-09-12T16:21:13.860 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6111 2023-09-12T16:21:13.860 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6112 2023-09-12T16:21:13.860 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6113 2023-09-12T16:21:13.860 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6117 2023-09-12T16:21:13.860 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826 2023-09-12T16:21:13.861 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 826_vm 2023-09-12T16:21:13.861 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 827 2023-09-12T16:21:13.861 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 828 2023-09-12T16:21:13.861 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 12 16:03 pg_filenode.map 2023-09-12T16:21:13.861 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Sep 12 16:03 pg_internal.init 2023-09-12T16:21:13.861 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:03 PG_VERSION 2023-09-12T16:21:13.862 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.862 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-09-12T16:21:13.862 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.862 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:05 . 2023-09-12T16:21:13.862 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 12 16:05 .. 2023-09-12T16:21:13.863 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.863 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/global: 2023-09-12T16:21:13.863 INFO:teuthology.orchestra.run.smithi130.stdout:total 574 2023-09-12T16:21:13.863 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Sep 12 16:03 . 2023-09-12T16:21:13.863 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.863 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1136 2023-09-12T16:21:13.864 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1136_fsm 2023-09-12T16:21:13.864 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1136_vm 2023-09-12T16:21:13.864 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 1137 2023-09-12T16:21:13.864 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1213 2023-09-12T16:21:13.864 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1213_fsm 2023-09-12T16:21:13.865 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1213_vm 2023-09-12T16:21:13.865 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1214 2023-09-12T16:21:13.865 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1214_fsm 2023-09-12T16:21:13.865 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1214_vm 2023-09-12T16:21:13.865 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 1232 2023-09-12T16:21:13.865 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 1233 2023-09-12T16:21:13.866 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1260 2023-09-12T16:21:13.866 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1260_fsm 2023-09-12T16:21:13.866 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1260_vm 2023-09-12T16:21:13.866 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1261 2023-09-12T16:21:13.866 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1261_fsm 2023-09-12T16:21:13.866 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1261_vm 2023-09-12T16:21:13.867 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1262 2023-09-12T16:21:13.867 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 1262_fsm 2023-09-12T16:21:13.867 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 1262_vm 2023-09-12T16:21:13.867 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2396 2023-09-12T16:21:13.867 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 16:03 2396_fsm 2023-09-12T16:21:13.868 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2396_vm 2023-09-12T16:21:13.868 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2397 2023-09-12T16:21:13.868 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2671 2023-09-12T16:21:13.868 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2672 2023-09-12T16:21:13.868 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2676 2023-09-12T16:21:13.868 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2677 2023-09-12T16:21:13.869 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2694 2023-09-12T16:21:13.869 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2695 2023-09-12T16:21:13.869 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2697 2023-09-12T16:21:13.869 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 16:03 2698 2023-09-12T16:21:13.869 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2846 2023-09-12T16:21:13.870 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2846_vm 2023-09-12T16:21:13.870 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2847 2023-09-12T16:21:13.870 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2964 2023-09-12T16:21:13.870 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2964_vm 2023-09-12T16:21:13.870 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2965 2023-09-12T16:21:13.870 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2966 2023-09-12T16:21:13.871 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 2966_vm 2023-09-12T16:21:13.871 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 2967 2023-09-12T16:21:13.871 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3592 2023-09-12T16:21:13.871 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 3592_vm 2023-09-12T16:21:13.871 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 3593 2023-09-12T16:21:13.872 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 4060 2023-09-12T16:21:13.872 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 4060_vm 2023-09-12T16:21:13.872 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 4061 2023-09-12T16:21:13.872 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6000 2023-09-12T16:21:13.872 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6000_vm 2023-09-12T16:21:13.872 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6001 2023-09-12T16:21:13.873 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6002 2023-09-12T16:21:13.873 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6100 2023-09-12T16:21:13.873 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 6100_vm 2023-09-12T16:21:13.873 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6114 2023-09-12T16:21:13.873 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 6115 2023-09-12T16:21:13.873 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:21 pg_control 2023-09-12T16:21:13.874 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 12 16:03 pg_filenode.map 2023-09-12T16:21:13.874 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Sep 12 16:03 pg_internal.init 2023-09-12T16:21:13.874 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.874 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/log: 2023-09-12T16:21:13.874 INFO:teuthology.orchestra.run.smithi130.stdout:total 7 2023-09-12T16:21:13.875 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:21:13.875 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.875 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 6487 Sep 12 16:21 postgresql-Tue.log 2023-09-12T16:21:13.875 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.875 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-09-12T16:21:13.875 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.875 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.876 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.876 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.876 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-09-12T16:21:13.876 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.876 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.876 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.877 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.877 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-09-12T16:21:13.877 INFO:teuthology.orchestra.run.smithi130.stdout:total 1 2023-09-12T16:21:13.877 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:21 . 2023-09-12T16:21:13.877 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.878 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 mappings 2023-09-12T16:21:13.878 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 12 16:21 replorigin_checkpoint 2023-09-12T16:21:13.878 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 snapshots 2023-09-12T16:21:13.878 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.878 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-09-12T16:21:13.878 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.879 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.879 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:21 .. 2023-09-12T16:21:13.879 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.879 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-09-12T16:21:13.879 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.879 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.880 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:21 .. 2023-09-12T16:21:13.880 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.880 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-09-12T16:21:13.880 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.880 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 . 2023-09-12T16:21:13.881 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.881 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 members 2023-09-12T16:21:13.881 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 offsets 2023-09-12T16:21:13.881 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.881 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-09-12T16:21:13.881 INFO:teuthology.orchestra.run.smithi130.stdout:total 8 2023-09-12T16:21:13.882 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:21:13.882 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 .. 2023-09-12T16:21:13.882 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 0000 2023-09-12T16:21:13.882 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.882 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-09-12T16:21:13.882 INFO:teuthology.orchestra.run.smithi130.stdout:total 8 2023-09-12T16:21:13.883 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:21:13.883 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:03 .. 2023-09-12T16:21:13.883 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:04 0000 2023-09-12T16:21:13.883 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.883 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-09-12T16:21:13.883 INFO:teuthology.orchestra.run.smithi130.stdout:total 8 2023-09-12T16:21:13.884 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:03 . 2023-09-12T16:21:13.884 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.884 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:03 0000 2023-09-12T16:21:13.884 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.884 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-09-12T16:21:13.885 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.885 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.885 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.885 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.885 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-09-12T16:21:13.885 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.886 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.886 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.886 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.886 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-09-12T16:21:13.886 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.886 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.887 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.887 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.887 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-09-12T16:21:13.887 INFO:teuthology.orchestra.run.smithi130.stdout:total 13 2023-09-12T16:21:13.887 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 16:21 . 2023-09-12T16:21:13.888 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.888 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Sep 12 16:21 db_0.stat 2023-09-12T16:21:13.888 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Sep 12 16:21 db_12347.stat 2023-09-12T16:21:13.888 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Sep 12 16:21 global.stat 2023-09-12T16:21:13.888 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.888 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-09-12T16:21:13.889 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.889 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:21 . 2023-09-12T16:21:13.889 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.889 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.889 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-09-12T16:21:13.889 INFO:teuthology.orchestra.run.smithi130.stdout:total 104 2023-09-12T16:21:13.890 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 16:20 . 2023-09-12T16:21:13.890 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.890 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 12 16:20 0010 2023-09-12T16:21:13.890 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.890 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-09-12T16:21:13.891 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.891 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.891 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.891 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.891 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-09-12T16:21:13.891 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.892 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.892 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.892 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.892 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-09-12T16:21:13.892 INFO:teuthology.orchestra.run.smithi130.stdout:total 1048576 2023-09-12T16:21:13.893 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 12 16:21 . 2023-09-12T16:21:13.893 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.893 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 000000010000000400000098 2023-09-12T16:21:13.893 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 000000010000000400000099 2023-09-12T16:21:13.893 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 00000001000000040000009A 2023-09-12T16:21:13.893 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 00000001000000040000009B 2023-09-12T16:21:13.894 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:21 00000001000000040000009C 2023-09-12T16:21:13.894 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 00000001000000040000009D 2023-09-12T16:21:13.894 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:18 00000001000000040000009E 2023-09-12T16:21:13.894 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 00000001000000040000009F 2023-09-12T16:21:13.894 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A0 2023-09-12T16:21:13.895 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A1 2023-09-12T16:21:13.895 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A2 2023-09-12T16:21:13.895 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A3 2023-09-12T16:21:13.895 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A4 2023-09-12T16:21:13.895 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A5 2023-09-12T16:21:13.895 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A6 2023-09-12T16:21:13.896 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A7 2023-09-12T16:21:13.896 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A8 2023-09-12T16:21:13.896 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000A9 2023-09-12T16:21:13.896 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000AA 2023-09-12T16:21:13.896 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000AB 2023-09-12T16:21:13.896 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000AC 2023-09-12T16:21:13.897 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000AD 2023-09-12T16:21:13.897 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000AE 2023-09-12T16:21:13.897 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000AF 2023-09-12T16:21:13.897 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000B0 2023-09-12T16:21:13.897 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000B1 2023-09-12T16:21:13.898 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000B2 2023-09-12T16:21:13.898 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000B3 2023-09-12T16:21:13.898 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000B4 2023-09-12T16:21:13.898 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000B5 2023-09-12T16:21:13.898 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000B6 2023-09-12T16:21:13.898 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000B7 2023-09-12T16:21:13.899 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000B8 2023-09-12T16:21:13.899 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000B9 2023-09-12T16:21:13.899 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000BA 2023-09-12T16:21:13.899 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000BB 2023-09-12T16:21:13.899 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000BC 2023-09-12T16:21:13.900 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:19 0000000100000004000000BD 2023-09-12T16:21:13.900 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000BE 2023-09-12T16:21:13.900 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000BF 2023-09-12T16:21:13.900 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C0 2023-09-12T16:21:13.900 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C1 2023-09-12T16:21:13.900 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C2 2023-09-12T16:21:13.900 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C3 2023-09-12T16:21:13.901 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C4 2023-09-12T16:21:13.901 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C5 2023-09-12T16:21:13.901 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C6 2023-09-12T16:21:13.901 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C7 2023-09-12T16:21:13.901 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C8 2023-09-12T16:21:13.902 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000C9 2023-09-12T16:21:13.902 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000CA 2023-09-12T16:21:13.902 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000CB 2023-09-12T16:21:13.902 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000CC 2023-09-12T16:21:13.902 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000CD 2023-09-12T16:21:13.903 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000CE 2023-09-12T16:21:13.903 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000CF 2023-09-12T16:21:13.903 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000D0 2023-09-12T16:21:13.903 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000D1 2023-09-12T16:21:13.903 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000D2 2023-09-12T16:21:13.903 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000D3 2023-09-12T16:21:13.904 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000D4 2023-09-12T16:21:13.904 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000D5 2023-09-12T16:21:13.904 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000D6 2023-09-12T16:21:13.904 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 16:20 0000000100000004000000D7 2023-09-12T16:21:13.904 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 archive_status 2023-09-12T16:21:13.904 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.905 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-09-12T16:21:13.905 INFO:teuthology.orchestra.run.smithi130.stdout:total 0 2023-09-12T16:21:13.905 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 16:03 . 2023-09-12T16:21:13.905 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 12 16:21 .. 2023-09-12T16:21:13.905 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:13.906 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-09-12T16:21:13.906 INFO:teuthology.orchestra.run.smithi130.stdout:total 264 2023-09-12T16:21:13.906 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 16:20 . 2023-09-12T16:21:13.906 INFO:teuthology.orchestra.run.smithi130.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 16:21 .. 2023-09-12T16:21:13.906 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 12 16:20 0000 2023-09-12T16:21:13.906 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 16:20 0001 2023-09-12T16:21:13.907 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-09-12T16:21:13.917 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-09-12T16:21:13.918 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-09-12T16:21:13.918 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-09-12T16:21:13.929 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:21:14.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 -- 172.21.15.130:0/723738573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e8101080 msgr2=0x7fd0e8107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/723738573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e8101080 0x7fd0e8107930 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000b0d0 tx=0x7fd0e000b3e0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 -- 172.21.15.130:0/723738573 shutdown_connections 2023-09-12T16:21:14.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/723738573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e8101080 0x7fd0e8107930 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/723738573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e80ff4f0 0x7fd0e80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/723738573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0e81006e0 0x7fd0e8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 -- 172.21.15.130:0/723738573 >> 172.21.15.130:0/723738573 conn(0x7fd0e80faa60 msgr2=0x7fd0e80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:14.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 -- 172.21.15.130:0/723738573 shutdown_connections 2023-09-12T16:21:14.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 -- 172.21.15.130:0/723738573 wait complete. 2023-09-12T16:21:14.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 Processor -- start 2023-09-12T16:21:14.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.072+0000 7fd0eed3f700 1 -- start start 2023-09-12T16:21:14.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0eed3f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0e80ff4f0 0x7fd0e8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0eed3f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e81006e0 0x7fd0e819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0eed3f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e8101080 0x7fd0e819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0eed3f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e819df20 con 0x7fd0e81006e0 2023-09-12T16:21:14.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0eed3f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e819e090 con 0x7fd0e80ff4f0 2023-09-12T16:21:14.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0eed3f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e819e250 con 0x7fd0e8101080 2023-09-12T16:21:14.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e7fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e81006e0 0x7fd0e819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0ecadb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0e80ff4f0 0x7fd0e8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e7fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e81006e0 0x7fd0e819b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56870/0 (socket says 172.21.15.130:56870) 2023-09-12T16:21:14.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0ecadb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0e80ff4f0 0x7fd0e8069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40126/0 (socket says 172.21.15.130:40126) 2023-09-12T16:21:14.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0ed2dc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e8101080 0x7fd0e819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.078 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e7fff700 1 -- 172.21.15.130:0/1843855168 learned_addr learned my addr 172.21.15.130:0/1843855168 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:14.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e7fff700 1 -- 172.21.15.130:0/1843855168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e8101080 msgr2=0x7fd0e819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e7fff700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e8101080 0x7fd0e819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e7fff700 1 -- 172.21.15.130:0/1843855168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0e80ff4f0 msgr2=0x7fd0e8069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e7fff700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0e80ff4f0 0x7fd0e8069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e7fff700 1 -- 172.21.15.130:0/1843855168 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e0009d00 con 0x7fd0e81006e0 2023-09-12T16:21:14.079 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e7fff700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e81006e0 0x7fd0e819b100 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400e9a0 tx=0x7fd0d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:14.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e5ffb700 1 -- 172.21.15.130:0/1843855168 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4009790 con 0x7fd0e81006e0 2023-09-12T16:21:14.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e819e530 con 0x7fd0e81006e0 2023-09-12T16:21:14.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e5ffb700 1 -- 172.21.15.130:0/1843855168 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0d4010430 con 0x7fd0e81006e0 2023-09-12T16:21:14.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.073+0000 7fd0e5ffb700 1 -- 172.21.15.130:0/1843855168 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4018680 con 0x7fd0e81006e0 2023-09-12T16:21:14.080 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.074+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e819ea90 con 0x7fd0e81006e0 2023-09-12T16:21:14.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.074+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e8106380 con 0x7fd0e81006e0 2023-09-12T16:21:14.081 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.075+0000 7fd0e5ffb700 1 -- 172.21.15.130:0/1843855168 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fd0d40105a0 con 0x7fd0e81006e0 2023-09-12T16:21:14.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.076+0000 7fd0e5ffb700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0d80a0690 0x7fd0d80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.076+0000 7fd0ecadb700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0d80a0690 0x7fd0d80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.076+0000 7fd0e5ffb700 1 -- 172.21.15.130:0/1843855168 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd0d4014070 con 0x7fd0e81006e0 2023-09-12T16:21:14.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.076+0000 7fd0ecadb700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0d80a0690 0x7fd0d80a2b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8100540 tx=0x7fd0dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:14.082 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.080+0000 7fd0e5ffb700 1 -- 172.21.15.130:0/1843855168 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd0d4095a50 con 0x7fd0e81006e0 2023-09-12T16:21:14.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:13 smithi130 ceph-mon[98670]: pgmap v841: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.3 MiB/s wr, 546 op/s 2023-09-12T16:21:14.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:13 smithi159 ceph-mon[102522]: pgmap v841: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.3 MiB/s wr, 546 op/s 2023-09-12T16:21:14.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:13 smithi203 ceph-mon[103431]: pgmap v841: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.3 MiB/s wr, 546 op/s 2023-09-12T16:21:14.275 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.274+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0e810aee0 con 0x7fd0e81006e0 2023-09-12T16:21:14.276 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.276+0000 7fd0e5ffb700 1 -- 172.21.15.130:0/1843855168 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24671 (secure 0 0 0) 0x7fd0d4026090 con 0x7fd0e81006e0 2023-09-12T16:21:14.277 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:14.277 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":90,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:03:51.888444+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:21:14.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0d80a0690 msgr2=0x7fd0d80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0d80a0690 0x7fd0d80a2b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8100540 tx=0x7fd0dc009260 comp rx=0 tx=0).stop 2023-09-12T16:21:14.279 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e81006e0 msgr2=0x7fd0e819b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e81006e0 0x7fd0e819b100 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400e9a0 tx=0x7fd0d400ecb0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 shutdown_connections 2023-09-12T16:21:14.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fd0d80a0690 0x7fd0d80a2b50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e8101080 0x7fd0e819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.280 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0e80ff4f0 0x7fd0e8069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 --2- 172.21.15.130:0/1843855168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e81006e0 0x7fd0e819b100 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 >> 172.21.15.130:0/1843855168 conn(0x7fd0e80faa60 msgr2=0x7fd0e80fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:14.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 shutdown_connections 2023-09-12T16:21:14.281 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.278+0000 7fd0eed3f700 1 -- 172.21.15.130:0/1843855168 wait complete. 2023-09-12T16:21:14.292 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:21:14.293 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:21:14.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.416+0000 7f3281781700 1 -- 172.21.15.130:0/2787871890 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 msgr2=0x7f327c0ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.416+0000 7f3281781700 1 --2- 172.21.15.130:0/2787871890 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 0x7f327c0ffc40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f326c009990 tx=0x7f326c009ca0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.422 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 -- 172.21.15.130:0/2787871890 shutdown_connections 2023-09-12T16:21:14.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 --2- 172.21.15.130:0/2787871890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327c100180 0x7f327c107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 --2- 172.21.15.130:0/2787871890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f327c100ab0 0x7f327c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 --2- 172.21.15.130:0/2787871890 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 0x7f327c0ffc40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.423 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 -- 172.21.15.130:0/2787871890 >> 172.21.15.130:0/2787871890 conn(0x7f327c0fa1f0 msgr2=0x7f327c0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:14.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 -- 172.21.15.130:0/2787871890 shutdown_connections 2023-09-12T16:21:14.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 -- 172.21.15.130:0/2787871890 wait complete. 2023-09-12T16:21:14.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 Processor -- start 2023-09-12T16:21:14.424 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 -- start start 2023-09-12T16:21:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 0x7f327c0ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327c100180 0x7f327c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f327c100ab0 0x7f327c19f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1a0000 con 0x7f327c0ff7e0 2023-09-12T16:21:14.425 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1a0170 con 0x7f327c100180 2023-09-12T16:21:14.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.417+0000 7f3281781700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1a0330 con 0x7f327c100ab0 2023-09-12T16:21:14.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 0x7f327c0ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.426 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327affd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 0x7f327c0ff240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56882/0 (socket says 172.21.15.130:56882) 2023-09-12T16:21:14.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327affd700 1 -- 172.21.15.130:0/843293157 learned_addr learned my addr 172.21.15.130:0/843293157 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:14.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327b7fe700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f327c100ab0 0x7f327c19f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327a7fc700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327c100180 0x7f327c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.427 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327affd700 1 -- 172.21.15.130:0/843293157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f327c100ab0 msgr2=0x7f327c19f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327affd700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f327c100ab0 0x7f327c19f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327affd700 1 -- 172.21.15.130:0/843293157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327c100180 msgr2=0x7f327c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327affd700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327c100180 0x7f327c19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327affd700 1 -- 172.21.15.130:0/843293157 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c0095f0 con 0x7f327c0ff7e0 2023-09-12T16:21:14.428 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f327affd700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 0x7f327c0ff240 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f327000e970 tx=0x7f327000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f3263fff700 1 -- 172.21.15.130:0/843293157 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270009790 con 0x7f327c0ff7e0 2023-09-12T16:21:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f3263fff700 1 -- 172.21.15.130:0/843293157 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f327000c7e0 con 0x7f327c0ff7e0 2023-09-12T16:21:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c1a0610 con 0x7f327c0ff7e0 2023-09-12T16:21:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f3263fff700 1 -- 172.21.15.130:0/843293157 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270003c70 con 0x7f327c0ff7e0 2023-09-12T16:21:14.429 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.418+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c1a0b20 con 0x7f327c0ff7e0 2023-09-12T16:21:14.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.419+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c106ba0 con 0x7f327c0ff7e0 2023-09-12T16:21:14.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.420+0000 7f3263fff700 1 -- 172.21.15.130:0/843293157 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f327000cc50 con 0x7f327c0ff7e0 2023-09-12T16:21:14.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.421+0000 7f3263fff700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32640a0640 0x7f32640a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.430 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.421+0000 7f3263fff700 1 -- 172.21.15.130:0/843293157 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3270014070 con 0x7f327c0ff7e0 2023-09-12T16:21:14.431 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.421+0000 7f327a7fc700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32640a0640 0x7f32640a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.431 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.421+0000 7f327a7fc700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32640a0640 0x7f32640a2b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f326c00b2e0 tx=0x7f326c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:14.431 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.424+0000 7f3263fff700 1 -- 172.21.15.130:0/843293157 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3270095a10 con 0x7f327c0ff7e0 2023-09-12T16:21:14.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.614+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f327c10aeb0 con 0x7f327c0ff7e0 2023-09-12T16:21:14.615 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.615+0000 7f3263fff700 1 -- 172.21.15.130:0/843293157 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3270003dd0 con 0x7f327c0ff7e0 2023-09-12T16:21:14.616 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:21:14.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32640a0640 msgr2=0x7f32640a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32640a0640 0x7f32640a2b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f326c00b2e0 tx=0x7f326c005fb0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 msgr2=0x7f327c0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 0x7f327c0ff240 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f327000e970 tx=0x7f327000ec80 comp rx=0 tx=0).stop 2023-09-12T16:21:14.618 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 shutdown_connections 2023-09-12T16:21:14.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f32640a0640 0x7f32640a2b00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f327c100ab0 0x7f327c19f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f327c0ff7e0 0x7f327c0ff240 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 --2- 172.21.15.130:0/843293157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327c100180 0x7f327c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.619 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 >> 172.21.15.130:0/843293157 conn(0x7f327c0fa1f0 msgr2=0x7f327c103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:14.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 shutdown_connections 2023-09-12T16:21:14.620 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.617+0000 7f3281781700 1 -- 172.21.15.130:0/843293157 wait complete. 2023-09-12T16:21:14.631 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:21:14.759 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.756+0000 7fe5079cd700 1 -- 172.21.15.130:0/4242195885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5001010b0 msgr2=0x7fe500107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.756+0000 7fe5079cd700 1 --2- 172.21.15.130:0/4242195885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5001010b0 0x7fe500107960 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800b0d0 tx=0x7fe4f800b3e0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.756+0000 7fe5079cd700 1 -- 172.21.15.130:0/4242195885 shutdown_connections 2023-09-12T16:21:14.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.756+0000 7fe5079cd700 1 --2- 172.21.15.130:0/4242195885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5001010b0 0x7fe500107960 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.760 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.756+0000 7fe5079cd700 1 --2- 172.21.15.130:0/4242195885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5000ff520 0x7fe5000ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.756+0000 7fe5079cd700 1 --2- 172.21.15.130:0/4242195885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe500100710 0x7fe500100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.756+0000 7fe5079cd700 1 -- 172.21.15.130:0/4242195885 >> 172.21.15.130:0/4242195885 conn(0x7fe5000faa90 msgr2=0x7fe5000fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:14.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.756+0000 7fe5079cd700 1 -- 172.21.15.130:0/4242195885 shutdown_connections 2023-09-12T16:21:14.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.756+0000 7fe5079cd700 1 -- 172.21.15.130:0/4242195885 wait complete. 2023-09-12T16:21:14.761 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe5079cd700 1 Processor -- start 2023-09-12T16:21:14.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe5079cd700 1 -- start start 2023-09-12T16:21:14.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe5079cd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5000ff520 0x7fe5001963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe5079cd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe500100710 0x7fe500196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe5079cd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5001010b0 0x7fe50019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.762 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe5079cd700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50019df60 con 0x7fe5000ff520 2023-09-12T16:21:14.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe5079cd700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50019e0a0 con 0x7fe500100710 2023-09-12T16:21:14.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe5079cd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50019e230 con 0x7fe5001010b0 2023-09-12T16:21:14.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe505769700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5000ff520 0x7fe5001963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe505769700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5000ff520 0x7fe5001963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56896/0 (socket says 172.21.15.130:56896) 2023-09-12T16:21:14.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe505769700 1 -- 172.21.15.130:0/2599812154 learned_addr learned my addr 172.21.15.130:0/2599812154 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:14.763 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe504f68700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe500100710 0x7fe500196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.757+0000 7fe505f6a700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5001010b0 0x7fe50019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe505769700 1 -- 172.21.15.130:0/2599812154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5001010b0 msgr2=0x7fe50019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe505769700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5001010b0 0x7fe50019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe505769700 1 -- 172.21.15.130:0/2599812154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe500100710 msgr2=0x7fe500196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.764 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe505769700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe500100710 0x7fe500196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe505769700 1 -- 172.21.15.130:0/2599812154 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f8009d00 con 0x7fe5000ff520 2023-09-12T16:21:14.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe504f68700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe500100710 0x7fe500196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:14.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe505769700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5000ff520 0x7fe5001963e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000e970 tx=0x7fe4f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:14.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe4f67fc700 1 -- 172.21.15.130:0/2599812154 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0009790 con 0x7fe5000ff520 2023-09-12T16:21:14.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe4f67fc700 1 -- 172.21.15.130:0/2599812154 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4f000c7e0 con 0x7fe5000ff520 2023-09-12T16:21:14.765 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50019e510 con 0x7fe5000ff520 2023-09-12T16:21:14.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe4f67fc700 1 -- 172.21.15.130:0/2599812154 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0003c70 con 0x7fe5000ff520 2023-09-12T16:21:14.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.758+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50019ea20 con 0x7fe5000ff520 2023-09-12T16:21:14.766 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.759+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50010ad50 con 0x7fe5000ff520 2023-09-12T16:21:14.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.760+0000 7fe4f67fc700 1 -- 172.21.15.130:0/2599812154 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fe4f000cc50 con 0x7fe5000ff520 2023-09-12T16:21:14.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.760+0000 7fe4f67fc700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4ec0a0640 0x7fe4ec0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:14.767 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.760+0000 7fe4f67fc700 1 -- 172.21.15.130:0/2599812154 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe4f0014070 con 0x7fe5000ff520 2023-09-12T16:21:14.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.760+0000 7fe504f68700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4ec0a0640 0x7fe4ec0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:14.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.761+0000 7fe504f68700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4ec0a0640 0x7fe4ec0a2b00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc011490 tx=0x7fe4fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:14.768 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.765+0000 7fe4f67fc700 1 -- 172.21.15.130:0/2599812154 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe4f0095890 con 0x7fe5000ff520 2023-09-12T16:21:14.959 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.959+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe50019eca0 con 0x7fe5000ff520 2023-09-12T16:21:14.960 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.959+0000 7fe4f67fc700 1 -- 172.21.15.130:0/2599812154 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4f0003dd0 con 0x7fe5000ff520 2023-09-12T16:21:14.960 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:21:14.962 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4ec0a0640 msgr2=0x7fe4ec0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4ec0a0640 0x7fe4ec0a2b00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc011490 tx=0x7fe4fc009260 comp rx=0 tx=0).stop 2023-09-12T16:21:14.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5000ff520 msgr2=0x7fe5001963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:14.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5000ff520 0x7fe5001963e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000e970 tx=0x7fe4f000ec80 comp rx=0 tx=0).stop 2023-09-12T16:21:14.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 shutdown_connections 2023-09-12T16:21:14.963 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe4ec0a0640 0x7fe4ec0a2b00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5001010b0 0x7fe50019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5000ff520 0x7fe5001963e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 --2- 172.21.15.130:0/2599812154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe500100710 0x7fe500196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:14.964 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 >> 172.21.15.130:0/2599812154 conn(0x7fe5000faa90 msgr2=0x7fe500068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:14.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 shutdown_connections 2023-09-12T16:21:14.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:14.961+0000 7fe5079cd700 1 -- 172.21.15.130:0/2599812154 wait complete. 2023-09-12T16:21:14.975 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:21:15.107 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.103+0000 7fec562a5700 1 -- 172.21.15.130:0/2038604392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 msgr2=0x7fec50105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.103+0000 7fec562a5700 1 --2- 172.21.15.130:0/2038604392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 0x7fec50105e20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fec44009990 tx=0x7fec44009ca0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.103+0000 7fec562a5700 1 -- 172.21.15.130:0/2038604392 shutdown_connections 2023-09-12T16:21:15.108 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.103+0000 7fec562a5700 1 --2- 172.21.15.130:0/2038604392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec50106360 0x7fec50108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.103+0000 7fec562a5700 1 --2- 172.21.15.130:0/2038604392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec50101100 0x7fec501034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.103+0000 7fec562a5700 1 --2- 172.21.15.130:0/2038604392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 0x7fec50105e20 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.109 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.103+0000 7fec562a5700 1 -- 172.21.15.130:0/2038604392 >> 172.21.15.130:0/2038604392 conn(0x7fec500faa90 msgr2=0x7fec500fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:15.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 -- 172.21.15.130:0/2038604392 shutdown_connections 2023-09-12T16:21:15.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 -- 172.21.15.130:0/2038604392 wait complete. 2023-09-12T16:21:15.110 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 Processor -- start 2023-09-12T16:21:15.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 -- start start 2023-09-12T16:21:15.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec50101100 0x7fec50198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.111 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 0x7fec50198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec50106360 0x7fec5019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.112 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec501a00c0 con 0x7fec50103a30 2023-09-12T16:21:15.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec501a0200 con 0x7fec50101100 2023-09-12T16:21:15.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.104+0000 7fec562a5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec501a0390 con 0x7fec50106360 2023-09-12T16:21:15.113 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 0x7fec50198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec50101100 0x7fec50198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 0x7fec50198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56912/0 (socket says 172.21.15.130:56912) 2023-09-12T16:21:15.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4f7fe700 1 -- 172.21.15.130:0/519079611 learned_addr learned my addr 172.21.15.130:0/519079611 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:15.114 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec54842700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec50106360 0x7fec5019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4ffff700 1 -- 172.21.15.130:0/519079611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec50106360 msgr2=0x7fec5019faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4ffff700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec50106360 0x7fec5019faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.115 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4ffff700 1 -- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 msgr2=0x7fec50198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4ffff700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 0x7fec50198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4ffff700 1 -- 172.21.15.130:0/519079611 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440095f0 con 0x7fec50101100 2023-09-12T16:21:15.116 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4f7fe700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 0x7fec50198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:15.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4ffff700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec50101100 0x7fec50198540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e9f0 tx=0x7fec4000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:15.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4d7fa700 1 -- 172.21.15.130:0/519079611 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4000c9c0 con 0x7fec50101100 2023-09-12T16:21:15.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec501a0670 con 0x7fec50101100 2023-09-12T16:21:15.117 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4d7fa700 1 -- 172.21.15.130:0/519079611 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec40010430 con 0x7fec50101100 2023-09-12T16:21:15.118 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec4d7fa700 1 -- 172.21.15.130:0/519079611 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40018680 con 0x7fec50101100 2023-09-12T16:21:15.123 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.105+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec501a0b80 con 0x7fec50101100 2023-09-12T16:21:15.124 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.106+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec50107f00 con 0x7fec50101100 2023-09-12T16:21:15.125 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.107+0000 7fec4d7fa700 1 -- 172.21.15.130:0/519079611 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fec40010860 con 0x7fec50101100 2023-09-12T16:21:15.126 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.108+0000 7fec4d7fa700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fec380a0640 0x7fec380a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.108+0000 7fec4f7fe700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fec380a0640 0x7fec380a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.108+0000 7fec4d7fa700 1 -- 172.21.15.130:0/519079611 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fec40014070 con 0x7fec50101100 2023-09-12T16:21:15.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.108+0000 7fec4f7fe700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fec380a0640 0x7fec380a2b00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fec4400b2e0 tx=0x7fec44005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:15.127 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.111+0000 7fec4d7fa700 1 -- 172.21.15.130:0/519079611 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fec400959d0 con 0x7fec50101100 2023-09-12T16:21:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:14 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1843855168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:21:15.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:14 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/843293157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:21:15.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:14 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1843855168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:21:15.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:14 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/843293157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:21:15.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:14 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1843855168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:21:15.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:14 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/843293157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:21:15.303 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.302+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec5010c180 con 0x7fec50101100 2023-09-12T16:21:15.304 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.303+0000 7fec4d7fa700 1 -- 172.21.15.130:0/519079611 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec40098ff0 con 0x7fec50101100 2023-09-12T16:21:15.304 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:21:15.306 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fec380a0640 msgr2=0x7fec380a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fec380a0640 0x7fec380a2b00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fec4400b2e0 tx=0x7fec44005fb0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.307 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec50101100 msgr2=0x7fec50198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec50101100 0x7fec50198540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e9f0 tx=0x7fec4000ed00 comp rx=0 tx=0).stop 2023-09-12T16:21:15.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 shutdown_connections 2023-09-12T16:21:15.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fec380a0640 0x7fec380a2b00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec50106360 0x7fec5019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.308 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec50101100 0x7fec50198540 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 --2- 172.21.15.130:0/519079611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec50103a30 0x7fec50198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 >> 172.21.15.130:0/519079611 conn(0x7fec500faa90 msgr2=0x7fec500ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:15.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 shutdown_connections 2023-09-12T16:21:15.309 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.305+0000 7fec562a5700 1 -- 172.21.15.130:0/519079611 wait complete. 2023-09-12T16:21:15.319 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:21:15.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.443+0000 7f7adc03c700 1 -- 172.21.15.130:0/4171554083 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4106330 msgr2=0x7f7ad4108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.443+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4171554083 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4106330 0x7f7ad4108740 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009f90 tx=0x7f7acc00b330 comp rx=0 tx=0).stop 2023-09-12T16:21:15.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 -- 172.21.15.130:0/4171554083 shutdown_connections 2023-09-12T16:21:15.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4171554083 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4106330 0x7f7ad4108740 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4171554083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ad41010d0 0x7f7ad41034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4171554083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad4103a00 0x7f7ad4105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 -- 172.21.15.130:0/4171554083 >> 172.21.15.130:0/4171554083 conn(0x7f7ad40faa60 msgr2=0x7f7ad40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:15.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 -- 172.21.15.130:0/4171554083 shutdown_connections 2023-09-12T16:21:15.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 -- 172.21.15.130:0/4171554083 wait complete. 2023-09-12T16:21:15.450 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 Processor -- start 2023-09-12T16:21:15.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 -- start start 2023-09-12T16:21:15.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ad41010d0 0x7f7ad4196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4103a00 0x7f7ad4196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.451 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad4106330 0x7f7ad419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.444+0000 7f7adc03c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad419dfb0 con 0x7f7ad4103a00 2023-09-12T16:21:15.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7adc03c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad419e0f0 con 0x7f7ad41010d0 2023-09-12T16:21:15.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7adc03c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad419e280 con 0x7f7ad4106330 2023-09-12T16:21:15.452 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad95d7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4103a00 0x7f7ad4196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad95d7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4103a00 0x7f7ad4196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56926/0 (socket says 172.21.15.130:56926) 2023-09-12T16:21:15.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad95d7700 1 -- 172.21.15.130:0/4237592491 learned_addr learned my addr 172.21.15.130:0/4237592491 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:15.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ada5d9700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad4106330 0x7f7ad419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.453 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad9dd8700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ad41010d0 0x7f7ad4196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad95d7700 1 -- 172.21.15.130:0/4237592491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad4106330 msgr2=0x7f7ad419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad95d7700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad4106330 0x7f7ad419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad95d7700 1 -- 172.21.15.130:0/4237592491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ad41010d0 msgr2=0x7f7ad4196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.454 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad95d7700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ad41010d0 0x7f7ad4196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad95d7700 1 -- 172.21.15.130:0/4237592491 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc009c40 con 0x7f7ad4103a00 2023-09-12T16:21:15.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ada5d9700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad4106330 0x7f7ad419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:15.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7ad95d7700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4103a00 0x7f7ad4196970 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e9c0 tx=0x7f7ac400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:15.455 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7acaffd700 1 -- 172.21.15.130:0/4237592491 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4004a30 con 0x7f7ad4103a00 2023-09-12T16:21:15.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad419e560 con 0x7f7ad4103a00 2023-09-12T16:21:15.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7acaffd700 1 -- 172.21.15.130:0/4237592491 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ac400cc80 con 0x7f7ad4103a00 2023-09-12T16:21:15.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7acaffd700 1 -- 172.21.15.130:0/4237592491 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40054e0 con 0x7f7ad4103a00 2023-09-12T16:21:15.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.445+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad419ea70 con 0x7f7ad4103a00 2023-09-12T16:21:15.456 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.446+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad40fc5e0 con 0x7f7ad4103a00 2023-09-12T16:21:15.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.448+0000 7f7acaffd700 1 -- 172.21.15.130:0/4237592491 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f7ac4004b90 con 0x7f7ad4103a00 2023-09-12T16:21:15.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.448+0000 7f7acaffd700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7ac00a0640 0x7f7ac00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.458 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.449+0000 7f7ad9dd8700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7ac00a0640 0x7f7ac00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.449+0000 7f7acaffd700 1 -- 172.21.15.130:0/4237592491 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7ac4014070 con 0x7f7ad4103a00 2023-09-12T16:21:15.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.449+0000 7f7ad9dd8700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7ac00a0640 0x7f7ac00a2b00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009c60 tx=0x7f7ad0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:15.459 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.452+0000 7f7acaffd700 1 -- 172.21.15.130:0/4237592491 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7ac4095940 con 0x7f7ad4103a00 2023-09-12T16:21:15.646 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.645+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7ad410c090 con 0x7f7ad4103a00 2023-09-12T16:21:15.647 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.646+0000 7f7acaffd700 1 -- 172.21.15.130:0/4237592491 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7ac4098f60 con 0x7f7ad4103a00 2023-09-12T16:21:15.647 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:21:15.649 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7ac00a0640 msgr2=0x7f7ac00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7ac00a0640 0x7f7ac00a2b00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009c60 tx=0x7f7ad0009260 comp rx=0 tx=0).stop 2023-09-12T16:21:15.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4103a00 msgr2=0x7f7ad4196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.650 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4103a00 0x7f7ad4196970 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e9c0 tx=0x7f7ac400ecd0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 shutdown_connections 2023-09-12T16:21:15.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7ac00a0640 0x7f7ac00a2b00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad4106330 0x7f7ad419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.651 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ad41010d0 0x7f7ad4196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 --2- 172.21.15.130:0/4237592491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ad4103a00 0x7f7ad4196970 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 >> 172.21.15.130:0/4237592491 conn(0x7f7ad40faa60 msgr2=0x7f7ad4101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:15.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 shutdown_connections 2023-09-12T16:21:15.652 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.648+0000 7f7adc03c700 1 -- 172.21.15.130:0/4237592491 wait complete. 2023-09-12T16:21:15.662 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-09-12T16:21:15.786 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.783+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/55607216 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc8100600 msgr2=0x7f1fc8100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.783+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/55607216 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc8100600 0x7f1fc8100a20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1fc40099c0 tx=0x7f1fc4009cd0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.783+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/55607216 shutdown_connections 2023-09-12T16:21:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.783+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/55607216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc8102190 0x7f1fc8108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.783+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/55607216 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc8100600 0x7f1fc8100a20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.783+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/55607216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc81017f0 0x7f1fc8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.783+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/55607216 >> 172.21.15.130:0/55607216 conn(0x7f1fc80fbb50 msgr2=0x7f1fc80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.783+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/55607216 shutdown_connections 2023-09-12T16:21:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/55607216 wait complete. 2023-09-12T16:21:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcfa8f700 1 Processor -- start 2023-09-12T16:21:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcfa8f700 1 -- start start 2023-09-12T16:21:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcfa8f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc8100600 0x7f1fc81962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcfa8f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc81017f0 0x7f1fc8196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcfa8f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc8102190 0x7f1fc819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcfa8f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc819de40 con 0x7f1fc81017f0 2023-09-12T16:21:15.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcfa8f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc819df80 con 0x7f1fc8100600 2023-09-12T16:21:15.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcfa8f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc819e110 con 0x7f1fc8102190 2023-09-12T16:21:15.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcd02a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc81017f0 0x7f1fc8196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcd02a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc81017f0 0x7f1fc8196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56940/0 (socket says 172.21.15.130:56940) 2023-09-12T16:21:15.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcd02a700 1 -- 172.21.15.130:0/1254178117 learned_addr learned my addr 172.21.15.130:0/1254178117 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:15.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fcd82b700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc8100600 0x7f1fc81962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.784+0000 7f1fce02c700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc8102190 0x7f1fc819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fcd02a700 1 -- 172.21.15.130:0/1254178117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc8102190 msgr2=0x7f1fc819d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fcd02a700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc8102190 0x7f1fc819d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fcd02a700 1 -- 172.21.15.130:0/1254178117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc8100600 msgr2=0x7f1fc81962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:15.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fcd02a700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc8100600 0x7f1fc81962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:15.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fcd02a700 1 -- 172.21.15.130:0/1254178117 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc40095f0 con 0x7f1fc81017f0 2023-09-12T16:21:15.791 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fcd02a700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc81017f0 0x7f1fc8196800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00e9a0 tx=0x7f1fbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:15.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fbaffd700 1 -- 172.21.15.130:0/1254178117 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc009790 con 0x7f1fc81017f0 2023-09-12T16:21:15.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc819e3f0 con 0x7f1fc81017f0 2023-09-12T16:21:15.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fbaffd700 1 -- 172.21.15.130:0/1254178117 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fbc010430 con 0x7f1fc81017f0 2023-09-12T16:21:15.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fbaffd700 1 -- 172.21.15.130:0/1254178117 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc018680 con 0x7f1fc81017f0 2023-09-12T16:21:15.792 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.785+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc819e900 con 0x7f1fc81017f0 2023-09-12T16:21:15.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.786+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc8107ba0 con 0x7f1fc81017f0 2023-09-12T16:21:15.793 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.787+0000 7f1fbaffd700 1 -- 172.21.15.130:0/1254178117 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f1fbc0105a0 con 0x7f1fc81017f0 2023-09-12T16:21:15.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.787+0000 7f1fbaffd700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fb40a0690 0x7f1fb40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:15.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.788+0000 7f1fcd82b700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fb40a0690 0x7f1fb40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:15.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.788+0000 7f1fbaffd700 1 -- 172.21.15.130:0/1254178117 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1fbc014070 con 0x7f1fc81017f0 2023-09-12T16:21:15.794 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.788+0000 7f1fcd82b700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fb40a0690 0x7f1fb40a2b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009990 tx=0x7f1fc4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:15.795 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:15.791+0000 7f1fbaffd700 1 -- 172.21.15.130:0/1254178117 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1fbc095a50 con 0x7f1fc81017f0 2023-09-12T16:21:15.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:15 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2599812154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:21:15.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:15 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/519079611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:21:15.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:15 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/4237592491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:21:15.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:15 smithi130 ceph-mon[98670]: pgmap v842: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.2 MiB/s wr, 419 op/s 2023-09-12T16:21:16.052 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.051+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f1fc810c000 con 0x7f1fc81017f0 2023-09-12T16:21:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.058+0000 7f1fbaffd700 1 -- 172.21.15.130:0/1254178117 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+729902 (secure 0 0 0) 0x7f1fbc026090 con 0x7f1fc81017f0 2023-09-12T16:21:16.071 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:16.071 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:21:16.072 INFO:teuthology.orchestra.run.smithi130.stdout: "epoch": 18, 2023-09-12T16:21:16.072 INFO:teuthology.orchestra.run.smithi130.stdout: "active_gid": 14150, 2023-09-12T16:21:16.072 INFO:teuthology.orchestra.run.smithi130.stdout: "active_name": "x", 2023-09-12T16:21:16.073 INFO:teuthology.orchestra.run.smithi130.stdout: "active_addrs": { 2023-09-12T16:21:16.073 INFO:teuthology.orchestra.run.smithi130.stdout: "addrvec": [ 2023-09-12T16:21:16.073 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.074 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v2", 2023-09-12T16:21:16.074 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:6800", 2023-09-12T16:21:16.074 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 4273925772 2023-09-12T16:21:16.074 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.074 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.075 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2023-09-12T16:21:16.075 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:6801", 2023-09-12T16:21:16.075 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 4273925772 2023-09-12T16:21:16.075 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.075 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T16:21:16.076 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.076 INFO:teuthology.orchestra.run.smithi130.stdout: "active_addr": "172.21.15.130:6801/4273925772", 2023-09-12T16:21:16.076 INFO:teuthology.orchestra.run.smithi130.stdout: "active_change": "2023-09-12T15:55:50.680354+0000", 2023-09-12T16:21:16.076 INFO:teuthology.orchestra.run.smithi130.stdout: "active_mgr_features": 4540138322906710015, 2023-09-12T16:21:16.076 INFO:teuthology.orchestra.run.smithi130.stdout: "available": true, 2023-09-12T16:21:16.077 INFO:teuthology.orchestra.run.smithi130.stdout: "standbys": [ 2023-09-12T16:21:16.077 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.077 INFO:teuthology.orchestra.run.smithi130.stdout: "gid": 24194, 2023-09-12T16:21:16.077 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "y", 2023-09-12T16:21:16.077 INFO:teuthology.orchestra.run.smithi130.stdout: "mgr_features": 4540138322906710015, 2023-09-12T16:21:16.078 INFO:teuthology.orchestra.run.smithi130.stdout: "available_modules": [ 2023-09-12T16:21:16.078 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.078 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "alerts", 2023-09-12T16:21:16.078 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.078 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.079 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.079 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:16.079 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:16.079 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.080 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.080 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.080 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:16.080 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.080 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.081 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.081 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How frequently to reexamine health status", 2023-09-12T16:21:16.081 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.081 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.081 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.082 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.082 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.082 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.082 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.082 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.083 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.083 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.083 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.083 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.083 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.084 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.084 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.084 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.084 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.085 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.085 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.085 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.085 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.085 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.085 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.086 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.086 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.086 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.086 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.087 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.087 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.087 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.087 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.087 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.087 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.088 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.088 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.088 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.088 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.089 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.089 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.089 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.089 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.090 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.090 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.090 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.090 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.090 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.091 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.091 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.091 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.091 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.091 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.092 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.092 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.092 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.092 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.092 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.093 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.093 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.093 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.093 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.093 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.093 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.094 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.094 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.094 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.094 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.095 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.095 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.095 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.095 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.095 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.096 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.096 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.096 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.096 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_destination": { 2023-09-12T16:21:16.096 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_destination", 2023-09-12T16:21:16.096 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.097 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.097 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.097 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.097 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.097 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.098 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.098 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Email address to send alerts to", 2023-09-12T16:21:16.098 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.098 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.098 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.099 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.099 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_from_name": { 2023-09-12T16:21:16.099 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_from_name", 2023-09-12T16:21:16.099 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.099 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.100 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.100 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "Ceph", 2023-09-12T16:21:16.100 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.100 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.100 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.100 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Email From: name", 2023-09-12T16:21:16.101 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.101 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.101 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.101 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.101 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_host": { 2023-09-12T16:21:16.102 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_host", 2023-09-12T16:21:16.102 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.102 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.102 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.102 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.103 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.103 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.103 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.103 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SMTP server", 2023-09-12T16:21:16.103 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.103 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.104 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.104 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.104 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_password": { 2023-09-12T16:21:16.104 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_password", 2023-09-12T16:21:16.104 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.105 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.105 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.105 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.105 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.105 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.105 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.106 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Password to authenticate with", 2023-09-12T16:21:16.106 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.106 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.106 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.106 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.107 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_port": { 2023-09-12T16:21:16.107 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_port", 2023-09-12T16:21:16.107 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.107 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.107 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.107 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "465", 2023-09-12T16:21:16.108 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.108 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.108 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.108 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SMTP port", 2023-09-12T16:21:16.108 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.109 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.109 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.109 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.109 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_sender": { 2023-09-12T16:21:16.109 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_sender", 2023-09-12T16:21:16.110 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.110 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.110 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.110 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.110 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.111 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.111 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.111 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SMTP envelope sender", 2023-09-12T16:21:16.111 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.111 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.111 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.112 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.112 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_ssl": { 2023-09-12T16:21:16.112 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_ssl", 2023-09-12T16:21:16.112 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.112 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.113 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.113 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.113 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.113 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.113 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.114 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-12T16:21:16.114 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.114 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.114 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.114 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.115 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_user": { 2023-09-12T16:21:16.115 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_user", 2023-09-12T16:21:16.115 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.115 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.115 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.115 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.116 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.116 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.116 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.116 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "User to authenticate as", 2023-09-12T16:21:16.116 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.117 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.117 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.117 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.117 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.117 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.118 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.118 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "balancer", 2023-09-12T16:21:16.118 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.118 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.118 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.118 INFO:teuthology.orchestra.run.smithi130.stdout: "active": { 2023-09-12T16:21:16.119 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "active", 2023-09-12T16:21:16.119 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.119 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.119 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.119 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.120 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.120 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.120 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.120 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "automatically balance PGs across cluster", 2023-09-12T16:21:16.120 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.120 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.121 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.121 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.121 INFO:teuthology.orchestra.run.smithi130.stdout: "begin_time": { 2023-09-12T16:21:16.121 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "begin_time", 2023-09-12T16:21:16.121 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.122 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.122 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.122 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0000", 2023-09-12T16:21:16.122 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.122 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.123 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.123 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "beginning time of day to automatically balance", 2023-09-12T16:21:16.123 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-12T16:21:16.123 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.123 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.123 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.124 INFO:teuthology.orchestra.run.smithi130.stdout: "begin_weekday": { 2023-09-12T16:21:16.124 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "begin_weekday", 2023-09-12T16:21:16.124 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:16.124 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.124 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.125 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:16.125 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "0", 2023-09-12T16:21:16.125 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "6", 2023-09-12T16:21:16.125 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.125 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-12T16:21:16.125 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-12T16:21:16.126 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.126 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.126 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.126 INFO:teuthology.orchestra.run.smithi130.stdout: "crush_compat_max_iterations": { 2023-09-12T16:21:16.126 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crush_compat_max_iterations", 2023-09-12T16:21:16.127 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:16.127 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.127 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.127 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "25", 2023-09-12T16:21:16.127 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:16.128 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "250", 2023-09-12T16:21:16.128 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.128 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-12T16:21:16.128 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.128 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.128 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.129 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.129 INFO:teuthology.orchestra.run.smithi130.stdout: "crush_compat_metrics": { 2023-09-12T16:21:16.129 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crush_compat_metrics", 2023-09-12T16:21:16.129 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.129 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.130 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.130 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "pgs,objects,bytes", 2023-09-12T16:21:16.130 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.130 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.130 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.131 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-12T16:21:16.131 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-12T16:21:16.131 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.131 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.131 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.131 INFO:teuthology.orchestra.run.smithi130.stdout: "crush_compat_step": { 2023-09-12T16:21:16.132 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crush_compat_step", 2023-09-12T16:21:16.132 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:16.132 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.132 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.132 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0.5", 2023-09-12T16:21:16.133 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "0.001", 2023-09-12T16:21:16.133 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "0.999", 2023-09-12T16:21:16.133 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.133 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "aggressiveness of optimization", 2023-09-12T16:21:16.133 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-12T16:21:16.133 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.134 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.134 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.134 INFO:teuthology.orchestra.run.smithi130.stdout: "end_time": { 2023-09-12T16:21:16.134 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "end_time", 2023-09-12T16:21:16.134 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.135 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.135 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.135 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "2359", 2023-09-12T16:21:16.135 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.135 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.135 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.136 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "ending time of day to automatically balance", 2023-09-12T16:21:16.136 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-12T16:21:16.136 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.136 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.136 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.137 INFO:teuthology.orchestra.run.smithi130.stdout: "end_weekday": { 2023-09-12T16:21:16.137 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "end_weekday", 2023-09-12T16:21:16.137 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:16.137 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.137 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.137 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:16.138 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "0", 2023-09-12T16:21:16.138 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "6", 2023-09-12T16:21:16.138 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.138 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-12T16:21:16.138 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-12T16:21:16.139 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.139 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.139 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.139 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.139 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.139 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.140 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.140 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.140 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.140 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.140 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.141 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.141 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.141 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.141 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.141 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.142 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.142 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.142 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.142 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.142 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.143 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.143 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.143 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.143 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.143 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.143 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.144 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.144 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.144 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.144 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.144 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.145 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.145 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.145 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.145 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.146 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.146 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.146 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.146 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.146 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.147 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.147 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.147 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.147 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.147 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.147 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.148 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.148 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.148 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.148 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.148 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.149 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.149 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.149 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.149 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.149 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.149 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.150 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.150 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.150 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.150 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.150 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.151 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.151 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.151 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.151 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.151 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.151 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.152 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.152 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.152 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.152 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.152 INFO:teuthology.orchestra.run.smithi130.stdout: "min_score": { 2023-09-12T16:21:16.153 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "min_score", 2023-09-12T16:21:16.153 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:16.153 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.153 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.153 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:16.153 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.154 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.154 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.154 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-12T16:21:16.154 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.154 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.155 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.155 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.155 INFO:teuthology.orchestra.run.smithi130.stdout: "mode": { 2023-09-12T16:21:16.155 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mode", 2023-09-12T16:21:16.155 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.155 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.156 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.156 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "upmap", 2023-09-12T16:21:16.156 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.156 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.156 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.157 INFO:teuthology.orchestra.run.smithi130.stdout: "crush-compat", 2023-09-12T16:21:16.157 INFO:teuthology.orchestra.run.smithi130.stdout: "none", 2023-09-12T16:21:16.157 INFO:teuthology.orchestra.run.smithi130.stdout: "upmap" 2023-09-12T16:21:16.157 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.157 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Balancer mode", 2023-09-12T16:21:16.157 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.158 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.158 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.158 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.158 INFO:teuthology.orchestra.run.smithi130.stdout: "pool_ids": { 2023-09-12T16:21:16.158 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pool_ids", 2023-09-12T16:21:16.159 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.159 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.159 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.159 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.159 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.159 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.160 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.160 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-12T16:21:16.160 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.160 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.160 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.161 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.161 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:16.161 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:16.161 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.161 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.161 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.162 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:16.162 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.162 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.162 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.162 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-12T16:21:16.163 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.163 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.163 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.163 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.163 INFO:teuthology.orchestra.run.smithi130.stdout: "upmap_max_deviation": { 2023-09-12T16:21:16.164 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "upmap_max_deviation", 2023-09-12T16:21:16.164 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.164 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.164 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.164 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:16.164 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:16.165 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.165 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.165 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-12T16:21:16.165 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-12T16:21:16.165 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.166 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.166 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.166 INFO:teuthology.orchestra.run.smithi130.stdout: "upmap_max_optimizations": { 2023-09-12T16:21:16.166 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "upmap_max_optimizations", 2023-09-12T16:21:16.166 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:16.167 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.167 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.167 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:16.167 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.167 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.167 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.168 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-12T16:21:16.168 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.168 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.168 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.168 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.169 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.169 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.169 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.169 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cephadm", 2023-09-12T16:21:16.169 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.170 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.170 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.170 INFO:teuthology.orchestra.run.smithi130.stdout: "agent_down_multiplier": { 2023-09-12T16:21:16.170 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "agent_down_multiplier", 2023-09-12T16:21:16.170 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:16.171 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.171 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.171 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "3.0", 2023-09-12T16:21:16.171 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.171 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.171 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.172 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-12T16:21:16.172 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.172 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.172 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.172 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.173 INFO:teuthology.orchestra.run.smithi130.stdout: "agent_refresh_rate": { 2023-09-12T16:21:16.173 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "agent_refresh_rate", 2023-09-12T16:21:16.173 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.173 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.173 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.173 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "20", 2023-09-12T16:21:16.174 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.174 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.174 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.174 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-12T16:21:16.174 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.175 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.175 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.175 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.175 INFO:teuthology.orchestra.run.smithi130.stdout: "agent_starting_port": { 2023-09-12T16:21:16.175 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "agent_starting_port", 2023-09-12T16:21:16.176 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.176 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.176 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.176 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "4721", 2023-09-12T16:21:16.176 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.176 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.177 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.177 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-12T16:21:16.177 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.177 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.178 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.178 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.178 INFO:teuthology.orchestra.run.smithi130.stdout: "allow_ptrace": { 2023-09-12T16:21:16.178 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "allow_ptrace", 2023-09-12T16:21:16.178 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.179 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.179 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.179 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.180 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.181 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.181 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.181 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-12T16:21:16.181 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-12T16:21:16.181 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.182 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.182 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.182 INFO:teuthology.orchestra.run.smithi130.stdout: "autotune_interval": { 2023-09-12T16:21:16.182 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "autotune_interval", 2023-09-12T16:21:16.182 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.182 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.183 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.183 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:16.183 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.183 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.183 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.184 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-12T16:21:16.184 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.184 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.184 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.184 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.184 INFO:teuthology.orchestra.run.smithi130.stdout: "autotune_memory_target_ratio": { 2023-09-12T16:21:16.185 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "autotune_memory_target_ratio", 2023-09-12T16:21:16.185 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:16.185 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.185 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.185 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0.7", 2023-09-12T16:21:16.186 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.186 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.186 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.186 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-12T16:21:16.186 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.187 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.187 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.187 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.187 INFO:teuthology.orchestra.run.smithi130.stdout: "cgroups_split": { 2023-09-12T16:21:16.187 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cgroups_split", 2023-09-12T16:21:16.187 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.188 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.188 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.188 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.188 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.188 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.189 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.189 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-12T16:21:16.189 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.189 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.189 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.190 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.190 INFO:teuthology.orchestra.run.smithi130.stdout: "config_checks_enabled": { 2023-09-12T16:21:16.190 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "config_checks_enabled", 2023-09-12T16:21:16.190 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.190 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.191 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.191 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.191 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.191 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.191 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.191 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-12T16:21:16.192 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.192 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.192 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.192 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.192 INFO:teuthology.orchestra.run.smithi130.stdout: "config_dashboard": { 2023-09-12T16:21:16.193 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "config_dashboard", 2023-09-12T16:21:16.193 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.193 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.193 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.193 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.193 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.194 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.194 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.194 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-12T16:21:16.194 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.194 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.195 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.195 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.195 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_alertmanager": { 2023-09-12T16:21:16.195 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_alertmanager", 2023-09-12T16:21:16.195 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.195 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.196 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.196 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-12T16:21:16.196 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.196 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.196 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.197 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:16.197 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.197 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.197 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.197 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.198 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_base": { 2023-09-12T16:21:16.198 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_base", 2023-09-12T16:21:16.198 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.198 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.198 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.199 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-12T16:21:16.199 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.199 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.199 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.199 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Container image name, without the tag", 2023-09-12T16:21:16.199 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.200 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.200 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.200 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.200 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_elasticsearch": { 2023-09-12T16:21:16.200 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_elasticsearch", 2023-09-12T16:21:16.201 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.201 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.201 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.201 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-12T16:21:16.201 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.201 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.202 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.202 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "elasticsearch container image", 2023-09-12T16:21:16.202 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.202 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.202 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.203 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.203 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_grafana": { 2023-09-12T16:21:16.203 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_grafana", 2023-09-12T16:21:16.203 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.203 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.204 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.204 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-12T16:21:16.204 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.204 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.204 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.204 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:16.205 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.205 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.205 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.205 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.205 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_haproxy": { 2023-09-12T16:21:16.206 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_haproxy", 2023-09-12T16:21:16.206 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.206 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.206 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.206 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-12T16:21:16.207 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.207 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.207 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.207 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "HAproxy container image", 2023-09-12T16:21:16.207 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.207 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.208 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.208 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.208 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_jaeger_agent": { 2023-09-12T16:21:16.208 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_jaeger_agent", 2023-09-12T16:21:16.208 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.209 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.209 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.209 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-12T16:21:16.209 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.209 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.209 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.210 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Jaeger agent container image", 2023-09-12T16:21:16.210 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.210 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.210 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.211 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.211 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_jaeger_collector": { 2023-09-12T16:21:16.211 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_jaeger_collector", 2023-09-12T16:21:16.211 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.211 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.212 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.212 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-12T16:21:16.212 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.212 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.212 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.212 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Jaeger collector container image", 2023-09-12T16:21:16.213 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.213 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.213 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.213 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.213 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_jaeger_query": { 2023-09-12T16:21:16.214 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_jaeger_query", 2023-09-12T16:21:16.214 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.214 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.214 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.214 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-12T16:21:16.215 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.215 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.215 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.215 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Jaeger query container image", 2023-09-12T16:21:16.215 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.215 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.216 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.216 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.216 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_keepalived": { 2023-09-12T16:21:16.216 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_keepalived", 2023-09-12T16:21:16.216 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.217 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.217 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.217 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-12T16:21:16.217 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.217 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.217 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.218 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Keepalived container image", 2023-09-12T16:21:16.218 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.218 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.219 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.219 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.219 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_loki": { 2023-09-12T16:21:16.219 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_loki", 2023-09-12T16:21:16.219 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.219 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.220 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.220 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-12T16:21:16.220 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.220 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.220 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.221 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Loki container image", 2023-09-12T16:21:16.221 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.221 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.221 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.221 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.221 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_node_exporter": { 2023-09-12T16:21:16.222 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_node_exporter", 2023-09-12T16:21:16.222 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.222 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.222 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.222 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-12T16:21:16.223 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.223 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.223 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.223 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:16.223 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.223 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.224 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.224 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.224 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_nvmeof": { 2023-09-12T16:21:16.224 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_nvmeof", 2023-09-12T16:21:16.224 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.224 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.225 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.225 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-12T16:21:16.225 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.225 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.225 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.226 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Nvme-of container image", 2023-09-12T16:21:16.226 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.226 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.226 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.226 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.226 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_prometheus": { 2023-09-12T16:21:16.227 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_prometheus", 2023-09-12T16:21:16.227 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.227 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.227 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.227 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-12T16:21:16.228 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.228 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.228 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.228 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:16.228 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.228 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.229 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.229 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.229 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_promtail": { 2023-09-12T16:21:16.229 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_promtail", 2023-09-12T16:21:16.229 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.229 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.230 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.230 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-12T16:21:16.230 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.230 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.230 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.231 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Promtail container image", 2023-09-12T16:21:16.231 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.231 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.231 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.231 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.232 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_snmp_gateway": { 2023-09-12T16:21:16.232 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_snmp_gateway", 2023-09-12T16:21:16.232 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.232 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.232 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.233 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-12T16:21:16.233 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.233 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.233 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.233 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SNMP Gateway container image", 2023-09-12T16:21:16.233 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.234 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.234 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.234 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.234 INFO:teuthology.orchestra.run.smithi130.stdout: "container_init": { 2023-09-12T16:21:16.234 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_init", 2023-09-12T16:21:16.235 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.235 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.235 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.235 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.235 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.236 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.236 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.236 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Run podman/docker with `--init`", 2023-09-12T16:21:16.236 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.236 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.236 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.237 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.237 INFO:teuthology.orchestra.run.smithi130.stdout: "daemon_cache_timeout": { 2023-09-12T16:21:16.237 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "daemon_cache_timeout", 2023-09-12T16:21:16.237 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.237 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.237 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.238 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:16.238 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.238 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.238 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.238 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-12T16:21:16.239 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.239 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.239 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.239 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.239 INFO:teuthology.orchestra.run.smithi130.stdout: "default_cephadm_command_timeout": { 2023-09-12T16:21:16.240 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "default_cephadm_command_timeout", 2023-09-12T16:21:16.240 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.240 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.240 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.240 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "900", 2023-09-12T16:21:16.241 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.241 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.241 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.241 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-12T16:21:16.241 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.242 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.242 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.242 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.242 INFO:teuthology.orchestra.run.smithi130.stdout: "default_registry": { 2023-09-12T16:21:16.243 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "default_registry", 2023-09-12T16:21:16.243 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.243 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.243 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.243 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io", 2023-09-12T16:21:16.243 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.244 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.244 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.244 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-12T16:21:16.244 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.244 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.245 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.245 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.245 INFO:teuthology.orchestra.run.smithi130.stdout: "device_cache_timeout": { 2023-09-12T16:21:16.245 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "device_cache_timeout", 2023-09-12T16:21:16.245 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.245 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.246 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.246 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "1800", 2023-09-12T16:21:16.246 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.246 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.246 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.246 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "seconds to cache device inventory", 2023-09-12T16:21:16.247 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.247 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.247 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.247 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.247 INFO:teuthology.orchestra.run.smithi130.stdout: "device_enhanced_scan": { 2023-09-12T16:21:16.247 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "device_enhanced_scan", 2023-09-12T16:21:16.248 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.248 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.248 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.248 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.248 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.248 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.249 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.249 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-12T16:21:16.249 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.249 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.249 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.250 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.250 INFO:teuthology.orchestra.run.smithi130.stdout: "facts_cache_timeout": { 2023-09-12T16:21:16.250 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "facts_cache_timeout", 2023-09-12T16:21:16.250 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.250 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.250 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.251 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:16.251 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.251 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.251 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.251 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "seconds to cache host facts data", 2023-09-12T16:21:16.252 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.252 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.252 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.252 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.252 INFO:teuthology.orchestra.run.smithi130.stdout: "host_check_interval": { 2023-09-12T16:21:16.252 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "host_check_interval", 2023-09-12T16:21:16.253 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.253 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.253 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.253 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:16.253 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.253 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.254 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.254 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to perform a host check", 2023-09-12T16:21:16.254 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.254 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.254 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.255 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.255 INFO:teuthology.orchestra.run.smithi130.stdout: "inventory_list_all": { 2023-09-12T16:21:16.255 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "inventory_list_all", 2023-09-12T16:21:16.255 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.255 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.255 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.256 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.256 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.256 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.256 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.256 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-12T16:21:16.256 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.257 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.257 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.257 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.257 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.257 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.258 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.258 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.258 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.258 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.258 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.258 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.259 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.259 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.259 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.259 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.259 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.259 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.260 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.260 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.260 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.260 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.260 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.261 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.261 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.261 INFO:teuthology.orchestra.run.smithi130.stdout: "log_refresh_metadata": { 2023-09-12T16:21:16.261 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_refresh_metadata", 2023-09-12T16:21:16.261 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.262 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.262 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.262 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.262 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.262 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.263 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.263 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-12T16:21:16.263 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.263 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.263 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.263 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.264 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.264 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.264 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.264 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.264 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.265 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.265 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.265 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.265 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.265 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-12T16:21:16.265 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.266 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.266 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.266 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.266 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.266 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.266 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.267 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.267 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.267 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.267 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.267 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.268 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.268 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.268 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.268 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.268 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.268 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.269 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.269 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.269 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.269 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.269 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.269 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.270 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.270 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.270 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.270 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.270 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.271 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.271 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.271 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.271 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.271 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.272 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.272 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.272 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.272 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.272 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.272 INFO:teuthology.orchestra.run.smithi130.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-12T16:21:16.273 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-12T16:21:16.273 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.273 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.273 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.273 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.274 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.274 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.274 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.274 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-12T16:21:16.274 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.275 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.275 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.275 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.275 INFO:teuthology.orchestra.run.smithi130.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-12T16:21:16.275 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-12T16:21:16.276 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.276 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.276 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.276 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "*", 2023-09-12T16:21:16.276 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.276 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.277 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.277 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-12T16:21:16.277 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.277 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.277 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.278 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.278 INFO:teuthology.orchestra.run.smithi130.stdout: "max_count_per_host": { 2023-09-12T16:21:16.278 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_count_per_host", 2023-09-12T16:21:16.278 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.278 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.279 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.279 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:16.279 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.279 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.279 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.279 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "max number of daemons per service per host", 2023-09-12T16:21:16.280 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.280 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.280 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.280 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.280 INFO:teuthology.orchestra.run.smithi130.stdout: "max_osd_draining_count": { 2023-09-12T16:21:16.280 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_osd_draining_count", 2023-09-12T16:21:16.281 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.281 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.281 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.281 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:16.281 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.282 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.282 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.282 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-12T16:21:16.282 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.282 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.283 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.283 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.283 INFO:teuthology.orchestra.run.smithi130.stdout: "migration_current": { 2023-09-12T16:21:16.283 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "migration_current", 2023-09-12T16:21:16.283 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.284 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.284 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.284 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.284 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.284 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.284 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.285 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "internal - do not modify", 2023-09-12T16:21:16.285 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.285 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.285 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.285 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.286 INFO:teuthology.orchestra.run.smithi130.stdout: "mode": { 2023-09-12T16:21:16.286 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mode", 2023-09-12T16:21:16.286 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.286 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.286 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.287 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "root", 2023-09-12T16:21:16.287 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.287 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.287 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.287 INFO:teuthology.orchestra.run.smithi130.stdout: "cephadm-package", 2023-09-12T16:21:16.287 INFO:teuthology.orchestra.run.smithi130.stdout: "root" 2023-09-12T16:21:16.288 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.288 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "mode for remote execution of cephadm", 2023-09-12T16:21:16.288 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.288 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.288 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.288 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.289 INFO:teuthology.orchestra.run.smithi130.stdout: "prometheus_alerts_path": { 2023-09-12T16:21:16.289 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "prometheus_alerts_path", 2023-09-12T16:21:16.289 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.289 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.289 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.290 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-12T16:21:16.290 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.290 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.290 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.290 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-12T16:21:16.290 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.291 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.291 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.291 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.291 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_insecure": { 2023-09-12T16:21:16.291 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_insecure", 2023-09-12T16:21:16.292 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.292 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.292 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.292 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.292 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.292 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.293 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.293 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-12T16:21:16.293 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.293 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.293 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.293 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.294 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_password": { 2023-09-12T16:21:16.294 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_password", 2023-09-12T16:21:16.294 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.294 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.294 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.295 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.295 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.295 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.295 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.295 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-12T16:21:16.295 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.296 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.296 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.296 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.296 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_url": { 2023-09-12T16:21:16.296 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_url", 2023-09-12T16:21:16.296 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.297 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.297 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.297 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.297 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.297 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.297 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.298 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-12T16:21:16.298 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.298 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.298 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.298 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.299 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_username": { 2023-09-12T16:21:16.299 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_username", 2023-09-12T16:21:16.299 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.299 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.299 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.299 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.300 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.300 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.300 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.300 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-12T16:21:16.300 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.301 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.301 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.301 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.301 INFO:teuthology.orchestra.run.smithi130.stdout: "secure_monitoring_stack": { 2023-09-12T16:21:16.301 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "secure_monitoring_stack", 2023-09-12T16:21:16.302 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.302 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.302 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.302 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.302 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.302 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.303 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.303 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-12T16:21:16.303 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.303 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.303 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.303 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.304 INFO:teuthology.orchestra.run.smithi130.stdout: "service_discovery_port": { 2023-09-12T16:21:16.304 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "service_discovery_port", 2023-09-12T16:21:16.304 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.304 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.304 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.305 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8765", 2023-09-12T16:21:16.305 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.305 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.305 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.305 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "cephadm service discovery port", 2023-09-12T16:21:16.306 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.306 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.306 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.306 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.306 INFO:teuthology.orchestra.run.smithi130.stdout: "ssh_config_file": { 2023-09-12T16:21:16.307 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssh_config_file", 2023-09-12T16:21:16.307 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.307 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.307 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.307 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.307 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.308 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.308 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.308 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-12T16:21:16.308 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.308 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.309 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.309 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.309 INFO:teuthology.orchestra.run.smithi130.stdout: "use_agent": { 2023-09-12T16:21:16.309 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "use_agent", 2023-09-12T16:21:16.309 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.309 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.310 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.310 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.310 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.310 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.310 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.311 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-12T16:21:16.311 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.311 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.311 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.311 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.312 INFO:teuthology.orchestra.run.smithi130.stdout: "use_repo_digest": { 2023-09-12T16:21:16.312 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "use_repo_digest", 2023-09-12T16:21:16.312 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.312 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.312 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.312 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.313 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.313 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.313 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.313 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-12T16:21:16.313 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.314 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.314 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.314 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.314 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_on_failed_host_check": { 2023-09-12T16:21:16.314 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_on_failed_host_check", 2023-09-12T16:21:16.315 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.315 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.315 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.315 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.315 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.315 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.316 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.316 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise a health warning if the host check fails", 2023-09-12T16:21:16.316 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.316 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.316 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.317 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.317 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_on_stray_daemons": { 2023-09-12T16:21:16.317 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_on_stray_daemons", 2023-09-12T16:21:16.317 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.317 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.318 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.318 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.318 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.318 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.318 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.319 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-12T16:21:16.319 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.319 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.319 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.319 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.320 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_on_stray_hosts": { 2023-09-12T16:21:16.320 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_on_stray_hosts", 2023-09-12T16:21:16.320 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.320 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.320 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.320 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.321 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.321 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.321 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.321 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-12T16:21:16.321 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.322 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.322 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.322 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.322 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.322 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.323 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.323 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crash", 2023-09-12T16:21:16.323 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.323 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.323 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.323 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.324 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.324 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.324 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.324 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.324 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.325 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.325 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.325 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.325 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.325 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.325 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.326 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.326 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.326 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.326 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.326 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.327 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.327 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.327 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.327 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.327 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.328 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.328 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.328 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.328 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.328 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.328 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.329 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.329 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.329 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.329 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.329 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.329 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.330 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.330 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.330 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.330 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.331 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.331 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.331 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.331 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.331 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.332 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.332 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.332 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.332 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.332 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.333 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.333 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.333 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.333 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.333 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.333 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.334 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.334 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.334 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.334 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.334 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.335 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.335 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.335 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.335 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.335 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.336 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.336 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.336 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.336 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.336 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.336 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.337 INFO:teuthology.orchestra.run.smithi130.stdout: "retain_interval": { 2023-09-12T16:21:16.337 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "retain_interval", 2023-09-12T16:21:16.337 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.337 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.337 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.337 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "31536000", 2023-09-12T16:21:16.338 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.338 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.338 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.338 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-12T16:21:16.338 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.339 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.339 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.339 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.339 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_recent_interval": { 2023-09-12T16:21:16.340 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_recent_interval", 2023-09-12T16:21:16.340 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.340 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.340 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.340 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "1209600", 2023-09-12T16:21:16.340 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.341 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.341 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.341 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-12T16:21:16.341 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.341 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.342 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.342 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.342 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.342 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.342 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.343 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "dashboard", 2023-09-12T16:21:16.343 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.343 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.343 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.343 INFO:teuthology.orchestra.run.smithi130.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-12T16:21:16.343 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-12T16:21:16.344 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.344 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.344 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.344 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:16.344 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.344 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.345 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.345 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.345 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.345 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.345 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.345 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.346 INFO:teuthology.orchestra.run.smithi130.stdout: "ALERTMANAGER_API_HOST": { 2023-09-12T16:21:16.346 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-12T16:21:16.346 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.346 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.346 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.347 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.347 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.347 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.347 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.347 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.347 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.348 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.348 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.348 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.348 INFO:teuthology.orchestra.run.smithi130.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-12T16:21:16.348 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-12T16:21:16.348 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.349 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.349 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.349 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.349 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.349 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.350 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.350 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.350 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.350 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.350 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.350 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.351 INFO:teuthology.orchestra.run.smithi130.stdout: "AUDIT_API_ENABLED": { 2023-09-12T16:21:16.351 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "AUDIT_API_ENABLED", 2023-09-12T16:21:16.351 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.351 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.351 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.352 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.352 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.352 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.352 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.352 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.352 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.353 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.353 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.353 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.353 INFO:teuthology.orchestra.run.smithi130.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-12T16:21:16.353 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-12T16:21:16.353 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.353 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.354 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.354 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.354 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.354 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.354 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.355 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.355 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.355 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.355 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.355 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.356 INFO:teuthology.orchestra.run.smithi130.stdout: "ENABLE_BROWSABLE_API": { 2023-09-12T16:21:16.356 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-12T16:21:16.356 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.356 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.356 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.356 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.357 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.357 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.357 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.357 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.357 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.358 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.358 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.358 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.358 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-12T16:21:16.358 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-12T16:21:16.358 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.359 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.359 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.359 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.359 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.359 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.359 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.360 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.360 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.360 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.360 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.360 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.360 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-12T16:21:16.361 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-12T16:21:16.361 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.361 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.361 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.361 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.362 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.362 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.362 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.362 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.362 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.363 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.363 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.363 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.363 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-12T16:21:16.363 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-12T16:21:16.364 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.364 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.364 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.364 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.364 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.364 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.365 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.365 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.365 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.365 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.365 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.365 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.366 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-12T16:21:16.366 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-12T16:21:16.366 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.366 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.366 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.366 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.367 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.367 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.367 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.367 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.367 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.368 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.368 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.368 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.368 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-12T16:21:16.368 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-12T16:21:16.368 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.369 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.369 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.369 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.369 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.369 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.369 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.370 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.370 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.370 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.370 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.371 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.371 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-12T16:21:16.371 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-12T16:21:16.371 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.371 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.371 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.372 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.372 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.372 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.372 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.372 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.373 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.373 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.373 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.373 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.373 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-12T16:21:16.374 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-12T16:21:16.374 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.374 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.374 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.374 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.375 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.375 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.375 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.375 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.375 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.376 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.376 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.376 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.376 INFO:teuthology.orchestra.run.smithi130.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-12T16:21:16.376 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-12T16:21:16.376 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.377 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.377 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.377 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.377 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.377 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.378 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.378 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.378 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.378 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.378 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.379 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.379 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_PASSWORD": { 2023-09-12T16:21:16.379 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-12T16:21:16.379 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.379 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.380 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.380 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "admin", 2023-09-12T16:21:16.380 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.380 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.380 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.380 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.381 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.381 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.381 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.381 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.381 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-12T16:21:16.382 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-12T16:21:16.382 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.382 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.382 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.382 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.383 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.383 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.383 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.383 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.383 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.383 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.384 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.384 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.384 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_URL": { 2023-09-12T16:21:16.384 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_URL", 2023-09-12T16:21:16.384 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.385 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.385 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.385 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.385 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.385 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.386 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.386 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.386 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.386 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.386 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.386 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.387 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_USERNAME": { 2023-09-12T16:21:16.387 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-12T16:21:16.387 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.387 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.387 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.388 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "admin", 2023-09-12T16:21:16.388 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.388 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.388 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.388 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.389 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.389 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.389 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.389 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.389 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-12T16:21:16.390 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-12T16:21:16.390 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.390 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.390 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.390 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.391 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.391 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.391 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.391 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.391 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.391 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.392 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.392 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.392 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-12T16:21:16.392 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-12T16:21:16.392 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.393 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.393 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.393 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.393 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.393 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.393 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.394 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.394 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.394 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.394 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.394 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.395 INFO:teuthology.orchestra.run.smithi130.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-12T16:21:16.395 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-12T16:21:16.395 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.395 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.395 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.396 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.396 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.396 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.396 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.396 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.397 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.397 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.397 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.397 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.397 INFO:teuthology.orchestra.run.smithi130.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-12T16:21:16.397 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-12T16:21:16.398 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.398 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.398 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.398 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.398 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.399 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.399 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.399 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.399 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.399 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.400 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.400 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.400 INFO:teuthology.orchestra.run.smithi130.stdout: "PROMETHEUS_API_HOST": { 2023-09-12T16:21:16.400 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-12T16:21:16.400 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.400 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.401 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.401 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.401 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.401 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.401 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.401 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.402 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.402 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.402 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.402 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.402 INFO:teuthology.orchestra.run.smithi130.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-12T16:21:16.403 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-12T16:21:16.403 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.403 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.403 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.403 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.404 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.404 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.404 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.404 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.404 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.404 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.405 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.405 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.405 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-12T16:21:16.405 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-12T16:21:16.405 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.406 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.406 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.406 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.406 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.406 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.407 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.407 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.407 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.407 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.407 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.408 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.408 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-12T16:21:16.408 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-12T16:21:16.408 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.408 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.409 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.409 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.409 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.409 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.409 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.409 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.410 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.410 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.410 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.410 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.411 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-12T16:21:16.411 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-12T16:21:16.411 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.411 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.411 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.411 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.412 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.412 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.412 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.412 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.412 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.413 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.413 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.413 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.413 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-12T16:21:16.413 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-12T16:21:16.413 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.414 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.414 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.414 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.414 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.414 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.415 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.415 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.415 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.415 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.415 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.416 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.416 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-12T16:21:16.416 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-12T16:21:16.416 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.416 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.417 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.417 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.417 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.417 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.417 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.417 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.418 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.418 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.418 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.418 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.418 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-12T16:21:16.419 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-12T16:21:16.419 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.419 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.419 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.419 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.419 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.420 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.420 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.420 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.420 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.420 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.420 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.421 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.421 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-12T16:21:16.421 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-12T16:21:16.421 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.421 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.422 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.422 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.422 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.422 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.422 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.423 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.423 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.423 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.423 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.423 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.423 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_ENABLED": { 2023-09-12T16:21:16.424 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-12T16:21:16.424 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.424 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.424 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.424 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.424 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.425 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.425 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.425 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.425 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.425 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.426 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.426 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.426 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-12T16:21:16.426 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-12T16:21:16.426 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.427 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.427 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.427 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-12T16:21:16.427 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.427 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.428 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.428 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.428 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.428 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.428 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.428 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.429 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-12T16:21:16.429 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-12T16:21:16.429 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.429 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.429 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.430 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:16.430 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.430 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.430 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.430 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.431 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.431 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.431 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.431 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.431 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-12T16:21:16.431 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-12T16:21:16.432 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.432 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.432 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.432 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8", 2023-09-12T16:21:16.432 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.433 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.433 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.433 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.433 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.433 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.433 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.434 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.434 INFO:teuthology.orchestra.run.smithi130.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-12T16:21:16.434 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-12T16:21:16.434 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.434 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.435 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.435 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "45", 2023-09-12T16:21:16.435 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.435 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.435 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.435 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.436 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.436 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.436 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.436 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.436 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_ACCESS_KEY": { 2023-09-12T16:21:16.437 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-12T16:21:16.437 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.437 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.437 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.437 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.437 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.438 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.438 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.438 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.438 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.438 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.439 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.439 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.439 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-12T16:21:16.439 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-12T16:21:16.439 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.439 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.440 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.440 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "admin", 2023-09-12T16:21:16.440 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.440 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.440 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.441 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.441 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.441 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.441 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.441 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.441 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_SECRET_KEY": { 2023-09-12T16:21:16.442 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-12T16:21:16.442 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.442 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.442 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.442 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.443 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.443 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.443 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.443 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.443 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.444 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.444 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.444 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.444 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_SSL_VERIFY": { 2023-09-12T16:21:16.444 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-12T16:21:16.444 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.445 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.445 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.445 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.445 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.445 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.446 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.446 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.446 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.446 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.446 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.447 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.447 INFO:teuthology.orchestra.run.smithi130.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-12T16:21:16.447 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-12T16:21:16.447 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.447 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.448 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.448 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:16.448 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.448 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.448 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.448 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.449 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.449 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.449 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.449 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.449 INFO:teuthology.orchestra.run.smithi130.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-12T16:21:16.450 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-12T16:21:16.450 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.450 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.450 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.450 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:16.451 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.451 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.451 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.451 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.451 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.452 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.452 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.452 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.452 INFO:teuthology.orchestra.run.smithi130.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-12T16:21:16.452 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-12T16:21:16.452 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.453 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.453 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.453 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:16.453 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.453 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.453 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.454 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.454 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.454 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.454 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.454 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.455 INFO:teuthology.orchestra.run.smithi130.stdout: "cross_origin_url": { 2023-09-12T16:21:16.455 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cross_origin_url", 2023-09-12T16:21:16.455 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.455 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.455 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.456 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.456 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.456 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.456 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.456 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.456 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.457 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.457 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.457 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.457 INFO:teuthology.orchestra.run.smithi130.stdout: "crt_file": { 2023-09-12T16:21:16.457 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crt_file", 2023-09-12T16:21:16.458 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.458 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.458 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.458 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.458 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.459 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.459 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.459 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.459 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.459 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.460 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.460 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.460 INFO:teuthology.orchestra.run.smithi130.stdout: "debug": { 2023-09-12T16:21:16.460 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "debug", 2023-09-12T16:21:16.460 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.460 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.461 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.461 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.461 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.461 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.461 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.462 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Enable/disable debug options", 2023-09-12T16:21:16.462 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.462 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.462 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.462 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.463 INFO:teuthology.orchestra.run.smithi130.stdout: "jwt_token_ttl": { 2023-09-12T16:21:16.463 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "jwt_token_ttl", 2023-09-12T16:21:16.463 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.463 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.463 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.463 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "28800", 2023-09-12T16:21:16.464 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.464 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.464 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.464 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.464 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.465 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.465 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.465 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.465 INFO:teuthology.orchestra.run.smithi130.stdout: "key_file": { 2023-09-12T16:21:16.465 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "key_file", 2023-09-12T16:21:16.465 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.466 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.466 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.466 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.466 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.466 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.467 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.467 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.467 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.467 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.467 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.468 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.468 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.468 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.468 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.468 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.469 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.469 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.469 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.469 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.469 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.469 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.470 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.470 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.470 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.470 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.470 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.471 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.471 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.471 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.471 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.471 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.472 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.472 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.472 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.472 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.472 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.472 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.473 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.473 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.473 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.473 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.473 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.474 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.474 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.474 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.474 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.474 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.475 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.475 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.475 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.475 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.475 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.475 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.476 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.476 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.476 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.476 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.476 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.477 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.477 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.477 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.477 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.477 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.478 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.478 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.478 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.478 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.478 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.479 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.479 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.479 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.479 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.479 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.479 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.480 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.480 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.480 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.480 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.480 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.481 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.481 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.481 INFO:teuthology.orchestra.run.smithi130.stdout: "motd": { 2023-09-12T16:21:16.481 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "motd", 2023-09-12T16:21:16.481 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.481 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.482 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.482 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.482 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.482 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.482 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.483 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "The message of the day", 2023-09-12T16:21:16.483 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.483 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.483 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.483 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.484 INFO:teuthology.orchestra.run.smithi130.stdout: "redirect_resolve_ip_addr": { 2023-09-12T16:21:16.484 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "redirect_resolve_ip_addr", 2023-09-12T16:21:16.484 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.484 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.484 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.485 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.485 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.485 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.485 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.485 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.486 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.486 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.486 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.486 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.486 INFO:teuthology.orchestra.run.smithi130.stdout: "server_addr": { 2023-09-12T16:21:16.486 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_addr", 2023-09-12T16:21:16.487 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.487 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.487 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.487 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "::", 2023-09-12T16:21:16.487 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.488 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.488 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.488 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.488 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.488 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.489 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.489 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.489 INFO:teuthology.orchestra.run.smithi130.stdout: "server_port": { 2023-09-12T16:21:16.489 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_port", 2023-09-12T16:21:16.489 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.489 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.490 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.490 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8080", 2023-09-12T16:21:16.490 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.490 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.490 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.491 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.491 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.491 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.491 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.491 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.492 INFO:teuthology.orchestra.run.smithi130.stdout: "ssl": { 2023-09-12T16:21:16.492 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssl", 2023-09-12T16:21:16.492 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.492 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.492 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.492 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.493 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.493 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.493 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.493 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.493 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.494 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.494 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.494 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.494 INFO:teuthology.orchestra.run.smithi130.stdout: "ssl_server_port": { 2023-09-12T16:21:16.494 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssl_server_port", 2023-09-12T16:21:16.495 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.495 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.495 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.495 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8443", 2023-09-12T16:21:16.495 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.496 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.496 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.496 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.496 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.496 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.496 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.497 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.497 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_behaviour": { 2023-09-12T16:21:16.497 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_behaviour", 2023-09-12T16:21:16.497 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.497 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.498 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.498 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "redirect", 2023-09-12T16:21:16.498 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.498 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.498 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.499 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.499 INFO:teuthology.orchestra.run.smithi130.stdout: "redirect" 2023-09-12T16:21:16.499 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.499 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.499 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.499 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.500 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.500 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.500 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_error_status_code": { 2023-09-12T16:21:16.500 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_error_status_code", 2023-09-12T16:21:16.500 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.501 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.501 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.501 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "500", 2023-09-12T16:21:16.501 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "400", 2023-09-12T16:21:16.501 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "599", 2023-09-12T16:21:16.502 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.502 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.502 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.502 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.502 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.503 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.503 INFO:teuthology.orchestra.run.smithi130.stdout: "url_prefix": { 2023-09-12T16:21:16.503 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "url_prefix", 2023-09-12T16:21:16.503 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.503 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.504 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.504 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.504 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.504 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.504 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.504 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.505 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.505 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.505 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.505 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.505 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.506 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.506 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.506 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "devicehealth", 2023-09-12T16:21:16.506 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.506 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.507 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.507 INFO:teuthology.orchestra.run.smithi130.stdout: "enable_monitoring": { 2023-09-12T16:21:16.507 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enable_monitoring", 2023-09-12T16:21:16.507 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.507 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.507 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.508 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.508 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.508 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.508 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.508 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "monitor device health metrics", 2023-09-12T16:21:16.509 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.509 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.509 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.509 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.509 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.510 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.510 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.510 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.510 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.510 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.510 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.511 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.511 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.511 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.511 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.511 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.512 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.512 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.512 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.512 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.512 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.513 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.513 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.513 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.513 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.513 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.514 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.514 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.514 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.514 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.514 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.515 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.515 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.515 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.515 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.515 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.516 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.516 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.516 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.516 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.516 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.517 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.517 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.517 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.517 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.517 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.517 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.518 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.518 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.518 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.518 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.518 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.519 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.519 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.519 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.519 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.519 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.520 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.520 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.520 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.520 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.520 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.520 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.521 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.521 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.521 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.521 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.521 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.522 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.522 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.522 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.522 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.522 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.523 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.523 INFO:teuthology.orchestra.run.smithi130.stdout: "mark_out_threshold": { 2023-09-12T16:21:16.523 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mark_out_threshold", 2023-09-12T16:21:16.523 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.523 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.523 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "2419200", 2023-09-12T16:21:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.524 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-12T16:21:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.525 INFO:teuthology.orchestra.run.smithi130.stdout: "pool_name": { 2023-09-12T16:21:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pool_name", 2023-09-12T16:21:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "device_health_metrics", 2023-09-12T16:21:16.526 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-12T16:21:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.527 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "retention_period": { 2023-09-12T16:21:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "retention_period", 2023-09-12T16:21:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.528 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "15552000", 2023-09-12T16:21:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.529 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.530 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how long to retain device health metrics", 2023-09-12T16:21:16.530 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.530 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.530 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.530 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.531 INFO:teuthology.orchestra.run.smithi130.stdout: "scrape_frequency": { 2023-09-12T16:21:16.531 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "scrape_frequency", 2023-09-12T16:21:16.531 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.531 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.531 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.531 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "86400", 2023-09-12T16:21:16.532 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.532 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.532 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.532 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-12T16:21:16.532 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.533 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.533 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.533 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.533 INFO:teuthology.orchestra.run.smithi130.stdout: "self_heal": { 2023-09-12T16:21:16.533 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "self_heal", 2023-09-12T16:21:16.534 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.534 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.534 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.534 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.534 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.534 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.535 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.535 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-12T16:21:16.535 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.535 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.535 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.536 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.536 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:16.536 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:16.536 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.536 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.537 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.537 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:16.537 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.537 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.537 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.537 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to wake up and check device health", 2023-09-12T16:21:16.538 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.538 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.538 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.538 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.538 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_threshold": { 2023-09-12T16:21:16.539 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_threshold", 2023-09-12T16:21:16.539 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.539 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.539 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.539 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "7257600", 2023-09-12T16:21:16.540 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.540 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.540 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.540 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-12T16:21:16.540 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.540 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.541 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.541 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.541 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.541 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.541 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.542 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "diskprediction_local", 2023-09-12T16:21:16.542 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.542 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.542 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.542 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.543 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.543 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.543 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.543 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.543 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.543 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.544 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.544 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.544 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.544 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.544 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.545 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.545 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.545 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.545 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.545 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.546 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.546 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.546 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.546 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.546 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.547 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.547 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.547 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.547 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.547 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.548 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.548 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.548 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.548 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.548 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.549 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.549 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.549 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.549 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.549 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.549 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.550 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.550 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.550 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.550 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.550 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.551 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.551 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.551 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.551 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.551 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.552 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.552 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.552 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.552 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.552 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.553 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.553 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.553 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.553 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.553 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.553 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.554 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.554 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.554 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.554 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.554 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.555 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.555 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.555 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.555 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.555 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.555 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.556 INFO:teuthology.orchestra.run.smithi130.stdout: "predict_interval": { 2023-09-12T16:21:16.556 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "predict_interval", 2023-09-12T16:21:16.556 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.556 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.556 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.557 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "86400", 2023-09-12T16:21:16.557 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.557 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.557 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.557 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.558 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.558 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.558 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.558 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.558 INFO:teuthology.orchestra.run.smithi130.stdout: "predictor_model": { 2023-09-12T16:21:16.559 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "predictor_model", 2023-09-12T16:21:16.559 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.559 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.559 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.559 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "prophetstor", 2023-09-12T16:21:16.559 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.560 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.560 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.560 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.560 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.560 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.561 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.561 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.561 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:16.561 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:16.561 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.562 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.562 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.562 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:16.562 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.562 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.563 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.563 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.563 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.563 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.563 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.563 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.564 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.564 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.564 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.564 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "influx", 2023-09-12T16:21:16.564 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": false, 2023-09-12T16:21:16.565 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "influxdb python module not found", 2023-09-12T16:21:16.565 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.565 INFO:teuthology.orchestra.run.smithi130.stdout: "batch_size": { 2023-09-12T16:21:16.565 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "batch_size", 2023-09-12T16:21:16.565 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.565 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.566 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.566 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5000", 2023-09-12T16:21:16.566 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.566 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.566 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.567 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-12T16:21:16.567 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.567 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.567 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.567 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.567 INFO:teuthology.orchestra.run.smithi130.stdout: "database": { 2023-09-12T16:21:16.568 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "database", 2023-09-12T16:21:16.568 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.568 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.568 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.568 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "ceph", 2023-09-12T16:21:16.568 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.569 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.569 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.569 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-12T16:21:16.569 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.569 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.570 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.570 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.570 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": { 2023-09-12T16:21:16.570 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "hostname", 2023-09-12T16:21:16.570 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.571 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.571 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.571 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.571 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.571 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.572 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.572 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "InfluxDB server hostname", 2023-09-12T16:21:16.572 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.572 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.572 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.572 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.573 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:16.573 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:16.573 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.573 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.573 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.574 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "30", 2023-09-12T16:21:16.574 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "5", 2023-09-12T16:21:16.574 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.574 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.574 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-12T16:21:16.575 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.575 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.575 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.575 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.575 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.575 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.576 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.576 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.576 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.576 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.576 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.577 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.577 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.577 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.577 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.577 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.578 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.578 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.578 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.578 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.578 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.579 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.579 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.579 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.579 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.579 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.579 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.580 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.580 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.580 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.580 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.580 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.581 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.581 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.581 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.581 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.581 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.582 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.582 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.582 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.582 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.582 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.583 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.583 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.583 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.583 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.583 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.584 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.584 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.584 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.584 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.584 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.585 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.585 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.585 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.585 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.585 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.585 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.586 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.586 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.586 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.586 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.586 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.587 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.587 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.587 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.587 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.587 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.588 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.588 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.588 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.588 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.588 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.589 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.589 INFO:teuthology.orchestra.run.smithi130.stdout: "password": { 2023-09-12T16:21:16.589 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "password", 2023-09-12T16:21:16.589 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.589 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.590 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.590 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.590 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.590 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.590 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.591 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "password of InfluxDB server user", 2023-09-12T16:21:16.591 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.591 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.591 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.591 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.591 INFO:teuthology.orchestra.run.smithi130.stdout: "port": { 2023-09-12T16:21:16.592 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "port", 2023-09-12T16:21:16.592 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.592 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.592 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.592 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8086", 2023-09-12T16:21:16.593 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.593 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.593 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.593 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "InfluxDB server port", 2023-09-12T16:21:16.593 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.593 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.594 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.594 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.594 INFO:teuthology.orchestra.run.smithi130.stdout: "ssl": { 2023-09-12T16:21:16.594 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssl", 2023-09-12T16:21:16.594 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.595 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.595 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.595 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "false", 2023-09-12T16:21:16.597 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.597 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.597 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.598 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-12T16:21:16.598 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.598 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.598 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.598 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.599 INFO:teuthology.orchestra.run.smithi130.stdout: "threads": { 2023-09-12T16:21:16.599 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "threads", 2023-09-12T16:21:16.599 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.599 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.599 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.599 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:16.600 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:16.600 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "32", 2023-09-12T16:21:16.600 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.600 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-12T16:21:16.600 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.601 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.601 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.601 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.601 INFO:teuthology.orchestra.run.smithi130.stdout: "username": { 2023-09-12T16:21:16.601 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "username", 2023-09-12T16:21:16.601 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.602 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.602 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.602 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.602 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.602 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.603 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.603 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "username of InfluxDB server user", 2023-09-12T16:21:16.603 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.603 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.603 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.604 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.604 INFO:teuthology.orchestra.run.smithi130.stdout: "verify_ssl": { 2023-09-12T16:21:16.604 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "verify_ssl", 2023-09-12T16:21:16.604 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.604 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.605 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.605 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "true", 2023-09-12T16:21:16.605 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.605 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.605 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.605 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-12T16:21:16.606 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.606 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.606 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.606 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.606 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.607 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.607 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.607 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "insights", 2023-09-12T16:21:16.607 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.607 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.608 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.608 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.608 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.608 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.608 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.609 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.609 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.609 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.609 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.609 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.609 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.610 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.610 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.610 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.610 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.610 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.611 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.611 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.611 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.611 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.611 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.612 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.612 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.612 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.612 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.612 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.612 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.613 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.613 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.613 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.613 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.613 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.614 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.614 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.614 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.614 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.614 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.615 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.615 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.615 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.615 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.615 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.615 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.616 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.616 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.616 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.616 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.616 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.617 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.617 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.617 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.617 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.617 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.618 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.618 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.618 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.618 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.618 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.618 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.619 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.619 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.619 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.619 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.619 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.620 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.620 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.620 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.620 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.621 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.621 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.621 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.621 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.621 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.621 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.622 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "iostat", 2023-09-12T16:21:16.622 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.622 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.622 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.622 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.623 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.623 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.623 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.623 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.623 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.624 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.624 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.624 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.624 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.624 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.625 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.625 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.625 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.625 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.625 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.625 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.626 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.626 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.626 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.626 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.626 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.627 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.627 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.627 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.627 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.627 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.628 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.628 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.628 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.628 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.628 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.628 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.629 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.629 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.629 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.629 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.629 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.630 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.630 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.630 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.630 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.630 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.630 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.631 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.631 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.631 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.631 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.631 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.632 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.632 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.632 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.632 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.632 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.633 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.633 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.633 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.633 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.633 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.633 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.634 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.634 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.634 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.634 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.634 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.635 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.635 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.635 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.635 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.635 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.635 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.636 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.636 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.636 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "k8sevents", 2023-09-12T16:21:16.636 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.636 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.637 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.637 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_event_retention_days": { 2023-09-12T16:21:16.637 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ceph_event_retention_days", 2023-09-12T16:21:16.637 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.637 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.638 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.638 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "7", 2023-09-12T16:21:16.638 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.638 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.638 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.638 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-12T16:21:16.639 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.639 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.639 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.639 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.639 INFO:teuthology.orchestra.run.smithi130.stdout: "config_check_secs": { 2023-09-12T16:21:16.640 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "config_check_secs", 2023-09-12T16:21:16.640 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.640 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.640 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.640 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:16.641 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "10", 2023-09-12T16:21:16.641 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.641 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.641 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-12T16:21:16.641 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.642 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.642 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.642 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.642 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.642 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.643 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.643 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.643 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.643 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.643 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.644 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.644 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.644 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.644 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.644 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.644 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.645 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.645 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.645 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.645 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.645 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.646 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.646 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.646 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.646 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.646 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.647 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.647 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.647 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.647 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.647 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.648 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.648 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.648 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.648 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.648 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.649 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.649 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.649 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.649 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.649 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.649 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.650 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.650 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.650 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.650 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.650 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.651 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.651 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.651 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.651 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.651 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.652 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.652 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.652 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.652 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.652 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.652 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.653 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.653 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.653 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.653 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.653 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.654 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.654 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.654 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.654 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.654 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.654 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.655 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.655 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.655 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.655 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.655 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.656 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.656 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.656 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "localpool", 2023-09-12T16:21:16.656 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.656 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.657 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.657 INFO:teuthology.orchestra.run.smithi130.stdout: "failure_domain": { 2023-09-12T16:21:16.657 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "failure_domain", 2023-09-12T16:21:16.657 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.657 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.658 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.658 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "host", 2023-09-12T16:21:16.658 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.658 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.658 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.659 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "failure domain for any created local pool", 2023-09-12T16:21:16.659 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-12T16:21:16.659 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.659 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.659 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.660 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.660 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.660 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.660 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.660 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.661 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.661 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.661 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.661 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.661 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.662 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.662 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.662 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.662 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.662 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.663 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.663 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.663 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.663 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.663 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.663 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.664 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.664 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.664 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.664 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.664 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.665 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.665 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.665 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.665 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.665 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.665 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.666 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.666 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.666 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.666 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.666 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.667 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.667 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.667 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.667 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.667 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.668 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.668 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.668 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.668 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.668 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.668 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.669 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.669 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.669 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.669 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.669 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.670 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.670 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.670 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.670 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.670 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.671 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.671 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.671 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.671 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.671 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.671 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.672 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.672 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.672 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.672 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.672 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.673 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.673 INFO:teuthology.orchestra.run.smithi130.stdout: "min_size": { 2023-09-12T16:21:16.673 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "min_size", 2023-09-12T16:21:16.673 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.673 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.674 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.674 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.674 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.674 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.674 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.675 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "default min_size for any created local pool", 2023-09-12T16:21:16.675 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-12T16:21:16.675 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.675 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.675 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.676 INFO:teuthology.orchestra.run.smithi130.stdout: "num_rep": { 2023-09-12T16:21:16.676 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "num_rep", 2023-09-12T16:21:16.676 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.676 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.676 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.676 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "3", 2023-09-12T16:21:16.677 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.677 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.677 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.677 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "default replica count for any created local pool", 2023-09-12T16:21:16.677 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.678 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.678 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.678 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.678 INFO:teuthology.orchestra.run.smithi130.stdout: "pg_num": { 2023-09-12T16:21:16.678 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pg_num", 2023-09-12T16:21:16.678 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.679 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.679 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.679 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "128", 2023-09-12T16:21:16.679 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.679 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.680 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.680 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "default pg_num for any created local pool", 2023-09-12T16:21:16.680 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.680 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.680 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.681 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.681 INFO:teuthology.orchestra.run.smithi130.stdout: "prefix": { 2023-09-12T16:21:16.681 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "prefix", 2023-09-12T16:21:16.681 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.681 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.681 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.682 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.682 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.682 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.682 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.682 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "name prefix for any created local pool", 2023-09-12T16:21:16.683 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.683 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.683 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.683 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.683 INFO:teuthology.orchestra.run.smithi130.stdout: "subtree": { 2023-09-12T16:21:16.684 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "subtree", 2023-09-12T16:21:16.684 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.684 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.684 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.684 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "rack", 2023-09-12T16:21:16.684 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.685 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.685 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.685 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-12T16:21:16.685 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-12T16:21:16.685 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.686 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.686 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.686 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.686 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.686 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.687 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mds_autoscaler", 2023-09-12T16:21:16.687 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.687 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.687 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.687 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.688 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.688 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.688 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.688 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.688 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.689 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.689 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.689 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.689 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.689 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.690 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.690 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.690 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.690 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.690 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.690 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.691 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.691 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.691 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.691 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.691 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.692 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.692 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.692 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.692 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.692 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.693 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.693 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.693 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.693 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.693 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.694 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.694 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.694 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.694 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.694 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.695 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.695 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.695 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.695 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.695 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.695 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.696 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.696 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.696 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.696 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.696 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.697 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.697 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.697 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.697 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.697 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.698 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.698 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.698 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.698 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.698 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.698 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.699 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.699 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.699 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.699 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.699 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.700 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.700 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.700 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.700 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.700 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.700 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.701 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.701 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.701 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.701 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mirroring", 2023-09-12T16:21:16.701 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.702 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.702 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.702 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.702 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.702 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.703 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.703 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.703 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.703 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.703 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.703 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.704 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.704 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.704 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.704 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.704 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.705 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.705 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.705 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.705 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.705 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.706 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.706 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.706 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.706 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.706 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.707 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.707 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.707 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.707 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.707 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.708 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.708 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.708 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.708 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.708 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.708 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.709 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.709 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.709 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.709 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.709 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.710 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.710 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.710 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.710 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.710 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.710 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.711 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.711 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.711 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.711 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.711 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.712 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.712 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.712 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.712 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.712 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.713 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.713 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.713 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.713 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.713 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.713 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.714 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.714 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.714 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.714 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.714 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.715 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.715 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.715 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.715 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.715 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.716 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.716 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "nfs", 2023-09-12T16:21:16.716 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.716 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.716 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.716 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.717 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.717 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.717 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.717 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.717 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.718 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.718 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.718 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.718 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.718 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.719 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.719 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.719 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.719 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.719 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.720 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.720 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.720 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.720 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.720 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.720 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.721 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.721 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.721 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.721 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.721 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.722 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.722 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.722 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.722 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.722 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.723 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.723 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.723 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.723 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.723 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.723 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.724 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.724 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.724 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.724 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.724 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.724 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.725 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.725 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.725 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.725 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.725 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.726 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.726 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.726 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.726 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.726 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.727 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.727 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.727 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.727 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.727 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.727 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.728 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.728 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.728 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.728 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.728 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.729 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.729 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.729 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.729 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.729 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.729 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.730 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.730 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.730 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "orchestrator", 2023-09-12T16:21:16.730 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.731 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.731 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.731 INFO:teuthology.orchestra.run.smithi130.stdout: "fail_fs": { 2023-09-12T16:21:16.731 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "fail_fs", 2023-09-12T16:21:16.731 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.732 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.732 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.732 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.732 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.732 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.732 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.733 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-12T16:21:16.733 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.733 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.733 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.733 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.734 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.734 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.734 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.734 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.734 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.735 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.735 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.735 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.735 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.735 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.735 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.736 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.736 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.736 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.736 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.736 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.737 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.737 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.737 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.737 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.737 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.737 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.738 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.738 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.738 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.738 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.738 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.739 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.739 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.739 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.739 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.739 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.739 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.740 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.740 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.740 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.740 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.740 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.741 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.741 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.741 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.741 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.741 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.742 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.742 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.742 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.742 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.742 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.743 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.743 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.743 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.743 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.743 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.743 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.744 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.744 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.744 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.744 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.744 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.745 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.745 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.745 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.745 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.745 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.745 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.746 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.746 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.746 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.746 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.746 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.747 INFO:teuthology.orchestra.run.smithi130.stdout: "orchestrator": { 2023-09-12T16:21:16.747 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "orchestrator", 2023-09-12T16:21:16.747 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.747 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.747 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.748 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.748 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.748 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.748 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.748 INFO:teuthology.orchestra.run.smithi130.stdout: "cephadm", 2023-09-12T16:21:16.748 INFO:teuthology.orchestra.run.smithi130.stdout: "rook", 2023-09-12T16:21:16.749 INFO:teuthology.orchestra.run.smithi130.stdout: "test_orchestrator" 2023-09-12T16:21:16.749 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.749 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Orchestrator backend", 2023-09-12T16:21:16.749 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.749 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.750 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.750 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.750 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.750 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.750 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.751 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "osd_perf_query", 2023-09-12T16:21:16.751 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.751 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.751 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.751 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.752 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.752 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.752 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.752 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.752 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.753 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.753 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.753 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.753 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.753 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.753 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.754 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.754 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.754 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.754 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.754 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.755 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.755 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.755 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.755 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.755 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.755 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.756 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.756 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.756 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.756 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.756 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.757 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.757 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.757 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.757 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.757 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.757 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.758 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.758 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.758 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.758 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.758 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.759 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.759 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.759 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.759 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.759 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.760 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.760 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.760 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.760 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.760 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.760 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.761 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.761 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.761 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.761 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.761 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.762 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.762 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.762 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.762 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.762 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.763 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.763 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.763 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.763 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.763 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.763 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.764 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.764 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.764 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.764 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.764 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.765 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.765 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.765 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "osd_support", 2023-09-12T16:21:16.765 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.765 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.766 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.766 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.766 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.766 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.766 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.766 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.767 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.767 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.767 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.767 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.767 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.768 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.768 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.768 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.768 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.768 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.769 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.769 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.769 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.769 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.769 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.770 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.770 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.770 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.770 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.770 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.771 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.771 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.771 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.771 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.771 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.772 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.772 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.772 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.772 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.772 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.772 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.773 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.773 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.773 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.773 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.773 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.773 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.774 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.774 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.774 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.774 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.774 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.775 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.775 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.775 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.775 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.775 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.776 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.776 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.776 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.776 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.776 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.777 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.777 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.777 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.777 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.777 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.777 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.778 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.778 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.778 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.778 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.778 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.779 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.779 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.779 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.779 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.779 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.780 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pg_autoscaler", 2023-09-12T16:21:16.780 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.780 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.780 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.780 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.780 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.781 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.781 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.781 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.781 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.781 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.782 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.782 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.782 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.782 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.782 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.783 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.783 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.783 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.783 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.783 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.783 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.784 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.784 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.784 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.784 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.784 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.785 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.785 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.785 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.785 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.785 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.786 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.786 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.786 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.786 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.786 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.786 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.787 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.787 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.787 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.787 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.787 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.788 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.788 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.788 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.788 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.788 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.788 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.789 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.789 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.789 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.789 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.789 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.790 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.790 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.790 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.790 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.790 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.791 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.791 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.791 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.791 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.791 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.791 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.792 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.792 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.792 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.792 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.792 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.793 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.793 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.793 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.793 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.793 INFO:teuthology.orchestra.run.smithi130.stdout: "noautoscale": { 2023-09-12T16:21:16.793 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "noautoscale", 2023-09-12T16:21:16.794 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.794 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.794 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.794 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.794 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.795 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.795 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.795 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "global autoscale flag", 2023-09-12T16:21:16.795 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-12T16:21:16.795 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.796 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.796 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.796 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:16.796 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:16.796 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.797 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.797 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.797 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:16.797 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.797 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.797 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.798 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.798 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.798 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.798 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.798 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.799 INFO:teuthology.orchestra.run.smithi130.stdout: "threshold": { 2023-09-12T16:21:16.799 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "threshold", 2023-09-12T16:21:16.799 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:16.801 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.801 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.802 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "3.0", 2023-09-12T16:21:16.802 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1.0", 2023-09-12T16:21:16.802 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.802 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.802 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "scaling threshold", 2023-09-12T16:21:16.803 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-12T16:21:16.803 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.803 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.803 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.803 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.804 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.804 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.804 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "progress", 2023-09-12T16:21:16.804 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.804 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.805 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.805 INFO:teuthology.orchestra.run.smithi130.stdout: "allow_pg_recovery_event": { 2023-09-12T16:21:16.805 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "allow_pg_recovery_event", 2023-09-12T16:21:16.805 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.805 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.806 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.806 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.806 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.806 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.806 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.807 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-12T16:21:16.807 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.807 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.807 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.807 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.808 INFO:teuthology.orchestra.run.smithi130.stdout: "enabled": { 2023-09-12T16:21:16.808 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enabled", 2023-09-12T16:21:16.808 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.808 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.808 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.808 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.809 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.809 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.809 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.809 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.809 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.810 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.810 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.810 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.810 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.810 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.811 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.811 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.811 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.811 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.811 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.811 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.812 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.812 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.812 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.812 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.812 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.813 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.813 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.813 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.813 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.813 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.814 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.814 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.814 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.814 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.814 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.814 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.815 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.815 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.815 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.815 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.815 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.816 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.816 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.816 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.816 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.816 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.816 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.817 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.817 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.817 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.817 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.817 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.818 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.818 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.818 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.818 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.818 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.818 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.819 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.819 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.819 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.819 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.819 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.820 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.820 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.820 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.820 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.820 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.821 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.821 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.821 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.821 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.821 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.822 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.822 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.822 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.822 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.822 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.823 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.823 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.823 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.823 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.823 INFO:teuthology.orchestra.run.smithi130.stdout: "max_completed_events": { 2023-09-12T16:21:16.823 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_completed_events", 2023-09-12T16:21:16.824 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.824 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.824 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.824 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "50", 2023-09-12T16:21:16.824 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.825 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.825 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.825 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "number of past completed events to remember", 2023-09-12T16:21:16.825 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.825 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.826 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.826 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.826 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:16.826 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:16.826 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:16.827 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.827 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.827 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:16.827 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.827 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.827 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.828 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how long the module is going to sleep", 2023-09-12T16:21:16.828 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.828 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.828 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.828 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.828 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.829 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.829 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.829 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "prometheus", 2023-09-12T16:21:16.829 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.829 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.830 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.830 INFO:teuthology.orchestra.run.smithi130.stdout: "cache": { 2023-09-12T16:21:16.830 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cache", 2023-09-12T16:21:16.830 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.830 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.831 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.831 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.831 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.831 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.831 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.831 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.832 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.832 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.832 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.832 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.832 INFO:teuthology.orchestra.run.smithi130.stdout: "exclude_perf_counters": { 2023-09-12T16:21:16.833 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "exclude_perf_counters", 2023-09-12T16:21:16.833 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.833 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.833 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.833 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.834 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.834 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.834 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.834 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-12T16:21:16.834 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-12T16:21:16.835 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.835 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.835 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.835 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.835 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.835 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.836 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.836 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.836 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.836 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.836 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.837 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.837 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.837 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.837 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.837 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.837 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.838 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.838 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.838 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.838 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.838 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.839 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.839 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.839 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.839 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.839 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.840 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.840 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.840 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.840 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.840 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.841 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.841 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.841 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.841 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.841 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.842 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.842 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.842 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.842 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.842 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.842 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.843 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.843 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.843 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.843 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.843 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.844 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.844 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.844 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.844 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.844 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.845 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.845 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.845 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.845 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.845 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.845 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.846 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.846 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.846 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.846 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.846 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.847 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.847 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.847 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.847 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.847 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.847 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.848 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.848 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.848 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.848 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_stats_pools": { 2023-09-12T16:21:16.848 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_stats_pools", 2023-09-12T16:21:16.849 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.849 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.849 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.849 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.849 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.849 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.850 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.850 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.850 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.850 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.850 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.851 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.851 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-12T16:21:16.851 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-12T16:21:16.851 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.851 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.852 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.852 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "300", 2023-09-12T16:21:16.852 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.852 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.852 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.852 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.853 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.853 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.853 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.853 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.853 INFO:teuthology.orchestra.run.smithi130.stdout: "scrape_interval": { 2023-09-12T16:21:16.854 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "scrape_interval", 2023-09-12T16:21:16.854 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:16.854 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.854 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.854 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "15.0", 2023-09-12T16:21:16.855 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.855 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.855 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.855 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.855 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.855 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.856 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.856 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.856 INFO:teuthology.orchestra.run.smithi130.stdout: "server_addr": { 2023-09-12T16:21:16.856 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_addr", 2023-09-12T16:21:16.856 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.856 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.857 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.857 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "::", 2023-09-12T16:21:16.857 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.857 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.857 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.858 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-12T16:21:16.858 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.858 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.858 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.858 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.859 INFO:teuthology.orchestra.run.smithi130.stdout: "server_port": { 2023-09-12T16:21:16.859 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_port", 2023-09-12T16:21:16.859 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.859 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.859 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.859 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "9283", 2023-09-12T16:21:16.860 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.860 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.860 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.860 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-12T16:21:16.860 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.861 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.861 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.861 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.861 INFO:teuthology.orchestra.run.smithi130.stdout: "stale_cache_strategy": { 2023-09-12T16:21:16.861 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "stale_cache_strategy", 2023-09-12T16:21:16.862 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.862 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.862 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.862 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "log", 2023-09-12T16:21:16.862 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.863 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.863 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.863 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.863 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.863 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.863 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.864 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.864 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_behaviour": { 2023-09-12T16:21:16.864 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_behaviour", 2023-09-12T16:21:16.864 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.864 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.864 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.865 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "default", 2023-09-12T16:21:16.865 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.865 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.865 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.865 INFO:teuthology.orchestra.run.smithi130.stdout: "default", 2023-09-12T16:21:16.866 INFO:teuthology.orchestra.run.smithi130.stdout: "error" 2023-09-12T16:21:16.866 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.866 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.866 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.866 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.867 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.867 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.867 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_error_status_code": { 2023-09-12T16:21:16.867 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_error_status_code", 2023-09-12T16:21:16.867 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.867 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.868 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.868 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "500", 2023-09-12T16:21:16.868 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "400", 2023-09-12T16:21:16.868 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "599", 2023-09-12T16:21:16.868 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.868 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.869 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.869 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.869 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.869 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.869 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.869 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.870 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.870 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_support", 2023-09-12T16:21:16.870 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.870 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.870 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.871 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.871 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.871 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.871 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.871 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.871 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.872 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.872 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.872 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.872 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.872 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.872 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.873 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.873 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.873 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.873 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.873 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.874 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.874 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.874 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.874 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.874 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.875 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.875 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.875 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.875 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.875 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.876 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.876 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.876 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.876 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.876 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.877 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.877 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.877 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.877 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.877 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.877 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.878 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.878 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.878 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.878 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.878 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.879 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.879 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.879 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.879 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.879 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.880 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.880 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.880 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.880 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.880 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.881 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.881 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.881 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.881 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.881 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.881 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.882 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.882 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.882 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.882 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.882 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.883 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.883 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.883 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.883 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.883 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.883 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.884 INFO:teuthology.orchestra.run.smithi130.stdout: "max_concurrent_snap_create": { 2023-09-12T16:21:16.884 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_concurrent_snap_create", 2023-09-12T16:21:16.884 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.884 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.884 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.885 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:16.885 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.885 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.885 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.885 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.886 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.886 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.886 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.886 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.886 INFO:teuthology.orchestra.run.smithi130.stdout: "mirror_snapshot_schedule": { 2023-09-12T16:21:16.886 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mirror_snapshot_schedule", 2023-09-12T16:21:16.887 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.887 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.887 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.887 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.887 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.888 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.888 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.888 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.888 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.888 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.889 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.889 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.889 INFO:teuthology.orchestra.run.smithi130.stdout: "trash_purge_schedule": { 2023-09-12T16:21:16.889 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "trash_purge_schedule", 2023-09-12T16:21:16.889 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.889 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.890 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.890 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.890 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.890 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.890 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.891 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.891 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.891 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.891 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.891 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.892 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.892 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.892 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.892 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "restful", 2023-09-12T16:21:16.892 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.892 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.893 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.893 INFO:teuthology.orchestra.run.smithi130.stdout: "enable_auth": { 2023-09-12T16:21:16.893 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enable_auth", 2023-09-12T16:21:16.893 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.893 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.894 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.894 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.894 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.894 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.894 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.895 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.895 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.895 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.895 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.895 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.895 INFO:teuthology.orchestra.run.smithi130.stdout: "key_file": { 2023-09-12T16:21:16.896 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "key_file", 2023-09-12T16:21:16.896 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.896 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.896 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.896 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.897 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.897 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.897 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.897 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.897 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.898 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.898 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.898 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.898 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.898 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.899 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.899 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.899 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.899 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.899 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.900 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.900 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.900 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.900 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.900 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.900 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.901 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.901 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.901 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.901 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.901 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.902 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.902 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.902 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.902 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.902 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.903 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.903 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.903 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.903 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.903 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.903 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.904 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.904 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.904 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.904 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.904 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.905 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.905 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.905 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.905 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.905 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.905 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.906 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.906 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.906 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.906 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.906 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.906 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.907 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.907 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.907 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.907 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.907 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.908 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.908 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.908 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.908 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.908 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.909 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.909 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.909 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.909 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.909 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.909 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.910 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.910 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.910 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.910 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.910 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.911 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.911 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.911 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.912 INFO:teuthology.orchestra.run.smithi130.stdout: "server_addr": { 2023-09-12T16:21:16.912 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_addr", 2023-09-12T16:21:16.912 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.912 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.912 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.913 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.913 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.913 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.913 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.913 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.914 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.914 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.914 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.914 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.914 INFO:teuthology.orchestra.run.smithi130.stdout: "server_port": { 2023-09-12T16:21:16.914 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_port", 2023-09-12T16:21:16.915 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.915 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.915 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.915 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.915 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.916 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.916 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.916 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.916 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.916 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.917 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.917 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.917 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.917 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.917 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.917 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rgw", 2023-09-12T16:21:16.918 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.918 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.918 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.918 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.918 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.919 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.919 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.919 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.919 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.919 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.920 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.920 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.920 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.920 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.920 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.921 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.921 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.921 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.921 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.921 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.921 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.922 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.922 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.922 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.922 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.922 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.923 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.923 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.923 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.923 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.923 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.923 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.924 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.924 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.924 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.924 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.924 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.925 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.925 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.925 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.925 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.925 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.925 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.926 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.926 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.926 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.926 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.926 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.927 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.927 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.927 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.927 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.927 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.928 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.928 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.928 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.928 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.928 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.928 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.929 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.929 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.929 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.929 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.929 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.930 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.930 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.930 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.930 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.930 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.931 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.931 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.931 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.931 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.931 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.932 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.932 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.932 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rook", 2023-09-12T16:21:16.932 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.932 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.932 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.933 INFO:teuthology.orchestra.run.smithi130.stdout: "drive_group_interval": { 2023-09-12T16:21:16.933 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "drive_group_interval", 2023-09-12T16:21:16.933 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:16.933 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.933 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.934 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "300.0", 2023-09-12T16:21:16.934 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.934 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.934 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.934 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-12T16:21:16.935 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.935 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.935 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.935 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.935 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.935 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.936 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.936 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.936 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.936 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.936 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.936 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.937 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.937 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.937 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.937 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.937 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.938 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.938 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.938 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.938 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.938 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.939 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.939 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.939 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.939 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.939 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.939 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.940 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.940 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.940 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.940 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.940 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.941 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.941 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.941 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.941 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.941 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.941 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.942 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.942 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.942 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.942 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.942 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.943 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.943 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.943 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.943 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.943 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.943 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.944 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.944 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.944 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.944 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.944 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.945 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.945 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.945 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.945 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.945 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.945 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.946 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.946 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.946 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.946 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.946 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.947 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.947 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.947 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.947 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.947 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.948 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.948 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.948 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.948 INFO:teuthology.orchestra.run.smithi130.stdout: "storage_class": { 2023-09-12T16:21:16.948 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "storage_class", 2023-09-12T16:21:16.949 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.949 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.949 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.949 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "local", 2023-09-12T16:21:16.949 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.950 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.950 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.950 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-12T16:21:16.950 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.950 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.951 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.951 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.951 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.951 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.951 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.952 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "selftest", 2023-09-12T16:21:16.952 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.952 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.952 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.952 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:16.953 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:16.953 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.953 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.953 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.953 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.953 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.954 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.954 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.954 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.954 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.954 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.955 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.955 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.955 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.955 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.955 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.956 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.956 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.956 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.956 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.956 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:16.957 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:16.957 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.957 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.957 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.957 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.957 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.958 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.958 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.958 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.958 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.959 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.959 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.959 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.959 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:16.960 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:16.960 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.960 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.960 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.960 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:16.961 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.961 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.961 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:16.961 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:16.961 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:16.962 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:16.962 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:16.962 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:16.962 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:16.962 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:16.963 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.963 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.963 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.963 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.963 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.964 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:16.964 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:16.964 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.964 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.964 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:16.964 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:16.965 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.965 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.965 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.965 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.965 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.966 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.966 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.966 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.966 INFO:teuthology.orchestra.run.smithi130.stdout: "roption1": { 2023-09-12T16:21:16.966 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "roption1", 2023-09-12T16:21:16.967 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.967 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.967 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.967 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.967 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.968 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.968 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.968 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.968 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.968 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.968 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.969 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.969 INFO:teuthology.orchestra.run.smithi130.stdout: "roption2": { 2023-09-12T16:21:16.969 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "roption2", 2023-09-12T16:21:16.969 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.969 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.969 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.970 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "xyz", 2023-09-12T16:21:16.970 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.970 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.970 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.971 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.971 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.971 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.971 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.971 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.971 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption1": { 2023-09-12T16:21:16.972 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption1", 2023-09-12T16:21:16.972 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.972 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.972 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.972 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.972 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.973 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.973 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.973 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.973 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.973 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.974 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.974 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.974 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption2": { 2023-09-12T16:21:16.974 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption2", 2023-09-12T16:21:16.974 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.975 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.975 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.975 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.975 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.975 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.975 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.976 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.976 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.976 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.976 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.976 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.977 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption3": { 2023-09-12T16:21:16.977 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption3", 2023-09-12T16:21:16.977 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:16.977 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.977 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.977 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.978 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.978 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.978 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.978 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.978 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.979 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.979 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.979 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.979 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption4": { 2023-09-12T16:21:16.979 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption4", 2023-09-12T16:21:16.979 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.980 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.980 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.980 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.980 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.980 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.980 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.981 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.981 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.981 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.981 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.981 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.982 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption5": { 2023-09-12T16:21:16.982 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption5", 2023-09-12T16:21:16.982 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.982 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.982 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.983 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.983 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.983 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.983 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.983 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.984 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.984 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.984 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.984 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.985 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption6": { 2023-09-12T16:21:16.985 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption6", 2023-09-12T16:21:16.985 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:16.985 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.985 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.986 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:16.986 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.986 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.986 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.986 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.987 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.987 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.987 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.987 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.987 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption7": { 2023-09-12T16:21:16.988 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption7", 2023-09-12T16:21:16.988 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:16.988 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.988 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.988 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.989 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:16.989 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "42", 2023-09-12T16:21:16.989 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.989 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.989 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.989 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.990 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.990 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.990 INFO:teuthology.orchestra.run.smithi130.stdout: "testkey": { 2023-09-12T16:21:16.990 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "testkey", 2023-09-12T16:21:16.990 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.991 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.991 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.991 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.991 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.991 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.992 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.992 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.992 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.992 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.992 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.992 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.993 INFO:teuthology.orchestra.run.smithi130.stdout: "testlkey": { 2023-09-12T16:21:16.993 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "testlkey", 2023-09-12T16:21:16.993 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.993 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.993 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.994 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.994 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.994 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.994 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.994 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.994 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.995 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.995 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.995 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.995 INFO:teuthology.orchestra.run.smithi130.stdout: "testnewline": { 2023-09-12T16:21:16.995 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "testnewline", 2023-09-12T16:21:16.996 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:16.996 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:16.996 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:16.996 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:16.996 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:16.997 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:16.997 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:16.997 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:16.997 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:16.997 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:16.997 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:16.998 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.998 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:16.998 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:16.998 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:16.998 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "snap_schedule", 2023-09-12T16:21:16.999 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:16.999 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:16.999 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:16.999 INFO:teuthology.orchestra.run.smithi130.stdout: "allow_m_granularity": { 2023-09-12T16:21:16.999 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "allow_m_granularity", 2023-09-12T16:21:17.000 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.000 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.000 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.000 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.000 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.000 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.001 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.001 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "allow minute scheduled snapshots", 2023-09-12T16:21:17.001 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.001 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.001 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.002 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.002 INFO:teuthology.orchestra.run.smithi130.stdout: "dump_on_update": { 2023-09-12T16:21:17.002 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "dump_on_update", 2023-09-12T16:21:17.002 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.002 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.003 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.003 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.004 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.005 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.005 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.005 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "dump database to debug log on update", 2023-09-12T16:21:17.006 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.006 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.006 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.006 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.006 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.007 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.007 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.007 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.007 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.007 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.008 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.008 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.008 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.008 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.008 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.009 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.009 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.009 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.009 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.009 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.010 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.010 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.010 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.010 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.010 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.011 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.011 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.011 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.011 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.011 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.012 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.012 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.012 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.012 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.012 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.013 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.013 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.013 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.013 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.013 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.013 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.014 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.014 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.014 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.014 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.014 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.015 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.015 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.015 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.015 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.015 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.016 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.016 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.016 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.016 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.016 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.017 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.017 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.017 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.017 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.017 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.018 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.018 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.018 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.018 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.018 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.018 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.019 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.019 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.019 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.019 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.019 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.020 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.020 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.020 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.020 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.020 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.020 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "stats", 2023-09-12T16:21:17.021 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.021 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.021 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.021 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.021 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.022 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.022 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.022 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.022 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.022 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.023 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.023 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.023 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.023 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.023 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.023 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.024 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.024 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.024 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.024 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.024 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.025 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.025 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.025 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.025 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.025 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.026 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.026 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.026 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.026 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.026 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.027 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.027 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.027 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.027 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.027 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.027 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.028 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.028 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.028 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.028 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.028 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.029 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.029 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.029 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.029 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.029 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.029 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.030 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.030 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.030 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.030 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.030 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.031 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.031 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.031 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.031 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.031 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.032 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.032 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.032 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.032 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.032 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.032 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.033 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.033 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.033 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.033 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.033 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.034 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.034 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.034 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.034 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.034 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.035 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.035 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.035 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "status", 2023-09-12T16:21:17.035 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.035 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.035 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.036 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.036 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.036 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.036 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.036 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.037 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.037 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.037 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.037 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.037 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.038 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.038 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.038 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.038 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.038 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.039 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.039 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.039 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.039 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.039 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.039 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.040 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.040 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.040 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.040 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.040 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.041 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.041 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.041 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.041 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.041 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.041 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.042 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.042 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.042 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.042 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.042 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.043 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.043 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.043 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.043 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.043 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.043 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.044 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.044 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.044 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.044 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.044 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.045 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.045 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.045 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.045 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.045 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.045 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.046 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.046 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.046 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.046 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.046 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.047 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.047 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.047 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.047 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.047 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.047 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.048 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.048 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.048 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.048 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.048 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.049 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.049 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.049 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.049 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "telegraf", 2023-09-12T16:21:17.049 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.050 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.050 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.050 INFO:teuthology.orchestra.run.smithi130.stdout: "address": { 2023-09-12T16:21:17.050 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "address", 2023-09-12T16:21:17.050 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.051 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.051 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.051 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-12T16:21:17.051 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.051 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.051 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.052 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.052 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.052 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.052 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.052 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.053 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:17.053 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:17.053 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.053 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.053 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.053 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "15", 2023-09-12T16:21:17.054 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.054 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.054 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.054 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.054 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.055 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.055 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.055 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.055 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.055 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.055 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.056 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.056 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.056 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.056 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.056 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.057 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.057 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.057 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.057 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.057 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.057 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.058 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.058 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.058 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.058 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.058 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.059 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.059 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.059 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.059 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.059 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.060 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.060 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.060 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.060 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.060 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.061 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.061 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.061 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.061 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.061 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.062 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.062 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.062 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.062 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.062 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.063 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.063 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.063 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.063 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.063 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.063 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.064 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.064 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.064 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.064 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.064 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.065 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.065 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.065 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.065 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.065 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.065 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.066 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.066 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.066 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.066 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.066 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.067 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.067 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.067 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.067 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.067 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.067 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.068 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.068 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.068 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.068 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.068 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.069 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.069 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "telemetry", 2023-09-12T16:21:17.069 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.069 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.069 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.070 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_basic": { 2023-09-12T16:21:17.070 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_basic", 2023-09-12T16:21:17.070 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.070 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.070 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.070 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.071 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.071 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.071 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.071 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-12T16:21:17.071 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.072 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.072 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.072 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.072 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_crash": { 2023-09-12T16:21:17.072 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_crash", 2023-09-12T16:21:17.073 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.073 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.073 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.073 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.073 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.073 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.074 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.074 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-12T16:21:17.074 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.074 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.074 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.075 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.075 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_device": { 2023-09-12T16:21:17.075 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_device", 2023-09-12T16:21:17.075 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.075 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.076 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.076 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.076 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.076 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.076 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.076 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-12T16:21:17.077 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.077 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.077 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.077 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.077 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_ident": { 2023-09-12T16:21:17.078 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_ident", 2023-09-12T16:21:17.078 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.078 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.078 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.078 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.079 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.079 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.079 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.079 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-12T16:21:17.079 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.079 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.080 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.080 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.080 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_perf": { 2023-09-12T16:21:17.080 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_perf", 2023-09-12T16:21:17.080 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.081 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.081 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.081 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.081 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.081 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.082 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.082 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-12T16:21:17.082 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.082 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.082 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.083 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.083 INFO:teuthology.orchestra.run.smithi130.stdout: "contact": { 2023-09-12T16:21:17.083 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "contact", 2023-09-12T16:21:17.083 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.083 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.083 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.084 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.084 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.084 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.084 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.084 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.085 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.085 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.085 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.085 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.085 INFO:teuthology.orchestra.run.smithi130.stdout: "description": { 2023-09-12T16:21:17.086 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "description", 2023-09-12T16:21:17.086 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.086 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.086 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.086 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.086 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.087 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.087 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.087 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.087 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.088 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.088 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.088 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.088 INFO:teuthology.orchestra.run.smithi130.stdout: "device_url": { 2023-09-12T16:21:17.088 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "device_url", 2023-09-12T16:21:17.088 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.089 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.089 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.089 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-12T16:21:17.089 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.089 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.090 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.090 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.090 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.090 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.090 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.091 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.091 INFO:teuthology.orchestra.run.smithi130.stdout: "enabled": { 2023-09-12T16:21:17.091 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enabled", 2023-09-12T16:21:17.091 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.091 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.091 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.092 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.092 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.092 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.092 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.092 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.093 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.093 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.093 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.093 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.093 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:17.093 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:17.094 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.094 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.094 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.094 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "24", 2023-09-12T16:21:17.094 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "8", 2023-09-12T16:21:17.095 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.095 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.095 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.095 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.095 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.096 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.096 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.096 INFO:teuthology.orchestra.run.smithi130.stdout: "last_opt_revision": { 2023-09-12T16:21:17.096 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "last_opt_revision", 2023-09-12T16:21:17.096 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.097 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.097 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.097 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "1", 2023-09-12T16:21:17.097 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.097 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.098 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.098 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.098 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.098 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.098 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.098 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.099 INFO:teuthology.orchestra.run.smithi130.stdout: "leaderboard": { 2023-09-12T16:21:17.099 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "leaderboard", 2023-09-12T16:21:17.099 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.099 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.099 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.100 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.100 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.100 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.100 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.100 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.101 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.101 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.101 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.101 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.101 INFO:teuthology.orchestra.run.smithi130.stdout: "leaderboard_description": { 2023-09-12T16:21:17.101 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "leaderboard_description", 2023-09-12T16:21:17.102 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.102 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.102 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.102 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.102 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.103 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.103 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.103 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.103 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.103 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.104 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.104 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.104 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.104 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.104 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.104 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.105 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.105 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.105 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.105 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.105 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.106 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.106 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.106 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.106 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.106 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.106 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.107 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.107 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.107 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.107 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.107 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.108 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.108 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.108 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.108 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.108 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.108 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.109 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.109 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.109 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.109 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.109 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.109 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.110 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.110 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.110 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.110 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.110 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.111 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.111 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.111 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.111 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.111 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.112 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.112 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.112 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.112 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.112 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.112 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.113 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.113 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.113 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.113 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.113 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.114 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.114 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.114 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.114 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.114 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.115 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.115 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.115 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.115 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.115 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.115 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.116 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.116 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.116 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.116 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.116 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.117 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.117 INFO:teuthology.orchestra.run.smithi130.stdout: "organization": { 2023-09-12T16:21:17.117 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "organization", 2023-09-12T16:21:17.117 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.117 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.118 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.118 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.118 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.118 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.118 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.119 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.119 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.119 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.119 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.119 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.120 INFO:teuthology.orchestra.run.smithi130.stdout: "proxy": { 2023-09-12T16:21:17.120 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "proxy", 2023-09-12T16:21:17.120 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.120 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.120 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.120 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.121 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.121 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.121 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.121 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.121 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.122 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.122 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.122 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.122 INFO:teuthology.orchestra.run.smithi130.stdout: "url": { 2023-09-12T16:21:17.122 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "url", 2023-09-12T16:21:17.123 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.123 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.123 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.123 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-12T16:21:17.123 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.123 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.124 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.124 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.124 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.124 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.124 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.124 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.125 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.125 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.125 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.125 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "test_orchestrator", 2023-09-12T16:21:17.125 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.126 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.126 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.126 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.126 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.126 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.126 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.127 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.127 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.127 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.127 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.127 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.127 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.128 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.128 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.128 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.128 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.128 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.129 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.129 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.129 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.129 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.129 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.129 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.130 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.130 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.130 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.130 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.130 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.131 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.131 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.131 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.131 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.131 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.131 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.132 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.132 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.132 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.132 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.133 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.133 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.133 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.133 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.133 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.134 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.134 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.134 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.134 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.134 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.135 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.135 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.135 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.135 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.135 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.135 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.136 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.136 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.136 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.136 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.136 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.137 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.137 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.137 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.137 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.137 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.137 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.138 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.138 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.138 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.138 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.138 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.139 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.139 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.139 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.139 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.139 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.139 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "volumes", 2023-09-12T16:21:17.140 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.140 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.140 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.140 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.140 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.141 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.141 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.141 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.141 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.141 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.142 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.142 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.142 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.142 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.142 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.143 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.143 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.143 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.143 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.143 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.143 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.144 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.144 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.144 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.144 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.144 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.145 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.145 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.145 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.145 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.145 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.146 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.146 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.146 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.146 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.146 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.147 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.147 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.147 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.147 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.147 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.147 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.148 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.148 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.148 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.148 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.148 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.149 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.149 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.149 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.149 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.149 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.149 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.150 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.150 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.150 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.150 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.150 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.151 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.151 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.151 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.151 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.151 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.152 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.152 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.152 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.152 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.152 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.153 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.153 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.153 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.153 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.153 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.154 INFO:teuthology.orchestra.run.smithi130.stdout: "max_concurrent_clones": { 2023-09-12T16:21:17.154 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_concurrent_clones", 2023-09-12T16:21:17.154 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.154 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.154 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.154 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "4", 2023-09-12T16:21:17.155 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.155 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.155 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.155 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-12T16:21:17.155 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.156 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.156 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.156 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.156 INFO:teuthology.orchestra.run.smithi130.stdout: "snapshot_clone_delay": { 2023-09-12T16:21:17.156 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "snapshot_clone_delay", 2023-09-12T16:21:17.157 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.157 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.157 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.157 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:17.157 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.158 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.158 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.158 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-12T16:21:17.158 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.158 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.159 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.159 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.159 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.159 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.159 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.160 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix", 2023-09-12T16:21:17.160 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.160 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.160 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.160 INFO:teuthology.orchestra.run.smithi130.stdout: "discovery_interval": { 2023-09-12T16:21:17.160 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "discovery_interval", 2023-09-12T16:21:17.161 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:17.161 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.161 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.161 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "100", 2023-09-12T16:21:17.161 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.162 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.162 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.162 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.162 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.162 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.163 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.163 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.163 INFO:teuthology.orchestra.run.smithi130.stdout: "identifier": { 2023-09-12T16:21:17.163 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "identifier", 2023-09-12T16:21:17.163 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.163 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.164 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.164 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.164 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.164 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.164 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.165 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.165 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.165 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.165 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.165 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.166 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:17.166 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:17.166 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.166 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.166 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.167 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:17.167 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.167 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.167 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.167 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.167 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.168 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.168 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.168 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.168 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.168 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.169 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.169 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.169 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.169 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.169 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.170 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.170 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.170 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.170 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.170 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.171 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.171 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.171 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.171 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.171 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.172 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.172 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.172 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.172 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.172 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.173 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.173 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.173 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.173 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.173 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.174 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.174 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.174 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.174 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.174 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.174 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.175 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.175 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.175 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.175 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.175 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.176 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.176 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.176 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.176 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.176 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.177 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.177 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.177 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.177 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.177 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.177 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.178 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.178 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.178 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.178 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.178 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.179 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.179 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.179 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.179 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.179 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.180 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.180 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.180 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.180 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.180 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.181 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.181 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.181 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.181 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.181 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.181 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.182 INFO:teuthology.orchestra.run.smithi130.stdout: "zabbix_host": { 2023-09-12T16:21:17.182 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix_host", 2023-09-12T16:21:17.182 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.182 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.182 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.183 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.183 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.183 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.183 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.183 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.184 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.184 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.184 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.184 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.184 INFO:teuthology.orchestra.run.smithi130.stdout: "zabbix_port": { 2023-09-12T16:21:17.185 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix_port", 2023-09-12T16:21:17.185 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.185 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.185 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.185 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10051", 2023-09-12T16:21:17.186 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.186 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.186 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.186 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.186 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.186 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.187 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.187 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.187 INFO:teuthology.orchestra.run.smithi130.stdout: "zabbix_sender": { 2023-09-12T16:21:17.187 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix_sender", 2023-09-12T16:21:17.187 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.188 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.188 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.188 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-12T16:21:17.188 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.188 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.189 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.189 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.189 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.189 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.189 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.189 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.190 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.190 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.190 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T16:21:17.190 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.190 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.191 INFO:teuthology.orchestra.run.smithi130.stdout: "gid": 24205, 2023-09-12T16:21:17.191 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "z", 2023-09-12T16:21:17.191 INFO:teuthology.orchestra.run.smithi130.stdout: "mgr_features": 4540138322906710015, 2023-09-12T16:21:17.191 INFO:teuthology.orchestra.run.smithi130.stdout: "available_modules": [ 2023-09-12T16:21:17.191 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.192 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "alerts", 2023-09-12T16:21:17.192 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.192 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.192 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.192 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:17.192 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:17.193 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.193 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.193 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.193 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:17.193 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.194 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.194 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.194 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How frequently to reexamine health status", 2023-09-12T16:21:17.194 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.194 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.195 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.195 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.195 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.195 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.195 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.195 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.196 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.196 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.196 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.196 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.196 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.197 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.197 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.197 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.197 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.197 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.198 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.198 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.198 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.198 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.198 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.198 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.199 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.199 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.199 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.199 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.199 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.200 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.200 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.200 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.200 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.200 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.201 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.201 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.201 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.201 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.201 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.202 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.202 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.202 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.202 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.202 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.202 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.203 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.203 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.203 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.203 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.203 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.204 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.204 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.204 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.204 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.204 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.205 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.205 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.205 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.205 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.205 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.205 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.206 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.206 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.206 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.206 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.206 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.207 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.207 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.207 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.207 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.207 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.208 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.208 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.208 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.208 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_destination": { 2023-09-12T16:21:17.208 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_destination", 2023-09-12T16:21:17.208 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.209 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.209 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.209 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.209 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.209 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.210 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.210 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Email address to send alerts to", 2023-09-12T16:21:17.210 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.210 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.210 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.211 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.211 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_from_name": { 2023-09-12T16:21:17.211 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_from_name", 2023-09-12T16:21:17.211 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.211 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.211 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.212 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "Ceph", 2023-09-12T16:21:17.213 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.213 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.213 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.213 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Email From: name", 2023-09-12T16:21:17.214 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.214 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.214 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.214 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.214 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_host": { 2023-09-12T16:21:17.214 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_host", 2023-09-12T16:21:17.215 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.215 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.215 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.215 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.215 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.216 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.216 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.216 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SMTP server", 2023-09-12T16:21:17.216 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.216 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.217 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.217 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.217 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_password": { 2023-09-12T16:21:17.217 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_password", 2023-09-12T16:21:17.217 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.217 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.218 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.218 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.218 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.218 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.218 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.219 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Password to authenticate with", 2023-09-12T16:21:17.219 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.219 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.219 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.219 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.220 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_port": { 2023-09-12T16:21:17.220 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_port", 2023-09-12T16:21:17.220 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.220 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.220 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.221 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "465", 2023-09-12T16:21:17.221 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.221 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.221 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.221 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SMTP port", 2023-09-12T16:21:17.222 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.222 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.222 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.222 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.222 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_sender": { 2023-09-12T16:21:17.222 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_sender", 2023-09-12T16:21:17.223 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.223 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.223 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.223 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.223 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.224 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.224 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.224 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SMTP envelope sender", 2023-09-12T16:21:17.224 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.224 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.225 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.225 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.225 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_ssl": { 2023-09-12T16:21:17.225 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_ssl", 2023-09-12T16:21:17.225 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.226 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.226 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.226 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.226 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.227 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.227 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.227 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-12T16:21:17.227 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.227 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.228 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.228 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.228 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_user": { 2023-09-12T16:21:17.228 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_user", 2023-09-12T16:21:17.228 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.229 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.229 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.229 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.229 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.229 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.230 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.230 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "User to authenticate as", 2023-09-12T16:21:17.230 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.230 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.230 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.231 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.231 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.231 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.231 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.231 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "balancer", 2023-09-12T16:21:17.232 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.232 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.232 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.232 INFO:teuthology.orchestra.run.smithi130.stdout: "active": { 2023-09-12T16:21:17.232 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "active", 2023-09-12T16:21:17.233 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.233 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.233 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.233 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.233 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.233 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.234 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.234 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "automatically balance PGs across cluster", 2023-09-12T16:21:17.234 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.234 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.234 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.235 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.235 INFO:teuthology.orchestra.run.smithi130.stdout: "begin_time": { 2023-09-12T16:21:17.235 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "begin_time", 2023-09-12T16:21:17.235 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.235 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.236 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.236 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0000", 2023-09-12T16:21:17.236 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.236 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.236 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.236 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "beginning time of day to automatically balance", 2023-09-12T16:21:17.237 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-12T16:21:17.237 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.237 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.237 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.237 INFO:teuthology.orchestra.run.smithi130.stdout: "begin_weekday": { 2023-09-12T16:21:17.238 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "begin_weekday", 2023-09-12T16:21:17.238 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:17.238 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.238 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.238 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:17.238 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "0", 2023-09-12T16:21:17.239 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "6", 2023-09-12T16:21:17.239 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.239 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-12T16:21:17.239 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-12T16:21:17.239 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.240 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.240 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.240 INFO:teuthology.orchestra.run.smithi130.stdout: "crush_compat_max_iterations": { 2023-09-12T16:21:17.240 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crush_compat_max_iterations", 2023-09-12T16:21:17.240 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:17.240 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.241 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.241 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "25", 2023-09-12T16:21:17.241 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:17.241 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "250", 2023-09-12T16:21:17.241 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.242 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-12T16:21:17.242 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.242 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.242 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.243 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.243 INFO:teuthology.orchestra.run.smithi130.stdout: "crush_compat_metrics": { 2023-09-12T16:21:17.243 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crush_compat_metrics", 2023-09-12T16:21:17.243 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.243 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.243 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.244 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "pgs,objects,bytes", 2023-09-12T16:21:17.244 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.244 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.244 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.244 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-12T16:21:17.245 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-12T16:21:17.245 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.245 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.245 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.245 INFO:teuthology.orchestra.run.smithi130.stdout: "crush_compat_step": { 2023-09-12T16:21:17.246 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crush_compat_step", 2023-09-12T16:21:17.246 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:17.246 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.246 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.246 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0.5", 2023-09-12T16:21:17.246 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "0.001", 2023-09-12T16:21:17.247 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "0.999", 2023-09-12T16:21:17.247 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.247 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "aggressiveness of optimization", 2023-09-12T16:21:17.247 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-12T16:21:17.247 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.248 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.248 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.248 INFO:teuthology.orchestra.run.smithi130.stdout: "end_time": { 2023-09-12T16:21:17.248 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "end_time", 2023-09-12T16:21:17.248 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.248 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.249 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.249 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "2359", 2023-09-12T16:21:17.249 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.249 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.249 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.250 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "ending time of day to automatically balance", 2023-09-12T16:21:17.250 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-12T16:21:17.250 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.250 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.250 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.251 INFO:teuthology.orchestra.run.smithi130.stdout: "end_weekday": { 2023-09-12T16:21:17.251 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "end_weekday", 2023-09-12T16:21:17.251 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:17.251 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.251 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.251 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:17.252 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "0", 2023-09-12T16:21:17.252 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "6", 2023-09-12T16:21:17.252 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.252 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-12T16:21:17.252 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-12T16:21:17.253 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.253 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.253 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.253 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.253 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.254 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.254 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.254 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.254 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.254 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.254 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.255 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.255 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.255 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.255 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.255 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.256 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.256 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.256 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.256 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.256 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.256 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.257 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.257 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.257 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.257 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.257 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.257 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.258 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.258 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.258 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.258 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.258 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.259 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.259 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.259 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.259 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.259 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.259 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.260 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.260 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.260 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.260 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.260 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.261 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.261 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.261 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.261 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.261 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.261 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.262 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.262 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.262 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.262 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.262 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.263 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.263 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.263 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.263 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.263 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.263 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.264 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.264 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.264 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.264 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.264 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.264 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.265 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.265 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.265 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.265 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.265 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.266 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.266 INFO:teuthology.orchestra.run.smithi130.stdout: "min_score": { 2023-09-12T16:21:17.266 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "min_score", 2023-09-12T16:21:17.266 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:17.266 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.267 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.267 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:17.267 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.267 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.267 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.267 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-12T16:21:17.268 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.268 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.268 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.268 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.268 INFO:teuthology.orchestra.run.smithi130.stdout: "mode": { 2023-09-12T16:21:17.269 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mode", 2023-09-12T16:21:17.269 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.269 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.269 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.269 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "upmap", 2023-09-12T16:21:17.269 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.270 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.270 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.270 INFO:teuthology.orchestra.run.smithi130.stdout: "crush-compat", 2023-09-12T16:21:17.270 INFO:teuthology.orchestra.run.smithi130.stdout: "none", 2023-09-12T16:21:17.270 INFO:teuthology.orchestra.run.smithi130.stdout: "upmap" 2023-09-12T16:21:17.271 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.271 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Balancer mode", 2023-09-12T16:21:17.271 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.271 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.271 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.272 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.272 INFO:teuthology.orchestra.run.smithi130.stdout: "pool_ids": { 2023-09-12T16:21:17.272 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pool_ids", 2023-09-12T16:21:17.272 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.272 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.272 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.273 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.273 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.273 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.273 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.273 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-12T16:21:17.273 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.274 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.274 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.274 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.274 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:17.274 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:17.274 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.275 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.275 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.275 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:17.275 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.276 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.276 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.276 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-12T16:21:17.276 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.276 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.276 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.277 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.277 INFO:teuthology.orchestra.run.smithi130.stdout: "upmap_max_deviation": { 2023-09-12T16:21:17.277 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "upmap_max_deviation", 2023-09-12T16:21:17.277 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.277 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.278 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.278 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:17.278 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:17.278 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.278 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.279 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-12T16:21:17.279 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-12T16:21:17.279 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.279 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.279 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.280 INFO:teuthology.orchestra.run.smithi130.stdout: "upmap_max_optimizations": { 2023-09-12T16:21:17.280 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "upmap_max_optimizations", 2023-09-12T16:21:17.280 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:17.280 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.280 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.280 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:17.281 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.281 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.281 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.281 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-12T16:21:17.281 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.282 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.282 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.282 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.282 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.282 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.283 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.283 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cephadm", 2023-09-12T16:21:17.283 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.283 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.283 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.284 INFO:teuthology.orchestra.run.smithi130.stdout: "agent_down_multiplier": { 2023-09-12T16:21:17.284 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "agent_down_multiplier", 2023-09-12T16:21:17.284 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:17.284 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.284 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.285 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "3.0", 2023-09-12T16:21:17.285 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.285 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.285 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.285 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-12T16:21:17.285 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.286 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.286 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.286 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.286 INFO:teuthology.orchestra.run.smithi130.stdout: "agent_refresh_rate": { 2023-09-12T16:21:17.286 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "agent_refresh_rate", 2023-09-12T16:21:17.287 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.287 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.287 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.287 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "20", 2023-09-12T16:21:17.287 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.287 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.288 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.288 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-12T16:21:17.288 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.288 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.288 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.289 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.289 INFO:teuthology.orchestra.run.smithi130.stdout: "agent_starting_port": { 2023-09-12T16:21:17.289 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "agent_starting_port", 2023-09-12T16:21:17.289 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.289 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.289 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.290 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "4721", 2023-09-12T16:21:17.290 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.290 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.290 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.290 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-12T16:21:17.291 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.291 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.291 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.291 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.291 INFO:teuthology.orchestra.run.smithi130.stdout: "allow_ptrace": { 2023-09-12T16:21:17.291 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "allow_ptrace", 2023-09-12T16:21:17.292 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.292 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.292 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.292 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.292 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.293 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.293 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.293 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-12T16:21:17.293 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-12T16:21:17.293 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.294 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.294 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.294 INFO:teuthology.orchestra.run.smithi130.stdout: "autotune_interval": { 2023-09-12T16:21:17.294 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "autotune_interval", 2023-09-12T16:21:17.294 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.295 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.295 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.295 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:17.295 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.295 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.295 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.296 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-12T16:21:17.296 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.296 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.296 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.296 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.296 INFO:teuthology.orchestra.run.smithi130.stdout: "autotune_memory_target_ratio": { 2023-09-12T16:21:17.297 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "autotune_memory_target_ratio", 2023-09-12T16:21:17.297 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:17.297 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.297 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.297 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0.7", 2023-09-12T16:21:17.298 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.298 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.298 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.298 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-12T16:21:17.298 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.299 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.299 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.299 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.299 INFO:teuthology.orchestra.run.smithi130.stdout: "cgroups_split": { 2023-09-12T16:21:17.299 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cgroups_split", 2023-09-12T16:21:17.299 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.300 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.300 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.300 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.300 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.300 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.301 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.301 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-12T16:21:17.301 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.301 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.301 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.301 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.302 INFO:teuthology.orchestra.run.smithi130.stdout: "config_checks_enabled": { 2023-09-12T16:21:17.302 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "config_checks_enabled", 2023-09-12T16:21:17.302 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.302 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.302 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.303 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.303 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.303 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.303 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.303 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-12T16:21:17.303 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.304 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.304 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.304 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.304 INFO:teuthology.orchestra.run.smithi130.stdout: "config_dashboard": { 2023-09-12T16:21:17.304 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "config_dashboard", 2023-09-12T16:21:17.305 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.305 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.305 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.305 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.305 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.305 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.306 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.306 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-12T16:21:17.306 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.306 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.306 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.307 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.307 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_alertmanager": { 2023-09-12T16:21:17.307 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_alertmanager", 2023-09-12T16:21:17.307 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.307 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.307 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.308 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-12T16:21:17.308 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.308 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.308 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.308 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:17.309 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.309 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.309 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.309 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.309 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_base": { 2023-09-12T16:21:17.309 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_base", 2023-09-12T16:21:17.310 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.310 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.310 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.310 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-12T16:21:17.310 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.311 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.311 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.311 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Container image name, without the tag", 2023-09-12T16:21:17.311 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.311 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.311 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.312 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.312 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_elasticsearch": { 2023-09-12T16:21:17.312 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_elasticsearch", 2023-09-12T16:21:17.312 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.312 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.313 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.313 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-12T16:21:17.313 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.313 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.313 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.313 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "elasticsearch container image", 2023-09-12T16:21:17.314 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.314 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.314 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.314 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.314 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_grafana": { 2023-09-12T16:21:17.315 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_grafana", 2023-09-12T16:21:17.315 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.315 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.315 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.315 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-12T16:21:17.316 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.316 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.316 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.316 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:17.316 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.316 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.317 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.317 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.317 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_haproxy": { 2023-09-12T16:21:17.317 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_haproxy", 2023-09-12T16:21:17.317 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.318 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.318 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.318 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-12T16:21:17.318 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.318 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.318 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.319 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "HAproxy container image", 2023-09-12T16:21:17.319 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.319 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.319 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.319 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.320 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_jaeger_agent": { 2023-09-12T16:21:17.320 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_jaeger_agent", 2023-09-12T16:21:17.320 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.320 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.320 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.321 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-12T16:21:17.321 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.321 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.321 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.321 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Jaeger agent container image", 2023-09-12T16:21:17.321 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.322 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.322 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.322 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.322 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_jaeger_collector": { 2023-09-12T16:21:17.322 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_jaeger_collector", 2023-09-12T16:21:17.323 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.323 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.323 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.323 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-12T16:21:17.323 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.324 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.324 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.324 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Jaeger collector container image", 2023-09-12T16:21:17.324 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.324 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.324 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.325 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.325 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_jaeger_query": { 2023-09-12T16:21:17.325 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_jaeger_query", 2023-09-12T16:21:17.325 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.325 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.326 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.326 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-12T16:21:17.326 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.326 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.326 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.326 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Jaeger query container image", 2023-09-12T16:21:17.327 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.327 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.327 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.327 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.327 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_keepalived": { 2023-09-12T16:21:17.328 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_keepalived", 2023-09-12T16:21:17.328 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.328 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.328 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.328 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-12T16:21:17.329 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.329 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.329 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.329 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Keepalived container image", 2023-09-12T16:21:17.329 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.329 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.330 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.330 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.330 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_loki": { 2023-09-12T16:21:17.330 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_loki", 2023-09-12T16:21:17.330 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.331 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.331 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.331 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-12T16:21:17.331 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.331 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.331 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.332 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Loki container image", 2023-09-12T16:21:17.332 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.332 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.332 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.332 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.333 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_node_exporter": { 2023-09-12T16:21:17.333 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_node_exporter", 2023-09-12T16:21:17.333 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.333 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.333 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.334 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-12T16:21:17.334 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.334 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.334 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.334 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:17.335 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.335 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.335 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.335 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.335 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_nvmeof": { 2023-09-12T16:21:17.336 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_nvmeof", 2023-09-12T16:21:17.336 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.336 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.336 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.336 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-12T16:21:17.336 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.337 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.337 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.337 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Nvme-of container image", 2023-09-12T16:21:17.337 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.337 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.338 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.338 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.338 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_prometheus": { 2023-09-12T16:21:17.338 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_prometheus", 2023-09-12T16:21:17.338 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.339 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.339 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.339 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-12T16:21:17.339 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.339 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.340 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.340 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:17.340 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.340 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.340 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.340 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.341 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_promtail": { 2023-09-12T16:21:17.341 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_promtail", 2023-09-12T16:21:17.341 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.341 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.341 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.342 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-12T16:21:17.342 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.342 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.342 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.342 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Promtail container image", 2023-09-12T16:21:17.343 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.343 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.343 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.343 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.343 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_snmp_gateway": { 2023-09-12T16:21:17.343 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_snmp_gateway", 2023-09-12T16:21:17.344 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.344 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.344 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.344 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-12T16:21:17.344 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.345 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.345 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.345 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SNMP Gateway container image", 2023-09-12T16:21:17.345 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.345 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.346 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.346 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.346 INFO:teuthology.orchestra.run.smithi130.stdout: "container_init": { 2023-09-12T16:21:17.346 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_init", 2023-09-12T16:21:17.346 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.347 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.347 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.347 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.347 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.347 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.348 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.348 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Run podman/docker with `--init`", 2023-09-12T16:21:17.348 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.348 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.348 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.348 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.349 INFO:teuthology.orchestra.run.smithi130.stdout: "daemon_cache_timeout": { 2023-09-12T16:21:17.349 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "daemon_cache_timeout", 2023-09-12T16:21:17.349 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.349 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.349 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.350 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:17.350 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.350 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.350 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.350 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-12T16:21:17.351 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.351 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.351 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.351 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.351 INFO:teuthology.orchestra.run.smithi130.stdout: "default_cephadm_command_timeout": { 2023-09-12T16:21:17.351 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "default_cephadm_command_timeout", 2023-09-12T16:21:17.352 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.352 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.352 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.352 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "900", 2023-09-12T16:21:17.352 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.353 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.353 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.353 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-12T16:21:17.353 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.353 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.353 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.354 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.354 INFO:teuthology.orchestra.run.smithi130.stdout: "default_registry": { 2023-09-12T16:21:17.354 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "default_registry", 2023-09-12T16:21:17.354 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.354 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.355 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.355 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io", 2023-09-12T16:21:17.355 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.355 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.355 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.355 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-12T16:21:17.356 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.356 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.356 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.356 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.356 INFO:teuthology.orchestra.run.smithi130.stdout: "device_cache_timeout": { 2023-09-12T16:21:17.357 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "device_cache_timeout", 2023-09-12T16:21:17.357 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.357 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.357 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.357 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "1800", 2023-09-12T16:21:17.358 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.358 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.358 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.358 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "seconds to cache device inventory", 2023-09-12T16:21:17.358 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.358 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.359 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.359 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.359 INFO:teuthology.orchestra.run.smithi130.stdout: "device_enhanced_scan": { 2023-09-12T16:21:17.359 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "device_enhanced_scan", 2023-09-12T16:21:17.359 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.360 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.360 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.360 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.360 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.360 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.360 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.361 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-12T16:21:17.361 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.361 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.361 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.361 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.362 INFO:teuthology.orchestra.run.smithi130.stdout: "facts_cache_timeout": { 2023-09-12T16:21:17.362 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "facts_cache_timeout", 2023-09-12T16:21:17.362 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.362 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.362 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.363 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:17.363 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.363 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.363 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.363 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "seconds to cache host facts data", 2023-09-12T16:21:17.364 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.364 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.364 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.364 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.364 INFO:teuthology.orchestra.run.smithi130.stdout: "host_check_interval": { 2023-09-12T16:21:17.364 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "host_check_interval", 2023-09-12T16:21:17.365 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.365 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.365 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.365 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:17.365 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to perform a host check", 2023-09-12T16:21:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.366 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: "inventory_list_all": { 2023-09-12T16:21:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "inventory_list_all", 2023-09-12T16:21:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.367 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.368 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-12T16:21:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.369 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.371 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.372 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.373 INFO:teuthology.orchestra.run.smithi130.stdout: "log_refresh_metadata": { 2023-09-12T16:21:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_refresh_metadata", 2023-09-12T16:21:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.374 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-12T16:21:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.375 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.376 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.377 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.378 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-12T16:21:17.378 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.378 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.378 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.378 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.379 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.379 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.379 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.379 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.379 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.380 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.380 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.380 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.380 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.380 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.380 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.381 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.381 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.381 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.381 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.381 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.382 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.382 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.382 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.382 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.382 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.382 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.383 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.383 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.383 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.383 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.383 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.384 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.384 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.384 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.384 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.384 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.385 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.385 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.385 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.385 INFO:teuthology.orchestra.run.smithi130.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-12T16:21:17.385 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-12T16:21:17.385 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.386 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.386 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.386 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.386 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.386 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.387 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.387 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-12T16:21:17.387 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.387 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.387 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.387 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.388 INFO:teuthology.orchestra.run.smithi130.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-12T16:21:17.388 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-12T16:21:17.388 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.388 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.388 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.389 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "*", 2023-09-12T16:21:17.389 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.389 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.389 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.389 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-12T16:21:17.389 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.390 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.390 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.390 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.390 INFO:teuthology.orchestra.run.smithi130.stdout: "max_count_per_host": { 2023-09-12T16:21:17.390 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_count_per_host", 2023-09-12T16:21:17.391 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.391 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.391 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.391 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:17.391 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.392 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.392 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.392 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "max number of daemons per service per host", 2023-09-12T16:21:17.392 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.392 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.392 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.393 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.393 INFO:teuthology.orchestra.run.smithi130.stdout: "max_osd_draining_count": { 2023-09-12T16:21:17.393 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_osd_draining_count", 2023-09-12T16:21:17.393 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.393 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.394 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.394 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:17.394 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.394 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.394 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.395 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-12T16:21:17.395 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.395 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.395 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.395 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.395 INFO:teuthology.orchestra.run.smithi130.stdout: "migration_current": { 2023-09-12T16:21:17.396 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "migration_current", 2023-09-12T16:21:17.396 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.396 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.396 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.396 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.397 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.397 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.397 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.397 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "internal - do not modify", 2023-09-12T16:21:17.397 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.398 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.398 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.398 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.398 INFO:teuthology.orchestra.run.smithi130.stdout: "mode": { 2023-09-12T16:21:17.398 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mode", 2023-09-12T16:21:17.399 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.399 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.399 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.399 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "root", 2023-09-12T16:21:17.399 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.400 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.400 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.400 INFO:teuthology.orchestra.run.smithi130.stdout: "cephadm-package", 2023-09-12T16:21:17.400 INFO:teuthology.orchestra.run.smithi130.stdout: "root" 2023-09-12T16:21:17.400 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.400 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "mode for remote execution of cephadm", 2023-09-12T16:21:17.401 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.401 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.401 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.401 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.401 INFO:teuthology.orchestra.run.smithi130.stdout: "prometheus_alerts_path": { 2023-09-12T16:21:17.402 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "prometheus_alerts_path", 2023-09-12T16:21:17.402 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.402 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.402 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.402 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-12T16:21:17.403 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.403 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.403 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.403 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-12T16:21:17.403 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.404 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.404 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.404 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.404 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_insecure": { 2023-09-12T16:21:17.404 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_insecure", 2023-09-12T16:21:17.404 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.405 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.405 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.405 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.405 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.405 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.406 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.406 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-12T16:21:17.406 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.406 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.406 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.407 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.407 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_password": { 2023-09-12T16:21:17.407 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_password", 2023-09-12T16:21:17.407 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.407 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.408 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.408 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.408 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.408 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.408 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.408 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-12T16:21:17.409 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.409 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.409 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.409 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.409 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_url": { 2023-09-12T16:21:17.410 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_url", 2023-09-12T16:21:17.410 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.410 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.410 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.411 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.411 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.411 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.411 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.411 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-12T16:21:17.411 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.412 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.412 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.412 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.412 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_username": { 2023-09-12T16:21:17.412 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_username", 2023-09-12T16:21:17.413 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.413 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.413 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.413 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.413 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.413 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.414 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.414 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-12T16:21:17.414 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.414 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.415 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.415 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.415 INFO:teuthology.orchestra.run.smithi130.stdout: "secure_monitoring_stack": { 2023-09-12T16:21:17.415 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "secure_monitoring_stack", 2023-09-12T16:21:17.415 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.415 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.416 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.416 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.416 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.416 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.416 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.417 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-12T16:21:17.417 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.417 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.417 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.417 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.418 INFO:teuthology.orchestra.run.smithi130.stdout: "service_discovery_port": { 2023-09-12T16:21:17.418 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "service_discovery_port", 2023-09-12T16:21:17.418 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.418 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.418 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.419 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8765", 2023-09-12T16:21:17.419 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.419 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.419 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.421 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "cephadm service discovery port", 2023-09-12T16:21:17.421 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.422 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.422 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.422 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.422 INFO:teuthology.orchestra.run.smithi130.stdout: "ssh_config_file": { 2023-09-12T16:21:17.422 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssh_config_file", 2023-09-12T16:21:17.423 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.423 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.423 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.423 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.423 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.424 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.424 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.424 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-12T16:21:17.424 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.424 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.425 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.425 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.425 INFO:teuthology.orchestra.run.smithi130.stdout: "use_agent": { 2023-09-12T16:21:17.425 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "use_agent", 2023-09-12T16:21:17.425 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.425 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.426 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.426 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.426 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.426 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.426 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.427 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-12T16:21:17.427 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.427 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.427 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.427 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.428 INFO:teuthology.orchestra.run.smithi130.stdout: "use_repo_digest": { 2023-09-12T16:21:17.428 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "use_repo_digest", 2023-09-12T16:21:17.428 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.428 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.428 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.428 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.429 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.429 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.429 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.429 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-12T16:21:17.429 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.430 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.430 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.430 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.430 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_on_failed_host_check": { 2023-09-12T16:21:17.430 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_on_failed_host_check", 2023-09-12T16:21:17.431 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.431 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.431 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.431 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.431 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.431 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.432 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.432 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise a health warning if the host check fails", 2023-09-12T16:21:17.432 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.432 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.432 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.433 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.433 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_on_stray_daemons": { 2023-09-12T16:21:17.433 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_on_stray_daemons", 2023-09-12T16:21:17.433 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.433 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.434 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.434 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.434 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.434 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.434 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.434 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-12T16:21:17.435 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.435 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.435 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.435 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.435 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_on_stray_hosts": { 2023-09-12T16:21:17.436 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_on_stray_hosts", 2023-09-12T16:21:17.436 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.436 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.436 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.436 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.436 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.437 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.437 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.437 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-12T16:21:17.437 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.437 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.438 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.438 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.438 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.438 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.438 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.439 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crash", 2023-09-12T16:21:17.439 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.439 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.439 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.439 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.439 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.440 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.440 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.440 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.440 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.440 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.441 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.441 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.441 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.441 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.441 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.442 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.442 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.442 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.442 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.442 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.442 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.443 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.443 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.443 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.443 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.443 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.444 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.444 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.444 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.444 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.444 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.444 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.445 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.445 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.445 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.445 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.445 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.446 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.446 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.446 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.446 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.446 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.446 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.447 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.447 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.447 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.447 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.447 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.448 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.448 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.448 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.448 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.448 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.449 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.449 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.449 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.449 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.449 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.449 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.450 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.450 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.450 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.450 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.450 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.451 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.451 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.451 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.451 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.451 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.452 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.452 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.452 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.452 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.452 INFO:teuthology.orchestra.run.smithi130.stdout: "retain_interval": { 2023-09-12T16:21:17.452 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "retain_interval", 2023-09-12T16:21:17.453 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.453 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.453 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.453 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "31536000", 2023-09-12T16:21:17.453 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.454 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.454 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.454 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-12T16:21:17.454 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.454 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.455 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.455 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.455 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_recent_interval": { 2023-09-12T16:21:17.455 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_recent_interval", 2023-09-12T16:21:17.455 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.456 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.456 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.456 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "1209600", 2023-09-12T16:21:17.456 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.456 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.456 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.457 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-12T16:21:17.457 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.457 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.457 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.457 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.458 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.458 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.458 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.458 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "dashboard", 2023-09-12T16:21:17.458 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.459 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.459 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.459 INFO:teuthology.orchestra.run.smithi130.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-12T16:21:17.459 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-12T16:21:17.459 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.460 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.460 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.460 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:17.460 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.461 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.461 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.461 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.461 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.461 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.462 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.462 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.462 INFO:teuthology.orchestra.run.smithi130.stdout: "ALERTMANAGER_API_HOST": { 2023-09-12T16:21:17.462 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-12T16:21:17.463 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.463 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.463 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.463 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.464 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.464 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.464 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.464 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.464 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.465 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.465 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.465 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.465 INFO:teuthology.orchestra.run.smithi130.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-12T16:21:17.466 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-12T16:21:17.466 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.466 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.466 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.467 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.467 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.467 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.467 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.468 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.468 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.468 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.468 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.469 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.469 INFO:teuthology.orchestra.run.smithi130.stdout: "AUDIT_API_ENABLED": { 2023-09-12T16:21:17.469 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "AUDIT_API_ENABLED", 2023-09-12T16:21:17.469 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.470 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.470 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.470 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.470 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.471 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.471 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.471 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.471 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.472 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.472 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.472 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.472 INFO:teuthology.orchestra.run.smithi130.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-12T16:21:17.473 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-12T16:21:17.473 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.473 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.474 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.474 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.474 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.475 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.475 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.475 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.475 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.476 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.476 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.476 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.476 INFO:teuthology.orchestra.run.smithi130.stdout: "ENABLE_BROWSABLE_API": { 2023-09-12T16:21:17.476 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-12T16:21:17.477 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.477 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.477 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.477 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.477 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.478 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.478 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.478 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.478 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.478 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.479 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.479 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.479 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-12T16:21:17.479 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-12T16:21:17.479 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.479 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.480 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.480 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.480 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.480 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.480 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.481 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.481 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.481 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.481 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.481 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.481 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-12T16:21:17.482 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-12T16:21:17.482 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.482 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.482 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.482 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.483 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.483 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.483 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.483 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.483 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.484 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.484 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.484 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.484 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-12T16:21:17.484 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-12T16:21:17.485 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.485 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.485 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.485 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.485 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.486 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.486 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.486 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.486 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.486 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.486 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.487 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.487 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-12T16:21:17.487 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-12T16:21:17.487 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.487 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.488 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.488 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.488 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.488 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.488 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.488 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.489 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.489 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.489 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.489 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.489 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-12T16:21:17.490 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-12T16:21:17.490 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.490 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.490 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.490 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.491 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.491 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.491 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.491 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.491 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.491 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.492 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.492 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.492 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-12T16:21:17.492 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-12T16:21:17.492 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.492 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.493 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.493 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.493 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.493 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.493 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.493 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.494 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.494 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.494 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.494 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.494 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-12T16:21:17.495 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-12T16:21:17.495 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.495 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.495 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.495 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.496 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.496 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.496 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.496 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.496 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.496 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.497 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.497 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.497 INFO:teuthology.orchestra.run.smithi130.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-12T16:21:17.497 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-12T16:21:17.497 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.498 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.498 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.498 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.498 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.498 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.499 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.499 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.499 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.499 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.499 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.500 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.500 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_PASSWORD": { 2023-09-12T16:21:17.500 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-12T16:21:17.500 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.500 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.501 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.501 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "admin", 2023-09-12T16:21:17.501 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.501 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.502 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.502 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.502 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.502 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.503 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.503 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.503 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-12T16:21:17.503 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-12T16:21:17.504 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.504 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.504 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.504 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.505 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.505 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.505 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.505 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.506 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.506 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.506 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.506 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.506 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_URL": { 2023-09-12T16:21:17.507 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_URL", 2023-09-12T16:21:17.507 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.507 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.507 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.507 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.508 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.508 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.508 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.508 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.509 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.509 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.509 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.509 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.510 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_USERNAME": { 2023-09-12T16:21:17.510 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-12T16:21:17.510 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.511 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.511 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.511 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "admin", 2023-09-12T16:21:17.511 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.511 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.512 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.512 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.512 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.512 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.512 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.513 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.513 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-12T16:21:17.513 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-12T16:21:17.513 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.513 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.514 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.514 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.514 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.514 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.514 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.515 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.515 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.515 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.515 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.515 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.516 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-12T16:21:17.516 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-12T16:21:17.516 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.516 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.516 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.516 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.517 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.517 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.517 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.517 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.518 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.518 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.518 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.518 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.518 INFO:teuthology.orchestra.run.smithi130.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-12T16:21:17.519 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-12T16:21:17.519 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.519 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.519 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.519 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.520 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.520 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.520 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.520 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.520 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.521 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.521 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.521 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.521 INFO:teuthology.orchestra.run.smithi130.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-12T16:21:17.521 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-12T16:21:17.521 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.522 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.522 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.522 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.522 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.522 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.523 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.523 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.523 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.523 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.523 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.524 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.524 INFO:teuthology.orchestra.run.smithi130.stdout: "PROMETHEUS_API_HOST": { 2023-09-12T16:21:17.524 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-12T16:21:17.524 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.524 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.525 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.525 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.525 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.525 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.525 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.526 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.526 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.526 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.526 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.526 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.526 INFO:teuthology.orchestra.run.smithi130.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-12T16:21:17.527 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-12T16:21:17.527 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.527 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.527 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.527 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.528 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.528 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.528 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.528 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.528 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.529 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.529 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.529 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.529 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-12T16:21:17.529 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-12T16:21:17.530 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.530 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.530 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.530 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.530 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.531 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.531 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.531 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.531 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.531 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.532 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.532 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.532 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-12T16:21:17.532 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-12T16:21:17.532 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.532 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.533 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.533 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.533 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.533 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.533 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.534 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.534 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.534 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.534 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.534 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.535 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-12T16:21:17.535 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-12T16:21:17.535 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.535 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.535 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.535 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.536 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.536 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.536 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.536 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.536 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.537 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.537 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.537 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.537 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-12T16:21:17.537 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-12T16:21:17.537 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.538 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.538 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.538 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.538 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.538 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.539 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.539 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.539 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.539 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.539 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.540 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.540 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-12T16:21:17.540 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-12T16:21:17.540 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.540 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.541 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.541 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.541 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.541 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.541 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.542 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.542 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.542 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.542 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.542 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.543 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-12T16:21:17.543 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-12T16:21:17.543 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.543 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.543 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.544 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.544 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.544 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.544 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.544 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.545 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.545 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.545 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.545 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.545 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-12T16:21:17.545 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-12T16:21:17.546 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.546 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.546 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.546 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.547 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.547 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.547 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.547 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.547 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.547 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.548 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.548 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.548 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_ENABLED": { 2023-09-12T16:21:17.548 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-12T16:21:17.548 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.549 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.549 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.549 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.549 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.549 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.550 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.550 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.550 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.550 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.550 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.551 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.551 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-12T16:21:17.551 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-12T16:21:17.551 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.551 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.552 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.552 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-12T16:21:17.552 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.552 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.552 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.552 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.553 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.553 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.553 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.553 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.553 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-12T16:21:17.554 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-12T16:21:17.554 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.554 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.554 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.554 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:17.555 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.555 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.555 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.555 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.555 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.556 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.556 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.556 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.556 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-12T16:21:17.556 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-12T16:21:17.556 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.557 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.557 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.557 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8", 2023-09-12T16:21:17.557 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.557 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.558 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.558 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.558 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.558 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.558 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.558 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.559 INFO:teuthology.orchestra.run.smithi130.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-12T16:21:17.559 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-12T16:21:17.559 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.559 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.559 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.559 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "45", 2023-09-12T16:21:17.560 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.560 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.560 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.560 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.560 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.560 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.561 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.561 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.561 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_ACCESS_KEY": { 2023-09-12T16:21:17.561 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-12T16:21:17.561 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.562 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.562 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.562 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.562 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.562 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.563 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.563 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.563 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.563 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.563 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.564 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.564 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-12T16:21:17.564 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-12T16:21:17.564 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.564 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.565 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.565 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "admin", 2023-09-12T16:21:17.565 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.565 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.565 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.566 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.566 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.566 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.566 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.566 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.566 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_SECRET_KEY": { 2023-09-12T16:21:17.567 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-12T16:21:17.567 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.567 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.567 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.567 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.568 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.568 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.568 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.568 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.568 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.568 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.569 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.569 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.569 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_SSL_VERIFY": { 2023-09-12T16:21:17.569 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-12T16:21:17.569 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.570 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.570 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.570 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.570 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.571 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.571 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.571 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.571 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.572 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.572 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.572 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.572 INFO:teuthology.orchestra.run.smithi130.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-12T16:21:17.573 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-12T16:21:17.573 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.573 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.573 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.574 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:17.574 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.574 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.574 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.574 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.575 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.575 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.575 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.575 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.576 INFO:teuthology.orchestra.run.smithi130.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-12T16:21:17.576 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-12T16:21:17.576 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.576 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.577 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.577 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:17.577 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.577 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.578 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.578 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.578 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.578 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.578 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.579 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.579 INFO:teuthology.orchestra.run.smithi130.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-12T16:21:17.579 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-12T16:21:17.579 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.579 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.579 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.580 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:17.580 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.580 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.580 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.581 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.581 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.581 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.581 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.582 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.582 INFO:teuthology.orchestra.run.smithi130.stdout: "cross_origin_url": { 2023-09-12T16:21:17.582 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cross_origin_url", 2023-09-12T16:21:17.582 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.583 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.583 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.583 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.583 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.583 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.584 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.584 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.584 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.584 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.584 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.585 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.585 INFO:teuthology.orchestra.run.smithi130.stdout: "crt_file": { 2023-09-12T16:21:17.585 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crt_file", 2023-09-12T16:21:17.585 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.585 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.585 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.586 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.586 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.586 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.586 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.587 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.587 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.587 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.587 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.587 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.587 INFO:teuthology.orchestra.run.smithi130.stdout: "debug": { 2023-09-12T16:21:17.588 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "debug", 2023-09-12T16:21:17.588 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.588 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.588 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.588 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.589 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.589 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.589 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.589 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Enable/disable debug options", 2023-09-12T16:21:17.589 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.589 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.590 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.590 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.590 INFO:teuthology.orchestra.run.smithi130.stdout: "jwt_token_ttl": { 2023-09-12T16:21:17.590 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "jwt_token_ttl", 2023-09-12T16:21:17.590 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.591 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.591 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.591 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "28800", 2023-09-12T16:21:17.591 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.591 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.592 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.592 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.592 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.592 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.592 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.593 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.593 INFO:teuthology.orchestra.run.smithi130.stdout: "key_file": { 2023-09-12T16:21:17.593 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "key_file", 2023-09-12T16:21:17.593 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.593 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.594 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.594 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.594 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.594 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.594 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.594 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.595 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.595 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.595 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.595 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.595 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.596 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.596 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.596 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.596 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.596 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.597 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.597 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.597 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.597 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.597 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.597 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.598 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.598 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.598 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.598 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.598 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.599 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.599 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.599 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.599 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.599 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.600 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.600 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.600 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.600 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.600 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.600 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.601 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.601 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.601 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.601 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.601 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.601 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.602 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.602 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.602 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.602 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.602 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.603 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.603 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.603 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.603 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.603 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.604 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.604 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.604 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.604 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.604 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.605 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.605 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.605 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.605 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.605 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.605 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.606 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.606 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.606 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.606 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.606 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.607 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.607 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.607 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.607 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.607 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.607 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.608 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.608 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.608 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.608 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.608 INFO:teuthology.orchestra.run.smithi130.stdout: "motd": { 2023-09-12T16:21:17.609 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "motd", 2023-09-12T16:21:17.609 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.609 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.609 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.609 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.610 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.610 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.610 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.610 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "The message of the day", 2023-09-12T16:21:17.610 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.610 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.611 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.611 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.611 INFO:teuthology.orchestra.run.smithi130.stdout: "redirect_resolve_ip_addr": { 2023-09-12T16:21:17.611 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "redirect_resolve_ip_addr", 2023-09-12T16:21:17.611 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.612 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.612 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.612 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.612 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.612 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.613 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.613 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.613 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.613 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.613 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.613 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.614 INFO:teuthology.orchestra.run.smithi130.stdout: "server_addr": { 2023-09-12T16:21:17.614 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_addr", 2023-09-12T16:21:17.614 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.614 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.614 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.615 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "::", 2023-09-12T16:21:17.615 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.615 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.615 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.615 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.616 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.616 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.616 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.616 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.616 INFO:teuthology.orchestra.run.smithi130.stdout: "server_port": { 2023-09-12T16:21:17.617 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_port", 2023-09-12T16:21:17.617 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.617 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.617 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.618 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8080", 2023-09-12T16:21:17.618 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.618 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.618 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.618 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.619 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.619 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.619 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.619 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.619 INFO:teuthology.orchestra.run.smithi130.stdout: "ssl": { 2023-09-12T16:21:17.620 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssl", 2023-09-12T16:21:17.620 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.620 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.620 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.620 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.620 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.621 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.621 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.621 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.621 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.621 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.622 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.622 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.622 INFO:teuthology.orchestra.run.smithi130.stdout: "ssl_server_port": { 2023-09-12T16:21:17.622 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssl_server_port", 2023-09-12T16:21:17.622 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.623 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.623 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.623 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8443", 2023-09-12T16:21:17.624 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.625 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.625 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.625 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.625 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.625 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.626 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.626 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.626 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_behaviour": { 2023-09-12T16:21:17.626 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_behaviour", 2023-09-12T16:21:17.627 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.627 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.627 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.627 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "redirect", 2023-09-12T16:21:17.628 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.628 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.628 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.628 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.628 INFO:teuthology.orchestra.run.smithi130.stdout: "redirect" 2023-09-12T16:21:17.628 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.629 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.629 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.629 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.629 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.629 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.630 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_error_status_code": { 2023-09-12T16:21:17.630 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_error_status_code", 2023-09-12T16:21:17.630 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.630 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.630 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.631 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "500", 2023-09-12T16:21:17.631 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "400", 2023-09-12T16:21:17.631 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "599", 2023-09-12T16:21:17.631 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.632 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.632 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.632 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.632 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.632 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.632 INFO:teuthology.orchestra.run.smithi130.stdout: "url_prefix": { 2023-09-12T16:21:17.633 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "url_prefix", 2023-09-12T16:21:17.633 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.633 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.633 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.634 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.634 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.634 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.634 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.634 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.635 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.635 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.635 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.635 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.635 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.635 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.636 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.636 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "devicehealth", 2023-09-12T16:21:17.636 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.636 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.636 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.637 INFO:teuthology.orchestra.run.smithi130.stdout: "enable_monitoring": { 2023-09-12T16:21:17.637 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enable_monitoring", 2023-09-12T16:21:17.637 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.637 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.637 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.637 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.638 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.638 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.638 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.638 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "monitor device health metrics", 2023-09-12T16:21:17.638 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.639 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.639 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.639 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.639 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.639 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.640 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.640 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.640 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.640 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.640 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.641 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.641 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.641 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.641 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.642 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.642 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.642 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.642 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.643 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.643 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.643 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.643 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.643 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.644 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.644 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.644 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.644 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.644 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.645 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.645 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.645 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.645 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.645 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.645 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.646 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.646 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.646 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.646 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.646 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.647 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.647 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.647 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.647 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.647 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.648 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.648 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.648 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.648 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.648 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.649 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.649 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.649 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.649 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.649 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.649 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.650 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.650 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.650 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.650 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.650 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.651 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.651 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.651 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.651 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.651 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.652 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.652 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.652 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.652 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.652 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.652 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.653 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.653 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.653 INFO:teuthology.orchestra.run.smithi130.stdout: "mark_out_threshold": { 2023-09-12T16:21:17.653 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mark_out_threshold", 2023-09-12T16:21:17.653 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.654 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.654 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.654 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "2419200", 2023-09-12T16:21:17.654 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.654 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.655 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.655 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-12T16:21:17.655 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.655 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.655 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.656 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.656 INFO:teuthology.orchestra.run.smithi130.stdout: "pool_name": { 2023-09-12T16:21:17.656 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pool_name", 2023-09-12T16:21:17.656 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.656 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.657 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.657 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "device_health_metrics", 2023-09-12T16:21:17.657 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.657 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.657 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.657 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-12T16:21:17.658 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.658 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.658 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.658 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.658 INFO:teuthology.orchestra.run.smithi130.stdout: "retention_period": { 2023-09-12T16:21:17.659 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "retention_period", 2023-09-12T16:21:17.659 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.659 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.659 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.659 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "15552000", 2023-09-12T16:21:17.659 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.660 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.660 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.660 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how long to retain device health metrics", 2023-09-12T16:21:17.660 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.660 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.661 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.661 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.661 INFO:teuthology.orchestra.run.smithi130.stdout: "scrape_frequency": { 2023-09-12T16:21:17.661 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "scrape_frequency", 2023-09-12T16:21:17.661 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.662 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.662 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.662 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "86400", 2023-09-12T16:21:17.662 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.662 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.663 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.663 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-12T16:21:17.663 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.663 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.663 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "self_heal": { 2023-09-12T16:21:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "self_heal", 2023-09-12T16:21:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.664 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.665 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-12T16:21:17.666 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.666 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.666 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.666 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.666 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:17.667 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:17.667 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.667 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.667 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.667 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:17.668 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.668 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.668 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.668 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to wake up and check device health", 2023-09-12T16:21:17.668 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.668 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.669 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.669 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.669 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_threshold": { 2023-09-12T16:21:17.669 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_threshold", 2023-09-12T16:21:17.669 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.670 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.670 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.670 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "7257600", 2023-09-12T16:21:17.670 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.670 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.670 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.671 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-12T16:21:17.671 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.671 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.671 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.671 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.672 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.672 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.672 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.672 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "diskprediction_local", 2023-09-12T16:21:17.672 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.673 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.673 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.673 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.673 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.673 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.673 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.674 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.674 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.674 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.674 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.674 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.675 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.675 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.675 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.675 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.675 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.676 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.676 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.676 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.676 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.676 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.677 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.677 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.677 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.677 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.677 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.678 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.678 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.678 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.678 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.678 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.679 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.679 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.679 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.679 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.679 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.680 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.680 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.680 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.680 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.680 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.680 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.681 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.681 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.681 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.681 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.681 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.682 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.682 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.682 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.682 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.682 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.682 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.683 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.683 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.683 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.683 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.683 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.684 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.684 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.684 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.684 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.684 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.684 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.685 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.685 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.685 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.685 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.685 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.686 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.686 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.686 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.686 INFO:teuthology.orchestra.run.smithi130.stdout: "predict_interval": { 2023-09-12T16:21:17.686 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "predict_interval", 2023-09-12T16:21:17.687 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.687 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.687 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.687 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "86400", 2023-09-12T16:21:17.687 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.687 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.688 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.688 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.688 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.688 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.688 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.689 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.689 INFO:teuthology.orchestra.run.smithi130.stdout: "predictor_model": { 2023-09-12T16:21:17.689 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "predictor_model", 2023-09-12T16:21:17.689 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.689 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.690 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.690 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "prophetstor", 2023-09-12T16:21:17.690 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.690 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.690 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.691 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.691 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.691 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.691 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.691 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.691 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:17.692 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:17.692 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.692 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.692 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.692 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:17.693 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.693 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.693 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.693 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.693 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.693 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.694 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.694 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.694 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.694 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.694 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.695 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "influx", 2023-09-12T16:21:17.695 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": false, 2023-09-12T16:21:17.695 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "influxdb python module not found", 2023-09-12T16:21:17.695 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.695 INFO:teuthology.orchestra.run.smithi130.stdout: "batch_size": { 2023-09-12T16:21:17.696 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "batch_size", 2023-09-12T16:21:17.696 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.696 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.696 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.696 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5000", 2023-09-12T16:21:17.697 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.697 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.697 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.697 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-12T16:21:17.697 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.697 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.698 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.698 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.698 INFO:teuthology.orchestra.run.smithi130.stdout: "database": { 2023-09-12T16:21:17.698 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "database", 2023-09-12T16:21:17.698 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.699 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.699 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.699 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "ceph", 2023-09-12T16:21:17.699 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.699 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.699 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.700 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-12T16:21:17.700 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.700 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.700 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.700 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.701 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": { 2023-09-12T16:21:17.701 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "hostname", 2023-09-12T16:21:17.701 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.701 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.701 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.701 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.702 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.702 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.702 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.702 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "InfluxDB server hostname", 2023-09-12T16:21:17.702 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.703 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.703 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.703 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.703 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:17.703 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:17.703 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.704 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.704 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.704 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "30", 2023-09-12T16:21:17.704 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "5", 2023-09-12T16:21:17.704 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.705 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.705 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-12T16:21:17.705 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.705 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.705 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.705 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.706 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.706 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.706 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.706 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.706 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.707 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.707 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.707 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.707 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.707 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.708 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.708 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.708 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.708 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.708 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.708 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.709 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.709 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.709 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.709 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.709 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.710 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.710 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.710 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.710 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.710 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.711 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.711 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.711 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.711 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.711 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.711 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.712 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.712 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.712 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.712 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.712 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.713 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.713 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.713 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.713 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.713 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.713 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.714 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.714 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.714 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.714 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.714 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.715 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.715 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.715 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.715 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.715 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.715 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.716 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.716 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.716 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.716 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.716 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.717 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.717 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.717 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.717 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.717 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.718 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.718 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.718 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.718 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.718 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.718 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.719 INFO:teuthology.orchestra.run.smithi130.stdout: "password": { 2023-09-12T16:21:17.719 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "password", 2023-09-12T16:21:17.719 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.719 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.719 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.720 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.720 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.720 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.720 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.720 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "password of InfluxDB server user", 2023-09-12T16:21:17.720 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.721 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.721 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.721 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.721 INFO:teuthology.orchestra.run.smithi130.stdout: "port": { 2023-09-12T16:21:17.721 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "port", 2023-09-12T16:21:17.722 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.722 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.722 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.722 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8086", 2023-09-12T16:21:17.722 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.723 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.723 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.723 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "InfluxDB server port", 2023-09-12T16:21:17.723 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.723 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.723 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: "ssl": { 2023-09-12T16:21:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssl", 2023-09-12T16:21:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "false", 2023-09-12T16:21:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-12T16:21:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: "threads": { 2023-09-12T16:21:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "threads", 2023-09-12T16:21:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "32", 2023-09-12T16:21:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-12T16:21:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: "username": { 2023-09-12T16:21:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "username", 2023-09-12T16:21:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "username of InfluxDB server user", 2023-09-12T16:21:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: "verify_ssl": { 2023-09-12T16:21:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "verify_ssl", 2023-09-12T16:21:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "true", 2023-09-12T16:21:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-12T16:21:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "insights", 2023-09-12T16:21:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.738 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.738 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.738 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.738 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.738 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.739 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.739 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.739 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.739 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.739 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.739 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.740 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.740 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.740 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.740 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.740 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.740 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.741 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.741 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.741 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.741 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.741 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.742 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.742 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.742 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.742 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.742 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.743 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.743 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.743 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.743 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.743 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.744 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.744 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.744 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.744 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.744 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.744 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.745 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.745 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.745 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.745 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.745 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.745 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.746 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.746 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.746 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.746 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.746 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.747 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.747 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.747 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.747 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.747 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.748 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.748 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.748 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.748 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.748 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.748 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.749 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.749 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "iostat", 2023-09-12T16:21:17.749 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.749 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.749 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.750 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.750 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.750 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.750 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.750 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.751 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.751 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.751 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.751 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.751 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.751 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.752 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.752 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.752 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.752 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.752 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.753 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.753 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.753 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.753 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.753 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.753 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.754 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.754 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.754 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.754 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.754 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.755 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.755 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.755 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.755 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.755 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.756 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.756 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.756 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.756 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.756 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.756 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.757 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.757 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.757 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.757 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.757 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.758 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.758 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.758 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.758 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.759 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.759 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.759 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.759 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.759 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.759 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.760 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.760 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.760 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.760 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.760 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.761 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.761 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.761 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.761 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.761 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.762 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.762 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.762 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.762 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.762 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.762 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.763 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.763 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.763 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.763 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.763 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "k8sevents", 2023-09-12T16:21:17.764 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.764 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.764 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.764 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_event_retention_days": { 2023-09-12T16:21:17.764 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ceph_event_retention_days", 2023-09-12T16:21:17.764 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.765 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.765 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.765 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "7", 2023-09-12T16:21:17.765 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.765 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.766 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.766 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-12T16:21:17.766 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.766 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.766 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.767 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.767 INFO:teuthology.orchestra.run.smithi130.stdout: "config_check_secs": { 2023-09-12T16:21:17.767 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "config_check_secs", 2023-09-12T16:21:17.767 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.767 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.768 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.768 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:17.768 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "10", 2023-09-12T16:21:17.768 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.768 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.768 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-12T16:21:17.769 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.769 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.769 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.769 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.769 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.770 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.770 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.770 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.770 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.770 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.771 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.771 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.771 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.771 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.771 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.772 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.772 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.772 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.772 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.772 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.773 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.773 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.773 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.773 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.773 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.773 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.774 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.774 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.774 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.774 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.774 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.775 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.775 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.775 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.775 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.775 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.775 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.776 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.776 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.776 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.776 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.776 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.776 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.777 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.777 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.777 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.777 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.777 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.778 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.778 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.778 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.778 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.778 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.778 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.779 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.779 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.779 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.779 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.779 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.779 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.780 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.780 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.780 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.780 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.780 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.781 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.781 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.781 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.781 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.781 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.781 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.782 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.782 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.782 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.782 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.782 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.783 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.783 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "localpool", 2023-09-12T16:21:17.783 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.783 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.783 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.784 INFO:teuthology.orchestra.run.smithi130.stdout: "failure_domain": { 2023-09-12T16:21:17.784 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "failure_domain", 2023-09-12T16:21:17.784 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.784 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.784 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.785 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "host", 2023-09-12T16:21:17.785 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.785 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.785 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.785 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "failure domain for any created local pool", 2023-09-12T16:21:17.786 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-12T16:21:17.786 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.786 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.786 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.786 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.786 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.787 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.787 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.787 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.787 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.787 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.788 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.788 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.788 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.788 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.788 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.789 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.789 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.789 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.789 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.789 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.790 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.790 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.790 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.790 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.790 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.791 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.791 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.791 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.791 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.791 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.791 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.792 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.792 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.792 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.792 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.792 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.793 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.793 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.793 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.793 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.793 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.794 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.794 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.794 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.794 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.794 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.794 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.795 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.795 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.795 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.795 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.795 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.796 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.796 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.796 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.796 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.796 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.796 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.797 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.797 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.797 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.797 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.797 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.798 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.798 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.798 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.798 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.798 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.799 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.799 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.799 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.799 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.799 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.799 INFO:teuthology.orchestra.run.smithi130.stdout: "min_size": { 2023-09-12T16:21:17.800 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "min_size", 2023-09-12T16:21:17.800 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.800 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.800 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.800 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.801 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.801 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.801 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.801 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "default min_size for any created local pool", 2023-09-12T16:21:17.801 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-12T16:21:17.801 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.802 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.802 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.802 INFO:teuthology.orchestra.run.smithi130.stdout: "num_rep": { 2023-09-12T16:21:17.802 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "num_rep", 2023-09-12T16:21:17.802 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.803 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.803 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.803 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "3", 2023-09-12T16:21:17.803 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.803 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.804 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.804 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "default replica count for any created local pool", 2023-09-12T16:21:17.804 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.804 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.804 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.804 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.805 INFO:teuthology.orchestra.run.smithi130.stdout: "pg_num": { 2023-09-12T16:21:17.805 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pg_num", 2023-09-12T16:21:17.805 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.805 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.805 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.806 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "128", 2023-09-12T16:21:17.806 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.806 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.806 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.806 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "default pg_num for any created local pool", 2023-09-12T16:21:17.806 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.807 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.807 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.807 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.807 INFO:teuthology.orchestra.run.smithi130.stdout: "prefix": { 2023-09-12T16:21:17.807 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "prefix", 2023-09-12T16:21:17.808 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.808 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.808 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.808 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.808 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.808 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.809 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.809 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "name prefix for any created local pool", 2023-09-12T16:21:17.809 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.809 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.809 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.809 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.810 INFO:teuthology.orchestra.run.smithi130.stdout: "subtree": { 2023-09-12T16:21:17.810 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "subtree", 2023-09-12T16:21:17.810 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.810 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.810 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.810 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "rack", 2023-09-12T16:21:17.811 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.811 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.811 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.811 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-12T16:21:17.811 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-12T16:21:17.812 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.812 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.812 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.812 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.812 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.812 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.813 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mds_autoscaler", 2023-09-12T16:21:17.813 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.813 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.813 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.813 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.814 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.814 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.814 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.814 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.814 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.814 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.815 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.815 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.815 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.815 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.815 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.815 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.816 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.816 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.816 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.816 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.816 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.816 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.817 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.817 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.817 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.817 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.817 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.818 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.818 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.818 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.818 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.818 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.819 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.819 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.819 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.819 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.819 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.819 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.820 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.820 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.820 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.820 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.820 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.821 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.821 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.821 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.821 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.821 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.822 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.822 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.822 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.822 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.822 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.823 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.823 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.823 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.823 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.823 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.823 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.824 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.824 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.824 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.824 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.824 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.825 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.825 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.825 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.825 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.825 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.825 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.826 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.826 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.826 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.826 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.826 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.827 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.827 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mirroring", 2023-09-12T16:21:17.827 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.827 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.827 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.828 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.828 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.828 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.828 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.828 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.828 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.829 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.829 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.829 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.829 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.829 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.830 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.830 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.830 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.830 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.830 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.831 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.831 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.831 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.832 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.832 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.833 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.833 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.833 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.833 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.833 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.834 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.834 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.834 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.834 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.834 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.835 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.835 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.835 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.835 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.835 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.836 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.836 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.836 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.836 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.836 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.836 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.837 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.837 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.837 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.837 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.837 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.838 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.838 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.838 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.838 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.839 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.839 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.839 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.839 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.839 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.840 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.840 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.840 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.840 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.840 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.841 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.841 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.841 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.841 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.841 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.841 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.842 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.842 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.842 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.842 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.842 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.843 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.843 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "nfs", 2023-09-12T16:21:17.843 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.843 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.843 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.844 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.844 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.844 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.844 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.844 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.845 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.845 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.845 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.845 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.845 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.845 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.846 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.846 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.846 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.846 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.846 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.847 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.847 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.847 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.847 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.847 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.847 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.848 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.848 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.848 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.848 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.848 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.849 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.849 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.849 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.849 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.849 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.849 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.850 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.850 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.850 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.850 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.850 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.851 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.851 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.851 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.851 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.851 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.852 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.852 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.852 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.852 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.852 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.853 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.853 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.853 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.853 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.853 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.853 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.854 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.854 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.854 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.854 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.855 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.855 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.855 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.855 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.855 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.856 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.856 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.856 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.856 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.856 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.857 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.857 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.857 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.857 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.857 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.857 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "orchestrator", 2023-09-12T16:21:17.858 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.858 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.858 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.858 INFO:teuthology.orchestra.run.smithi130.stdout: "fail_fs": { 2023-09-12T16:21:17.858 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "fail_fs", 2023-09-12T16:21:17.859 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.859 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.859 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.859 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.859 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.860 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.860 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.860 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-12T16:21:17.860 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.860 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.860 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.861 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.861 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.861 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.861 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.861 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.862 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.862 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.862 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.862 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.862 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.863 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.863 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.863 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.863 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.863 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.863 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.864 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.864 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.864 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.864 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.864 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.865 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.865 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.865 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.865 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.865 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.866 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.866 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.866 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.866 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.866 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.867 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.867 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.867 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.867 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.867 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.868 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.868 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.868 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.868 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.868 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.869 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.869 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.869 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.869 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.869 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.869 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.870 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.870 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.870 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.870 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.870 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.873 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.873 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.873 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.873 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.873 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.874 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.874 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.874 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.874 INFO:teuthology.orchestra.run.smithi130.stdout: "orchestrator": { 2023-09-12T16:21:17.874 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "orchestrator", 2023-09-12T16:21:17.874 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.875 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.875 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.875 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.875 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.875 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.876 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.876 INFO:teuthology.orchestra.run.smithi130.stdout: "cephadm", 2023-09-12T16:21:17.876 INFO:teuthology.orchestra.run.smithi130.stdout: "rook", 2023-09-12T16:21:17.876 INFO:teuthology.orchestra.run.smithi130.stdout: "test_orchestrator" 2023-09-12T16:21:17.877 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.877 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Orchestrator backend", 2023-09-12T16:21:17.877 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.877 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.877 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.877 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.878 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.878 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.878 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.878 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "osd_perf_query", 2023-09-12T16:21:17.878 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.879 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.879 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.879 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.879 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.879 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.879 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.880 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.880 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.880 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.880 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.880 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.881 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.881 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.881 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.881 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.881 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.881 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.882 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.882 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.882 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.882 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.882 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.883 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.883 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.883 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.883 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.883 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.884 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.884 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.884 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.884 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.884 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.884 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.885 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.885 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.885 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.885 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.885 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.885 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.886 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.886 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.886 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.886 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.886 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.887 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.887 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.887 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.887 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.887 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.888 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.888 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.888 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.888 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.888 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.888 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.889 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.889 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.889 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.889 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.889 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.890 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.890 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.890 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.890 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.890 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.891 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.891 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.891 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.891 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.891 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.892 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.892 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.892 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.892 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.892 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.893 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "osd_support", 2023-09-12T16:21:17.893 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.893 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.893 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.893 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.893 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.894 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.894 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.894 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.894 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.894 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.895 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.895 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.895 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.895 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.895 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.895 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.896 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.896 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.896 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.896 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.896 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.897 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.897 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.897 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.897 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.897 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.897 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.898 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.898 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.898 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.898 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.898 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.899 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.899 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.899 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.899 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.899 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.899 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.900 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.900 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.900 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.900 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.900 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.901 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.901 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.901 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.901 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.901 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.901 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.902 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.902 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.902 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.902 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.902 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.903 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.903 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.903 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.903 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.903 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.904 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.904 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.904 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.904 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.904 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.904 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.905 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.905 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.905 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.905 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.905 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.906 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.906 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.906 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.906 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.906 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.907 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.907 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pg_autoscaler", 2023-09-12T16:21:17.907 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.907 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.907 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.907 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.908 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.908 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.908 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.908 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.908 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.908 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.909 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.909 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.909 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.909 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.909 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.910 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.910 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.910 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.910 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.910 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.911 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.911 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.911 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.911 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.911 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.911 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.912 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.912 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.912 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.912 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.912 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.913 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.913 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.913 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.913 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.913 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.913 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.914 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.914 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.914 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.914 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.914 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.915 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.915 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.915 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.915 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.915 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.916 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.916 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.916 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.916 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.916 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.916 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.917 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.917 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.917 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.917 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.917 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.918 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.918 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.918 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.918 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.918 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.919 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.919 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.919 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.919 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.919 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.919 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.920 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.920 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.920 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.920 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.920 INFO:teuthology.orchestra.run.smithi130.stdout: "noautoscale": { 2023-09-12T16:21:17.921 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "noautoscale", 2023-09-12T16:21:17.921 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.921 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.921 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.921 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.921 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.922 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.922 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.922 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "global autoscale flag", 2023-09-12T16:21:17.922 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-12T16:21:17.922 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.923 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.923 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.923 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:17.923 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:17.923 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.923 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.924 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.924 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:17.924 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.924 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.924 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.924 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.925 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.925 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.925 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.925 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.925 INFO:teuthology.orchestra.run.smithi130.stdout: "threshold": { 2023-09-12T16:21:17.925 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "threshold", 2023-09-12T16:21:17.926 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:17.926 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.926 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.926 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "3.0", 2023-09-12T16:21:17.926 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1.0", 2023-09-12T16:21:17.927 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.927 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.927 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "scaling threshold", 2023-09-12T16:21:17.927 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-12T16:21:17.927 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.928 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.928 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.928 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.928 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.928 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.928 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "progress", 2023-09-12T16:21:17.929 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.929 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.929 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.929 INFO:teuthology.orchestra.run.smithi130.stdout: "allow_pg_recovery_event": { 2023-09-12T16:21:17.929 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "allow_pg_recovery_event", 2023-09-12T16:21:17.930 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.930 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.930 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.930 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.930 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.931 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.931 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.931 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-12T16:21:17.931 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.931 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.931 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.932 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.932 INFO:teuthology.orchestra.run.smithi130.stdout: "enabled": { 2023-09-12T16:21:17.932 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enabled", 2023-09-12T16:21:17.932 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.932 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.933 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.933 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.933 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.933 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.933 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.933 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.934 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.934 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.934 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.934 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.934 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.935 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.935 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.935 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.935 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.935 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.935 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.936 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.936 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.936 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.936 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.936 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.937 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.937 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.937 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.937 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.937 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.937 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.938 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.938 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.938 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.938 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.938 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.939 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.939 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.939 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.939 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.939 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.939 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.940 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.940 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.940 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.940 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.940 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.941 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.941 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.941 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.941 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.941 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.941 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.942 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.942 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.942 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.942 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.942 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.943 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.943 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.943 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.943 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.943 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.944 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.944 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.944 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.944 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.944 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.944 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.945 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.945 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.945 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.945 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.945 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.946 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.946 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.946 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.946 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.946 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.947 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.947 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.947 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.947 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.947 INFO:teuthology.orchestra.run.smithi130.stdout: "max_completed_events": { 2023-09-12T16:21:17.948 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_completed_events", 2023-09-12T16:21:17.948 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.948 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.948 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.948 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "50", 2023-09-12T16:21:17.949 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.949 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.949 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.949 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "number of past completed events to remember", 2023-09-12T16:21:17.949 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.949 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.950 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.950 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.950 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:17.950 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:17.950 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:17.951 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.951 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.951 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:17.951 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.951 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.952 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.952 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how long the module is going to sleep", 2023-09-12T16:21:17.952 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.952 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.952 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.952 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.953 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.953 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.953 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.953 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "prometheus", 2023-09-12T16:21:17.954 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.954 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.954 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.954 INFO:teuthology.orchestra.run.smithi130.stdout: "cache": { 2023-09-12T16:21:17.954 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cache", 2023-09-12T16:21:17.955 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.955 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.955 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.955 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.955 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.956 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.956 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.956 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.956 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.956 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.956 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.957 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.957 INFO:teuthology.orchestra.run.smithi130.stdout: "exclude_perf_counters": { 2023-09-12T16:21:17.957 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "exclude_perf_counters", 2023-09-12T16:21:17.957 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.957 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.958 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.958 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:17.958 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.958 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.958 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.959 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-12T16:21:17.959 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-12T16:21:17.959 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.959 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.959 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.960 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.960 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.960 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.960 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.960 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.961 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.961 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.961 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.961 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.961 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.961 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.962 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.962 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.962 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.962 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.963 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.963 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.963 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.963 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.963 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.964 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.964 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:17.964 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:17.964 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.964 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.964 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.965 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.965 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.965 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.965 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.965 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.966 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.966 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.966 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.966 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.966 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:17.966 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:17.967 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.967 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.967 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.967 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:17.967 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.968 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.968 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.968 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.968 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.968 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.969 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.969 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.969 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.969 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.969 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.970 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.970 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.970 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.970 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.970 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:17.971 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:17.971 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:17.971 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.971 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.971 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:17.972 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.972 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.972 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.972 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.972 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.972 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.973 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.973 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.973 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_stats_pools": { 2023-09-12T16:21:17.973 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_stats_pools", 2023-09-12T16:21:17.973 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.974 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.974 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.974 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.974 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.974 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.975 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.975 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.975 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.975 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.975 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.976 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.976 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-12T16:21:17.976 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-12T16:21:17.976 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.976 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.976 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.977 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "300", 2023-09-12T16:21:17.977 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.977 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.977 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.977 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.978 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.978 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.978 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.978 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.978 INFO:teuthology.orchestra.run.smithi130.stdout: "scrape_interval": { 2023-09-12T16:21:17.978 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "scrape_interval", 2023-09-12T16:21:17.979 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:17.979 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.979 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.979 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "15.0", 2023-09-12T16:21:17.979 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.980 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.980 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.980 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.980 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.980 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.981 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.981 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.981 INFO:teuthology.orchestra.run.smithi130.stdout: "server_addr": { 2023-09-12T16:21:17.981 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_addr", 2023-09-12T16:21:17.981 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.981 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.982 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.982 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "::", 2023-09-12T16:21:17.982 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.982 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.982 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.983 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-12T16:21:17.983 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.983 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.983 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.983 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.984 INFO:teuthology.orchestra.run.smithi130.stdout: "server_port": { 2023-09-12T16:21:17.984 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_port", 2023-09-12T16:21:17.984 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.984 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.984 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.985 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "9283", 2023-09-12T16:21:17.985 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.985 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.985 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.985 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-12T16:21:17.986 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.986 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.986 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.986 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.986 INFO:teuthology.orchestra.run.smithi130.stdout: "stale_cache_strategy": { 2023-09-12T16:21:17.987 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "stale_cache_strategy", 2023-09-12T16:21:17.987 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.987 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.987 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:17.987 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "log", 2023-09-12T16:21:17.987 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.988 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.988 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.988 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.988 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.988 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.988 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.989 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.989 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_behaviour": { 2023-09-12T16:21:17.989 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_behaviour", 2023-09-12T16:21:17.989 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.989 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.989 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.990 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "default", 2023-09-12T16:21:17.990 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.990 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.990 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.990 INFO:teuthology.orchestra.run.smithi130.stdout: "default", 2023-09-12T16:21:17.990 INFO:teuthology.orchestra.run.smithi130.stdout: "error" 2023-09-12T16:21:17.991 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.991 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.991 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.991 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.991 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.991 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.992 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_error_status_code": { 2023-09-12T16:21:17.992 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_error_status_code", 2023-09-12T16:21:17.992 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:17.992 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.992 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.993 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "500", 2023-09-12T16:21:17.993 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "400", 2023-09-12T16:21:17.993 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "599", 2023-09-12T16:21:17.993 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:17.993 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.993 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.994 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:17.994 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:17.994 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.994 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:17.994 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:17.995 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:17.995 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_support", 2023-09-12T16:21:17.995 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:17.995 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:17.995 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:17.996 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:17.996 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:17.996 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:17.996 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:17.997 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:17.997 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:17.997 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:17.997 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:17.997 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:17.997 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:17.998 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:17.998 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:17.998 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:17.998 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:17.999 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:17.999 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:17.999 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:17.999 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:17.999 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.000 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.000 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.000 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.000 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.000 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.000 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.001 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.001 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.001 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.001 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.001 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.002 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.002 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.002 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.002 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.002 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.003 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.003 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.003 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.003 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.003 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.004 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.004 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.004 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.004 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.004 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.005 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.005 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.005 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.005 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.005 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.006 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.006 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.006 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.006 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.006 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.006 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.007 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.007 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.007 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.007 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.007 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.008 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.008 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.008 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.008 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.008 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.009 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.009 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.009 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.009 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.009 INFO:teuthology.orchestra.run.smithi130.stdout: "max_concurrent_snap_create": { 2023-09-12T16:21:18.009 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_concurrent_snap_create", 2023-09-12T16:21:18.010 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.010 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.010 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.010 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:18.010 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.011 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.011 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.011 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.011 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.011 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.012 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.012 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.012 INFO:teuthology.orchestra.run.smithi130.stdout: "mirror_snapshot_schedule": { 2023-09-12T16:21:18.012 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mirror_snapshot_schedule", 2023-09-12T16:21:18.012 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.013 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.013 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.013 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.013 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.013 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.013 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.014 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.014 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.014 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.014 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.014 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.015 INFO:teuthology.orchestra.run.smithi130.stdout: "trash_purge_schedule": { 2023-09-12T16:21:18.015 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "trash_purge_schedule", 2023-09-12T16:21:18.015 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.015 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.015 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.015 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.016 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.016 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.016 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.016 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.016 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.017 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.017 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.017 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.017 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.017 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.017 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.018 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "restful", 2023-09-12T16:21:18.018 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.018 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.018 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.019 INFO:teuthology.orchestra.run.smithi130.stdout: "enable_auth": { 2023-09-12T16:21:18.019 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enable_auth", 2023-09-12T16:21:18.019 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.019 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.019 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.020 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.020 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.020 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.020 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.020 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.020 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.021 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.021 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.021 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.021 INFO:teuthology.orchestra.run.smithi130.stdout: "key_file": { 2023-09-12T16:21:18.021 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "key_file", 2023-09-12T16:21:18.022 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.022 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.022 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.022 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.022 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.022 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.023 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.023 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.023 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.023 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.023 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.024 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.024 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.024 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.024 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.024 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.025 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.025 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.025 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.025 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.025 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.025 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.026 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.026 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.026 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.026 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.026 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.027 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.027 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.027 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.027 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.027 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.028 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.028 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.028 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.028 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.028 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.029 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.029 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.029 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.029 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.029 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.029 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.030 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.030 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.030 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.030 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.030 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.031 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.031 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.031 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.031 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.031 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.032 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.032 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.032 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.032 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.032 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.033 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.033 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.033 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.033 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.033 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.034 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.034 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.034 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.034 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.034 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.035 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.035 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.035 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.035 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.035 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.035 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.036 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.036 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.036 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.036 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.036 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.037 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.037 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.037 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.037 INFO:teuthology.orchestra.run.smithi130.stdout: "server_addr": { 2023-09-12T16:21:18.037 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_addr", 2023-09-12T16:21:18.037 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.038 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.038 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.038 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.038 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.038 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.039 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.039 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.039 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.039 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.039 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.040 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.040 INFO:teuthology.orchestra.run.smithi130.stdout: "server_port": { 2023-09-12T16:21:18.040 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_port", 2023-09-12T16:21:18.040 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.040 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.040 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.041 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.041 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.041 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.041 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.041 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.042 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.042 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.042 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.042 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.042 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.043 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.043 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.043 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rgw", 2023-09-12T16:21:18.043 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.043 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.043 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.044 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.044 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.044 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.044 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.045 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.045 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.045 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.045 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.045 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.045 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.046 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.046 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.046 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.046 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.046 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.047 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.047 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.047 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.047 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.047 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.048 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.048 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.048 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.048 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.048 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.049 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.049 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.049 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.049 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.049 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.049 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.050 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.050 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.050 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.050 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.050 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.051 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.051 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.051 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.051 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.051 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.051 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.052 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.052 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.052 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.052 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.052 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.053 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.053 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.053 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.053 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.053 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.054 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.054 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.054 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.054 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.054 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.054 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.055 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.055 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.055 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.055 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.056 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.056 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.056 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.056 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.056 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.056 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.057 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.057 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.057 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.057 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.057 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rook", 2023-09-12T16:21:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "drive_group_interval": { 2023-09-12T16:21:18.058 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "drive_group_interval", 2023-09-12T16:21:18.059 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:18.059 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.059 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.059 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "300.0", 2023-09-12T16:21:18.059 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.060 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.060 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.060 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-12T16:21:18.060 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.060 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.061 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.061 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.061 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.061 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.061 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.061 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.062 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.062 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.062 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.062 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.062 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.062 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.063 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.063 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.063 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.063 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.063 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.064 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.064 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.064 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.064 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.064 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.064 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.065 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.065 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.065 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.065 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.065 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.066 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.066 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.066 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.066 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.066 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.067 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.067 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.067 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.067 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.067 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.068 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.068 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.068 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.068 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.068 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.069 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.069 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.069 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.069 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.069 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.069 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.070 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.070 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.070 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.070 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.070 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.071 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.071 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.071 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.071 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.071 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.072 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.072 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.072 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.072 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.072 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.073 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.073 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.073 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.073 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.073 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.074 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.074 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.074 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.074 INFO:teuthology.orchestra.run.smithi130.stdout: "storage_class": { 2023-09-12T16:21:18.074 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "storage_class", 2023-09-12T16:21:18.075 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.075 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.075 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.075 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "local", 2023-09-12T16:21:18.075 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.075 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.076 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.076 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-12T16:21:18.076 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.076 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.076 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.077 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.077 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.077 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.077 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.077 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "selftest", 2023-09-12T16:21:18.077 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.078 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.078 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.078 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.078 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.078 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.079 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.079 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.079 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.079 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.079 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.080 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.080 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.080 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.080 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.080 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.080 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.081 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.081 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.081 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.081 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.081 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.082 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.082 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.082 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.082 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.082 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.083 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.083 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.083 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.083 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.083 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.083 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.084 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.084 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.084 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.084 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.084 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.085 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.085 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.085 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.085 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.085 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.086 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.086 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.086 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.086 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.086 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.086 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.087 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.087 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.087 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.087 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.087 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.088 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.088 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.088 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.088 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.088 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.088 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.089 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.089 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.089 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.089 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.089 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.090 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.090 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.090 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.090 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.090 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.090 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.091 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.091 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.091 INFO:teuthology.orchestra.run.smithi130.stdout: "roption1": { 2023-09-12T16:21:18.091 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "roption1", 2023-09-12T16:21:18.091 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.092 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.092 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.092 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.092 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.092 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.093 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.093 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.093 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.093 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.093 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.094 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.094 INFO:teuthology.orchestra.run.smithi130.stdout: "roption2": { 2023-09-12T16:21:18.094 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "roption2", 2023-09-12T16:21:18.094 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.094 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.094 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.095 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "xyz", 2023-09-12T16:21:18.095 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.095 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.095 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.095 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.096 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.096 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.096 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.096 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.096 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption1": { 2023-09-12T16:21:18.096 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption1", 2023-09-12T16:21:18.097 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.097 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.097 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.097 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.097 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.098 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.098 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.098 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.098 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.098 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.098 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.099 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.099 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption2": { 2023-09-12T16:21:18.099 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption2", 2023-09-12T16:21:18.099 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.099 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.100 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.100 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.100 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.100 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.100 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.101 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.101 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.101 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.101 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.101 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.101 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption3": { 2023-09-12T16:21:18.102 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption3", 2023-09-12T16:21:18.102 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:18.102 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.102 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.102 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.103 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.103 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.103 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.103 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.103 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.104 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.104 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.104 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.104 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption4": { 2023-09-12T16:21:18.104 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption4", 2023-09-12T16:21:18.105 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.105 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.105 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.105 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.105 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.105 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.106 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.106 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.106 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.106 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.106 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.107 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.107 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption5": { 2023-09-12T16:21:18.107 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption5", 2023-09-12T16:21:18.107 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.107 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.108 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.108 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.108 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.108 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.108 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.108 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.109 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.109 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.109 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.109 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.109 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption6": { 2023-09-12T16:21:18.110 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption6", 2023-09-12T16:21:18.110 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.110 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.110 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.110 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.111 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.111 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.111 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.111 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.111 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.111 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.112 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.112 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.112 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption7": { 2023-09-12T16:21:18.112 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption7", 2023-09-12T16:21:18.112 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.113 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.113 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.113 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.113 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:18.113 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "42", 2023-09-12T16:21:18.114 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.114 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.114 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.114 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.114 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.114 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.115 INFO:teuthology.orchestra.run.smithi130.stdout: "testkey": { 2023-09-12T16:21:18.115 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "testkey", 2023-09-12T16:21:18.115 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.115 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.115 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.116 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.116 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.116 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.116 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.116 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.117 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.117 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.117 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.117 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.117 INFO:teuthology.orchestra.run.smithi130.stdout: "testlkey": { 2023-09-12T16:21:18.117 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "testlkey", 2023-09-12T16:21:18.118 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.118 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.118 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.118 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.118 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.119 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.119 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.119 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.119 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.119 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.120 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.120 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.120 INFO:teuthology.orchestra.run.smithi130.stdout: "testnewline": { 2023-09-12T16:21:18.120 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "testnewline", 2023-09-12T16:21:18.120 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.121 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.121 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.121 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.121 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.121 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.121 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.122 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.122 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.122 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.122 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.122 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.123 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.123 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.123 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.123 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "snap_schedule", 2023-09-12T16:21:18.123 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.124 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.124 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.124 INFO:teuthology.orchestra.run.smithi130.stdout: "allow_m_granularity": { 2023-09-12T16:21:18.124 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "allow_m_granularity", 2023-09-12T16:21:18.124 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.124 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.125 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.125 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.125 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.125 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.125 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.126 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "allow minute scheduled snapshots", 2023-09-12T16:21:18.126 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.126 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.126 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.126 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.127 INFO:teuthology.orchestra.run.smithi130.stdout: "dump_on_update": { 2023-09-12T16:21:18.127 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "dump_on_update", 2023-09-12T16:21:18.127 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.127 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.127 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.127 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.128 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.128 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.128 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.128 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "dump database to debug log on update", 2023-09-12T16:21:18.128 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.129 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.129 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.129 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.129 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.129 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.130 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.130 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.130 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.130 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.130 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.131 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.131 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.131 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.131 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.131 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.131 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.132 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.132 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.132 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.132 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.132 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.132 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.133 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.133 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.133 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.133 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.133 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.134 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.134 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.134 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.134 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.134 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.135 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.135 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.135 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.135 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.135 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.135 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.136 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.136 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.136 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.136 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.136 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.137 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.137 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.137 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.137 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.137 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.137 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.138 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.138 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.138 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.138 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.138 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.139 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.139 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.139 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.139 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.139 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.140 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.140 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.140 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.140 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.140 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.140 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.141 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.141 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.141 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.141 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.141 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.142 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.142 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.142 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.142 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.142 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.143 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.143 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "stats", 2023-09-12T16:21:18.143 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.143 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.143 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.144 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.144 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.144 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.144 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.144 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.144 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.145 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.145 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.145 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.145 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.145 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.145 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.146 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.146 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.146 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.146 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.146 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.147 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.147 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.147 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.147 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.147 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.147 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.148 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.148 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.148 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.148 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.148 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.149 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.149 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.149 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.149 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.149 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.149 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.150 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.150 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.150 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.150 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.150 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.151 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.151 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.151 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.151 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.151 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.151 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.152 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.152 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.152 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.152 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.152 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.153 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.153 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.153 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.153 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.153 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.153 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.154 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.154 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.154 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.154 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.154 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.155 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.155 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.155 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.155 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.155 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.155 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.156 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.156 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.156 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.156 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.156 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.157 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.157 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "status", 2023-09-12T16:21:18.157 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.157 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.157 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.157 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.158 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.158 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.158 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.158 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.158 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.159 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.159 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.159 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.159 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.159 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.160 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.160 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.160 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.160 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.160 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.160 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.161 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.161 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.161 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.161 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.161 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.162 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.162 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.162 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.162 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.162 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.162 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.163 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.163 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.163 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.163 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.163 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.164 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.164 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.164 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.164 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.164 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.164 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.165 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.165 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.165 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.165 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.166 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.166 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.166 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.166 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.166 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.166 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.167 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.167 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.167 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.167 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.167 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.168 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.168 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.168 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.168 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.168 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.168 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.169 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.169 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.169 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.169 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.169 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.170 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.170 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.170 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.170 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.170 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.170 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.171 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.171 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.171 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "telegraf", 2023-09-12T16:21:18.171 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.171 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.172 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.172 INFO:teuthology.orchestra.run.smithi130.stdout: "address": { 2023-09-12T16:21:18.172 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "address", 2023-09-12T16:21:18.172 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.172 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.172 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.173 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-12T16:21:18.173 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.173 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.173 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.173 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.174 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.174 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.174 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.174 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.174 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:18.174 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:18.175 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.175 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.175 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.175 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "15", 2023-09-12T16:21:18.175 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.176 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.176 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.176 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.176 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.176 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.177 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.177 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.177 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.177 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.177 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.177 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.178 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.178 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.178 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.178 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.178 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.179 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.179 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.179 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.179 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.179 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.180 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.180 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.180 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.180 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.180 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.180 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.181 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.181 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.181 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.181 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.181 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.182 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.182 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.182 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.182 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.182 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.182 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.183 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.183 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.183 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.183 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.183 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.184 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.184 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.184 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.184 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.184 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.185 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.185 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.185 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.185 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.185 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.186 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.186 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.186 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.186 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.186 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.187 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.187 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.187 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.187 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.187 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.187 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.188 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.188 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.188 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.188 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.188 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.189 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.189 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.189 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.189 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.189 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.189 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.190 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.190 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.190 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.190 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.190 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.191 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "telemetry", 2023-09-12T16:21:18.191 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.191 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.191 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.191 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_basic": { 2023-09-12T16:21:18.192 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_basic", 2023-09-12T16:21:18.192 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.192 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.192 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.192 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.192 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.193 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.193 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.193 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-12T16:21:18.193 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.193 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.194 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.194 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.194 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_crash": { 2023-09-12T16:21:18.194 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_crash", 2023-09-12T16:21:18.194 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.195 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.195 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.195 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.195 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.195 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.195 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.196 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-12T16:21:18.196 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.196 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.196 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.196 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.197 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_device": { 2023-09-12T16:21:18.197 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_device", 2023-09-12T16:21:18.197 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.197 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.197 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.198 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.198 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.198 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.198 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.198 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-12T16:21:18.198 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.199 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.199 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.199 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.199 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_ident": { 2023-09-12T16:21:18.199 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_ident", 2023-09-12T16:21:18.200 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.200 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.200 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.200 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.200 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.201 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.201 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.201 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-12T16:21:18.201 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.201 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.202 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.202 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.202 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_perf": { 2023-09-12T16:21:18.202 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_perf", 2023-09-12T16:21:18.202 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.203 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.203 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.203 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.203 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.203 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.203 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.204 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-12T16:21:18.204 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.204 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.204 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.204 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.205 INFO:teuthology.orchestra.run.smithi130.stdout: "contact": { 2023-09-12T16:21:18.205 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "contact", 2023-09-12T16:21:18.205 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.205 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.205 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.206 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.206 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.206 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.206 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.206 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.206 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.207 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.207 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.207 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.207 INFO:teuthology.orchestra.run.smithi130.stdout: "description": { 2023-09-12T16:21:18.207 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "description", 2023-09-12T16:21:18.208 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.208 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.208 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.208 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.208 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.208 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.209 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.209 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.209 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.209 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.209 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.210 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.210 INFO:teuthology.orchestra.run.smithi130.stdout: "device_url": { 2023-09-12T16:21:18.210 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "device_url", 2023-09-12T16:21:18.210 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.210 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.211 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.211 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-12T16:21:18.211 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.211 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.211 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.211 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.212 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.212 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.212 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.212 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.212 INFO:teuthology.orchestra.run.smithi130.stdout: "enabled": { 2023-09-12T16:21:18.212 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enabled", 2023-09-12T16:21:18.213 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.213 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.213 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.213 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.213 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.213 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.214 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.214 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.214 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.214 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.214 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.215 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.215 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:18.215 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:18.215 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.215 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.216 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.216 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "24", 2023-09-12T16:21:18.216 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "8", 2023-09-12T16:21:18.216 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.216 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.217 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.217 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.217 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.217 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.217 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.217 INFO:teuthology.orchestra.run.smithi130.stdout: "last_opt_revision": { 2023-09-12T16:21:18.218 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "last_opt_revision", 2023-09-12T16:21:18.218 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.218 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.218 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.218 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "1", 2023-09-12T16:21:18.219 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.219 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.219 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.219 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.219 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.220 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.220 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.220 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.220 INFO:teuthology.orchestra.run.smithi130.stdout: "leaderboard": { 2023-09-12T16:21:18.220 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "leaderboard", 2023-09-12T16:21:18.221 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.221 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.221 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.221 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.221 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.221 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.222 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.222 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.222 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.222 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.222 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.223 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.223 INFO:teuthology.orchestra.run.smithi130.stdout: "leaderboard_description": { 2023-09-12T16:21:18.223 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "leaderboard_description", 2023-09-12T16:21:18.223 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.223 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.224 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.224 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.224 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.224 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.224 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.225 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.225 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.225 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.225 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.225 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.225 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.226 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.226 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.226 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.226 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.226 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.227 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.227 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.227 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.227 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.227 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.227 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.228 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.228 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.228 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.228 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.228 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.229 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.229 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.229 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.229 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.229 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.230 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.230 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.230 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.230 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.230 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.230 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.231 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.231 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.231 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.231 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.231 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.232 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.232 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.232 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.232 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.232 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.232 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.233 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.233 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.233 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.233 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.233 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.234 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.234 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.234 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.234 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.234 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.234 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.235 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.235 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.235 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.235 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.235 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.236 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.236 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.236 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.236 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.236 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.237 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.237 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.237 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.237 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.237 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.237 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.238 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.238 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.238 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.238 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.238 INFO:teuthology.orchestra.run.smithi130.stdout: "organization": { 2023-09-12T16:21:18.239 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "organization", 2023-09-12T16:21:18.239 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.239 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.239 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.239 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.240 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.240 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.240 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.240 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.240 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.240 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.241 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.241 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.241 INFO:teuthology.orchestra.run.smithi130.stdout: "proxy": { 2023-09-12T16:21:18.241 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "proxy", 2023-09-12T16:21:18.241 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.242 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.242 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.245 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.245 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.245 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.246 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.246 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.246 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.246 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.246 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.247 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.247 INFO:teuthology.orchestra.run.smithi130.stdout: "url": { 2023-09-12T16:21:18.247 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "url", 2023-09-12T16:21:18.247 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.247 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.247 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.248 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-12T16:21:18.248 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.248 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.248 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.248 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.249 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.249 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.249 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.249 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.249 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.249 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.250 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.250 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "test_orchestrator", 2023-09-12T16:21:18.250 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.250 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.250 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.251 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.251 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.251 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.251 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.251 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.251 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.252 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.252 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.252 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.252 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.252 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.253 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.253 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.253 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.253 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.253 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.253 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.254 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.254 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.254 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.254 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.254 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.255 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.255 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.255 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.255 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.255 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.255 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.256 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.256 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.256 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.256 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.256 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.257 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.257 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.257 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.257 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.257 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.258 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.258 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.258 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.258 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.258 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.258 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.259 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.259 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.259 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.259 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.259 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.260 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.260 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.260 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.260 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.260 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.260 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.261 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.261 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.261 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.261 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.261 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.262 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.262 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.262 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.262 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.262 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.263 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.263 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.263 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.263 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.263 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.263 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.264 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.264 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.264 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "volumes", 2023-09-12T16:21:18.264 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.264 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.265 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.265 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.265 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.265 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.265 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.265 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.266 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.266 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.266 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.266 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.266 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.267 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.267 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.267 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.267 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.267 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.267 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.268 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.268 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.268 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.268 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.268 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.269 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.269 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.269 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.269 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.269 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.269 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.270 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.270 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.270 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.270 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.270 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.271 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.271 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.271 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.271 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.271 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.272 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.272 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.272 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.272 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.272 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.272 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.273 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.273 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.273 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.273 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.273 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.274 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.274 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.274 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.274 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.274 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.274 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.275 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.275 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.275 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.275 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.275 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.276 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.276 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.276 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.276 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.276 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.277 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.277 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.277 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.277 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.277 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.277 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.278 INFO:teuthology.orchestra.run.smithi130.stdout: "max_concurrent_clones": { 2023-09-12T16:21:18.278 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_concurrent_clones", 2023-09-12T16:21:18.278 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.278 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.278 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.279 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "4", 2023-09-12T16:21:18.279 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.279 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.279 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.279 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-12T16:21:18.280 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.280 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.280 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.280 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.280 INFO:teuthology.orchestra.run.smithi130.stdout: "snapshot_clone_delay": { 2023-09-12T16:21:18.280 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "snapshot_clone_delay", 2023-09-12T16:21:18.281 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.281 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.281 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.281 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:18.281 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.282 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.282 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.282 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-12T16:21:18.282 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.282 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.282 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.283 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.283 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.283 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.283 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.283 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix", 2023-09-12T16:21:18.284 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.284 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.284 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.284 INFO:teuthology.orchestra.run.smithi130.stdout: "discovery_interval": { 2023-09-12T16:21:18.284 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "discovery_interval", 2023-09-12T16:21:18.284 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:18.285 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.285 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.285 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "100", 2023-09-12T16:21:18.285 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.285 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.286 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.286 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.286 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.286 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.286 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.287 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.287 INFO:teuthology.orchestra.run.smithi130.stdout: "identifier": { 2023-09-12T16:21:18.287 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "identifier", 2023-09-12T16:21:18.287 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.287 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.287 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.288 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.288 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.288 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.288 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.288 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.289 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.289 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.289 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.289 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.289 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:18.290 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:18.290 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.290 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.290 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.290 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:18.290 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.291 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.291 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.291 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.291 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.291 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.292 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.292 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.292 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.292 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.292 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.293 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.293 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.293 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.293 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.293 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.293 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.294 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.294 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.294 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.294 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.294 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.295 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.295 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.295 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.295 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.295 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.296 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.296 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.296 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.296 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.296 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.297 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.297 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.297 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.297 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.297 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.297 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.298 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.298 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.298 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.298 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.298 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.299 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.299 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.299 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.299 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.299 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.299 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.300 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.300 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.300 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.300 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.300 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.301 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.301 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.301 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.301 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.301 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.302 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.302 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.302 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.302 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.302 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.302 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.303 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.303 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.303 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.303 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.303 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.304 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.304 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.304 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.304 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.304 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.305 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.305 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.305 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.305 INFO:teuthology.orchestra.run.smithi130.stdout: "zabbix_host": { 2023-09-12T16:21:18.305 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix_host", 2023-09-12T16:21:18.305 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.306 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.306 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.306 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.306 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.306 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.307 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.307 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.307 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.307 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.307 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.308 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.308 INFO:teuthology.orchestra.run.smithi130.stdout: "zabbix_port": { 2023-09-12T16:21:18.308 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix_port", 2023-09-12T16:21:18.308 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.308 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.308 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.309 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10051", 2023-09-12T16:21:18.309 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.309 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.309 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.309 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.310 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.310 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.310 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.310 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.310 INFO:teuthology.orchestra.run.smithi130.stdout: "zabbix_sender": { 2023-09-12T16:21:18.310 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix_sender", 2023-09-12T16:21:18.311 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.311 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.311 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.311 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-12T16:21:18.312 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.312 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.312 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.312 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.312 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.312 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.313 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.313 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.313 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.313 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.313 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T16:21:18.314 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.314 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.314 INFO:teuthology.orchestra.run.smithi130.stdout: "modules": [ 2023-09-12T16:21:18.314 INFO:teuthology.orchestra.run.smithi130.stdout: "cephadm", 2023-09-12T16:21:18.314 INFO:teuthology.orchestra.run.smithi130.stdout: "dashboard", 2023-09-12T16:21:18.314 INFO:teuthology.orchestra.run.smithi130.stdout: "iostat", 2023-09-12T16:21:18.315 INFO:teuthology.orchestra.run.smithi130.stdout: "nfs", 2023-09-12T16:21:18.315 INFO:teuthology.orchestra.run.smithi130.stdout: "restful" 2023-09-12T16:21:18.315 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.315 INFO:teuthology.orchestra.run.smithi130.stdout: "available_modules": [ 2023-09-12T16:21:18.315 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.316 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "alerts", 2023-09-12T16:21:18.316 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.316 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.316 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.316 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:18.316 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:18.317 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.317 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.317 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.317 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:18.317 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.318 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.318 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.318 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How frequently to reexamine health status", 2023-09-12T16:21:18.318 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.318 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.318 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.319 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.319 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.319 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.319 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.319 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.320 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.320 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.320 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.320 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.320 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.320 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.321 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.321 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.321 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.321 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.321 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.322 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.322 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.322 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.322 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.322 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.323 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.323 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.323 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.323 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.323 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.324 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.324 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.324 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.324 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.324 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.325 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.325 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.325 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.325 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.325 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.325 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.326 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.326 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.326 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.326 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.326 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.327 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.327 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.327 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.327 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.327 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.328 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.328 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.328 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.328 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.328 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.328 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.329 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.329 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.329 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.329 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.329 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.330 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.330 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.330 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.330 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.330 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.330 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.331 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.331 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.331 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.331 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.331 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.332 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.332 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.332 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_destination": { 2023-09-12T16:21:18.332 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_destination", 2023-09-12T16:21:18.332 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.333 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.333 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.333 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.333 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.333 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.334 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.334 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Email address to send alerts to", 2023-09-12T16:21:18.334 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.334 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.334 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.335 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.335 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_from_name": { 2023-09-12T16:21:18.335 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_from_name", 2023-09-12T16:21:18.335 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.335 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.335 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.336 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "Ceph", 2023-09-12T16:21:18.336 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.336 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.336 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.336 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Email From: name", 2023-09-12T16:21:18.337 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.337 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.337 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.337 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.337 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_host": { 2023-09-12T16:21:18.338 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_host", 2023-09-12T16:21:18.338 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.338 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.338 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.338 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.338 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.339 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.339 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.339 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SMTP server", 2023-09-12T16:21:18.339 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.339 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.340 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.340 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.340 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_password": { 2023-09-12T16:21:18.340 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_password", 2023-09-12T16:21:18.340 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.341 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.341 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.341 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.341 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.341 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.341 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.342 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Password to authenticate with", 2023-09-12T16:21:18.342 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.342 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.342 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.342 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.343 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_port": { 2023-09-12T16:21:18.343 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_port", 2023-09-12T16:21:18.343 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.343 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.343 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.343 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "465", 2023-09-12T16:21:18.344 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.344 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.344 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.344 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SMTP port", 2023-09-12T16:21:18.344 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.345 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.345 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.345 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.345 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_sender": { 2023-09-12T16:21:18.345 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_sender", 2023-09-12T16:21:18.345 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.346 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.346 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.346 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.346 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.346 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.347 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.347 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SMTP envelope sender", 2023-09-12T16:21:18.347 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.347 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.347 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.348 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.348 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_ssl": { 2023-09-12T16:21:18.348 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_ssl", 2023-09-12T16:21:18.348 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.348 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.348 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.349 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.349 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.349 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.349 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.349 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-12T16:21:18.350 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.350 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.350 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.350 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.350 INFO:teuthology.orchestra.run.smithi130.stdout: "smtp_user": { 2023-09-12T16:21:18.350 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "smtp_user", 2023-09-12T16:21:18.351 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.351 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.351 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.351 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.351 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.352 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.352 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.352 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "User to authenticate as", 2023-09-12T16:21:18.352 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.352 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.352 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.353 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.353 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.353 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.353 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.353 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "balancer", 2023-09-12T16:21:18.354 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.354 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.354 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.354 INFO:teuthology.orchestra.run.smithi130.stdout: "active": { 2023-09-12T16:21:18.354 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "active", 2023-09-12T16:21:18.354 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.355 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.355 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.355 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.355 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.355 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.356 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.356 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "automatically balance PGs across cluster", 2023-09-12T16:21:18.356 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.356 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.356 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.357 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.357 INFO:teuthology.orchestra.run.smithi130.stdout: "begin_time": { 2023-09-12T16:21:18.357 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "begin_time", 2023-09-12T16:21:18.357 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.357 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.357 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.358 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0000", 2023-09-12T16:21:18.358 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.358 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.358 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.358 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "beginning time of day to automatically balance", 2023-09-12T16:21:18.359 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-12T16:21:18.359 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.359 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.359 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.359 INFO:teuthology.orchestra.run.smithi130.stdout: "begin_weekday": { 2023-09-12T16:21:18.360 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "begin_weekday", 2023-09-12T16:21:18.360 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:18.360 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.360 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.360 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:18.360 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "0", 2023-09-12T16:21:18.361 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "6", 2023-09-12T16:21:18.361 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.361 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-12T16:21:18.361 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-12T16:21:18.361 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.362 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.362 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.362 INFO:teuthology.orchestra.run.smithi130.stdout: "crush_compat_max_iterations": { 2023-09-12T16:21:18.362 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crush_compat_max_iterations", 2023-09-12T16:21:18.362 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:18.363 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.363 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.363 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "25", 2023-09-12T16:21:18.363 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:18.363 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "250", 2023-09-12T16:21:18.363 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.364 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-12T16:21:18.364 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.364 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.364 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.364 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.365 INFO:teuthology.orchestra.run.smithi130.stdout: "crush_compat_metrics": { 2023-09-12T16:21:18.365 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crush_compat_metrics", 2023-09-12T16:21:18.365 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.365 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.365 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.365 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "pgs,objects,bytes", 2023-09-12T16:21:18.366 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.366 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.366 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.366 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-12T16:21:18.366 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-12T16:21:18.367 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.367 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.367 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.367 INFO:teuthology.orchestra.run.smithi130.stdout: "crush_compat_step": { 2023-09-12T16:21:18.367 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crush_compat_step", 2023-09-12T16:21:18.368 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:18.368 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.368 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.368 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0.5", 2023-09-12T16:21:18.368 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "0.001", 2023-09-12T16:21:18.368 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "0.999", 2023-09-12T16:21:18.369 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.369 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "aggressiveness of optimization", 2023-09-12T16:21:18.369 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-12T16:21:18.369 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.369 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.370 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.370 INFO:teuthology.orchestra.run.smithi130.stdout: "end_time": { 2023-09-12T16:21:18.370 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "end_time", 2023-09-12T16:21:18.370 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.370 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.370 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.371 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "2359", 2023-09-12T16:21:18.371 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.371 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.371 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.371 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "ending time of day to automatically balance", 2023-09-12T16:21:18.372 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-12T16:21:18.372 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.372 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.372 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.372 INFO:teuthology.orchestra.run.smithi130.stdout: "end_weekday": { 2023-09-12T16:21:18.372 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "end_weekday", 2023-09-12T16:21:18.373 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:18.373 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.373 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.373 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:18.373 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "0", 2023-09-12T16:21:18.374 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "6", 2023-09-12T16:21:18.374 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.374 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-12T16:21:18.374 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-12T16:21:18.374 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.375 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.375 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.375 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.375 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.375 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.376 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.376 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.376 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.376 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.376 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.377 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.377 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.377 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.377 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.377 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.378 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.378 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.378 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.378 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.378 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.378 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.379 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.379 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.379 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.379 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.379 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.380 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.380 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.380 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.380 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.380 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.381 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.381 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.381 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.381 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.381 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.381 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.382 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.382 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.382 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.382 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.382 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.383 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.383 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.383 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.383 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.383 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.383 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.384 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.384 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.384 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.384 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.384 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.385 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.385 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.385 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.385 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.385 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.385 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.386 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.386 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.386 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.386 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.386 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.387 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.387 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.387 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.387 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.387 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.387 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.388 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.388 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.388 INFO:teuthology.orchestra.run.smithi130.stdout: "min_score": { 2023-09-12T16:21:18.388 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "min_score", 2023-09-12T16:21:18.388 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:18.389 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.389 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.389 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:18.389 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.389 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.389 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.390 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-12T16:21:18.390 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.390 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.390 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.390 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.391 INFO:teuthology.orchestra.run.smithi130.stdout: "mode": { 2023-09-12T16:21:18.391 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mode", 2023-09-12T16:21:18.391 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.391 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.391 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.392 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "upmap", 2023-09-12T16:21:18.392 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.392 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.392 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.392 INFO:teuthology.orchestra.run.smithi130.stdout: "crush-compat", 2023-09-12T16:21:18.392 INFO:teuthology.orchestra.run.smithi130.stdout: "none", 2023-09-12T16:21:18.393 INFO:teuthology.orchestra.run.smithi130.stdout: "upmap" 2023-09-12T16:21:18.393 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.393 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Balancer mode", 2023-09-12T16:21:18.393 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.393 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.394 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.394 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.394 INFO:teuthology.orchestra.run.smithi130.stdout: "pool_ids": { 2023-09-12T16:21:18.394 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pool_ids", 2023-09-12T16:21:18.394 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.395 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.395 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.395 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.395 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.395 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.396 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.396 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-12T16:21:18.396 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.396 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.396 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.396 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.397 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:18.397 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:18.397 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.397 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.397 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.398 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:18.398 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.398 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.398 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.398 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-12T16:21:18.399 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.399 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.399 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.399 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.399 INFO:teuthology.orchestra.run.smithi130.stdout: "upmap_max_deviation": { 2023-09-12T16:21:18.400 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "upmap_max_deviation", 2023-09-12T16:21:18.400 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.400 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.400 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.400 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:18.400 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:18.401 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.401 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.401 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-12T16:21:18.401 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-12T16:21:18.401 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.401 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.402 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.402 INFO:teuthology.orchestra.run.smithi130.stdout: "upmap_max_optimizations": { 2023-09-12T16:21:18.402 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "upmap_max_optimizations", 2023-09-12T16:21:18.402 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:18.402 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.403 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.403 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:18.403 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.403 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.403 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.404 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-12T16:21:18.404 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.404 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.404 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.404 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.404 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.405 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.405 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.405 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cephadm", 2023-09-12T16:21:18.405 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.405 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.406 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.406 INFO:teuthology.orchestra.run.smithi130.stdout: "agent_down_multiplier": { 2023-09-12T16:21:18.406 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "agent_down_multiplier", 2023-09-12T16:21:18.406 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:18.406 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.406 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.407 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "3.0", 2023-09-12T16:21:18.407 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.407 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.407 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.407 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-12T16:21:18.408 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.408 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.408 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.408 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.408 INFO:teuthology.orchestra.run.smithi130.stdout: "agent_refresh_rate": { 2023-09-12T16:21:18.409 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "agent_refresh_rate", 2023-09-12T16:21:18.409 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.409 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.409 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.409 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "20", 2023-09-12T16:21:18.410 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.410 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.410 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.410 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-12T16:21:18.410 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.411 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.411 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.411 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.411 INFO:teuthology.orchestra.run.smithi130.stdout: "agent_starting_port": { 2023-09-12T16:21:18.411 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "agent_starting_port", 2023-09-12T16:21:18.411 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.412 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.412 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.412 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "4721", 2023-09-12T16:21:18.412 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.412 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.413 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.413 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-12T16:21:18.413 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.413 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.413 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.414 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.414 INFO:teuthology.orchestra.run.smithi130.stdout: "allow_ptrace": { 2023-09-12T16:21:18.414 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "allow_ptrace", 2023-09-12T16:21:18.414 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.414 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.414 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.415 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.415 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.415 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.415 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.415 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-12T16:21:18.416 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-12T16:21:18.416 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.416 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.416 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.416 INFO:teuthology.orchestra.run.smithi130.stdout: "autotune_interval": { 2023-09-12T16:21:18.417 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "autotune_interval", 2023-09-12T16:21:18.417 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.417 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.417 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.417 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:18.417 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.418 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.418 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.418 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-12T16:21:18.418 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.418 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.419 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.419 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.419 INFO:teuthology.orchestra.run.smithi130.stdout: "autotune_memory_target_ratio": { 2023-09-12T16:21:18.419 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "autotune_memory_target_ratio", 2023-09-12T16:21:18.419 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:18.420 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.420 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.420 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0.7", 2023-09-12T16:21:18.420 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.420 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.420 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.421 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-12T16:21:18.421 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.421 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.421 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.421 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.422 INFO:teuthology.orchestra.run.smithi130.stdout: "cgroups_split": { 2023-09-12T16:21:18.422 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cgroups_split", 2023-09-12T16:21:18.422 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.422 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.422 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.423 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.423 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.423 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.423 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.423 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-12T16:21:18.423 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.424 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.424 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.424 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.424 INFO:teuthology.orchestra.run.smithi130.stdout: "config_checks_enabled": { 2023-09-12T16:21:18.424 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "config_checks_enabled", 2023-09-12T16:21:18.425 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.425 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.425 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.425 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.425 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.426 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.426 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.426 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-12T16:21:18.426 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.426 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.426 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.427 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.427 INFO:teuthology.orchestra.run.smithi130.stdout: "config_dashboard": { 2023-09-12T16:21:18.427 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "config_dashboard", 2023-09-12T16:21:18.427 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.427 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.428 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.428 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.428 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.428 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.428 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.429 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-12T16:21:18.429 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.429 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.429 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.429 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.430 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_alertmanager": { 2023-09-12T16:21:18.430 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_alertmanager", 2023-09-12T16:21:18.430 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.430 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.430 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.431 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-12T16:21:18.431 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.431 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.431 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.431 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:18.431 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.432 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.432 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.432 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.432 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_base": { 2023-09-12T16:21:18.432 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_base", 2023-09-12T16:21:18.433 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.433 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.433 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.433 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-12T16:21:18.434 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.434 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.434 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.434 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Container image name, without the tag", 2023-09-12T16:21:18.434 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.435 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.435 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.435 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.435 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_elasticsearch": { 2023-09-12T16:21:18.435 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_elasticsearch", 2023-09-12T16:21:18.435 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.436 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.436 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.436 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-12T16:21:18.436 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.436 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.437 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.437 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "elasticsearch container image", 2023-09-12T16:21:18.437 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.437 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.437 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.438 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.438 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_grafana": { 2023-09-12T16:21:18.438 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_grafana", 2023-09-12T16:21:18.438 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.438 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.438 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.439 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-12T16:21:18.439 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.439 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.439 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.439 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:18.440 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.440 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.440 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.440 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.440 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_haproxy": { 2023-09-12T16:21:18.441 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_haproxy", 2023-09-12T16:21:18.441 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.441 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.441 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.441 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-12T16:21:18.441 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.442 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.442 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.442 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "HAproxy container image", 2023-09-12T16:21:18.442 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.442 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.443 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.443 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.443 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_jaeger_agent": { 2023-09-12T16:21:18.443 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_jaeger_agent", 2023-09-12T16:21:18.443 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.443 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.444 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.444 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-12T16:21:18.444 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.444 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.444 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.445 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Jaeger agent container image", 2023-09-12T16:21:18.445 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.445 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.445 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.445 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.445 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_jaeger_collector": { 2023-09-12T16:21:18.446 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_jaeger_collector", 2023-09-12T16:21:18.446 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.446 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.446 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.446 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-12T16:21:18.447 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.447 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.449 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.449 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Jaeger collector container image", 2023-09-12T16:21:18.449 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.450 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.450 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.450 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.450 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_jaeger_query": { 2023-09-12T16:21:18.450 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_jaeger_query", 2023-09-12T16:21:18.451 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.451 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.451 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.451 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-12T16:21:18.451 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.451 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.452 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.452 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Jaeger query container image", 2023-09-12T16:21:18.452 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.452 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.452 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.453 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.453 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_keepalived": { 2023-09-12T16:21:18.453 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_keepalived", 2023-09-12T16:21:18.453 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.453 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.454 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.454 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-12T16:21:18.454 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.454 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.454 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.455 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Keepalived container image", 2023-09-12T16:21:18.455 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.455 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.455 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.455 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.455 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_loki": { 2023-09-12T16:21:18.456 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_loki", 2023-09-12T16:21:18.456 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.456 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.456 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.456 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-12T16:21:18.457 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.457 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.457 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.457 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Loki container image", 2023-09-12T16:21:18.457 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.458 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.458 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.458 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.458 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_node_exporter": { 2023-09-12T16:21:18.458 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_node_exporter", 2023-09-12T16:21:18.458 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.459 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.459 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.459 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-12T16:21:18.459 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.459 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.460 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.460 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:18.460 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.460 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.460 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.461 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.461 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_nvmeof": { 2023-09-12T16:21:18.461 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_nvmeof", 2023-09-12T16:21:18.461 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.461 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.462 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.462 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-12T16:21:18.462 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.462 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.462 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.463 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Nvme-of container image", 2023-09-12T16:21:18.463 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.463 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.463 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.463 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.463 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_prometheus": { 2023-09-12T16:21:18.464 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_prometheus", 2023-09-12T16:21:18.464 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.464 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.464 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.464 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-12T16:21:18.465 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.465 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.465 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.465 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Prometheus container image", 2023-09-12T16:21:18.465 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.465 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.466 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.466 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.466 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_promtail": { 2023-09-12T16:21:18.466 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_promtail", 2023-09-12T16:21:18.466 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.467 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.467 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.467 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-12T16:21:18.467 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.467 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.467 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.468 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Promtail container image", 2023-09-12T16:21:18.468 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.468 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.468 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.468 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.469 INFO:teuthology.orchestra.run.smithi130.stdout: "container_image_snmp_gateway": { 2023-09-12T16:21:18.469 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_image_snmp_gateway", 2023-09-12T16:21:18.469 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.469 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.469 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.469 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-12T16:21:18.470 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.470 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.470 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.470 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "SNMP Gateway container image", 2023-09-12T16:21:18.470 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.471 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.471 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.471 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.471 INFO:teuthology.orchestra.run.smithi130.stdout: "container_init": { 2023-09-12T16:21:18.471 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "container_init", 2023-09-12T16:21:18.472 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.472 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.472 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.472 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.472 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.473 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.473 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.473 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Run podman/docker with `--init`", 2023-09-12T16:21:18.473 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.473 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.474 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.474 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.474 INFO:teuthology.orchestra.run.smithi130.stdout: "daemon_cache_timeout": { 2023-09-12T16:21:18.474 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "daemon_cache_timeout", 2023-09-12T16:21:18.475 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.475 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.475 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.475 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:18.476 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.476 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.476 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.476 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-12T16:21:18.477 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.477 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.477 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.478 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.478 INFO:teuthology.orchestra.run.smithi130.stdout: "default_cephadm_command_timeout": { 2023-09-12T16:21:18.478 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "default_cephadm_command_timeout", 2023-09-12T16:21:18.478 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.479 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.479 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.479 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "900", 2023-09-12T16:21:18.479 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.479 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.480 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.480 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-12T16:21:18.480 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.480 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.480 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.481 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.481 INFO:teuthology.orchestra.run.smithi130.stdout: "default_registry": { 2023-09-12T16:21:18.481 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "default_registry", 2023-09-12T16:21:18.481 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.481 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.482 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.482 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "docker.io", 2023-09-12T16:21:18.482 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.482 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.482 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.483 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-12T16:21:18.483 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.483 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.483 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.483 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.484 INFO:teuthology.orchestra.run.smithi130.stdout: "device_cache_timeout": { 2023-09-12T16:21:18.484 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "device_cache_timeout", 2023-09-12T16:21:18.484 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.484 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.484 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.484 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "1800", 2023-09-12T16:21:18.485 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.485 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.485 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.485 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "seconds to cache device inventory", 2023-09-12T16:21:18.485 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.486 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.486 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.486 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.486 INFO:teuthology.orchestra.run.smithi130.stdout: "device_enhanced_scan": { 2023-09-12T16:21:18.486 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "device_enhanced_scan", 2023-09-12T16:21:18.487 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.487 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.487 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.487 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.487 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.488 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.488 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.488 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-12T16:21:18.488 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.488 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.489 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.489 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.489 INFO:teuthology.orchestra.run.smithi130.stdout: "facts_cache_timeout": { 2023-09-12T16:21:18.489 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "facts_cache_timeout", 2023-09-12T16:21:18.489 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.490 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.490 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.490 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:18.490 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.490 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.491 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.491 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "seconds to cache host facts data", 2023-09-12T16:21:18.491 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.491 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.491 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.491 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.492 INFO:teuthology.orchestra.run.smithi130.stdout: "host_check_interval": { 2023-09-12T16:21:18.492 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "host_check_interval", 2023-09-12T16:21:18.492 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.492 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.492 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.493 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:18.493 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.493 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.493 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.493 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to perform a host check", 2023-09-12T16:21:18.494 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.494 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.494 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.494 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.494 INFO:teuthology.orchestra.run.smithi130.stdout: "inventory_list_all": { 2023-09-12T16:21:18.495 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "inventory_list_all", 2023-09-12T16:21:18.495 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.495 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.495 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.495 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.496 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.496 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.496 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.496 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-12T16:21:18.496 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.497 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.497 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.497 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.497 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.497 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.498 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.498 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.498 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.498 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.498 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.499 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.499 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.499 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.499 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.499 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.500 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.500 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.500 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.500 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.500 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.500 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.501 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.501 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.501 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.501 INFO:teuthology.orchestra.run.smithi130.stdout: "log_refresh_metadata": { 2023-09-12T16:21:18.501 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_refresh_metadata", 2023-09-12T16:21:18.502 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.502 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.502 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.502 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.502 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.503 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.503 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.503 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-12T16:21:18.503 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.503 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.503 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.504 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.504 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.504 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.504 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.504 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.505 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.505 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.505 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.505 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.505 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.506 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-12T16:21:18.506 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.506 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.506 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.506 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.506 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.507 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.507 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.507 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.507 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.507 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.508 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.508 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.508 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.508 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.508 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.508 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.509 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.509 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.509 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.509 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.509 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.510 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.510 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.510 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.510 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.510 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.511 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.511 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.511 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.511 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.511 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.511 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.512 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.512 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.512 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.512 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.512 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.513 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.513 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.513 INFO:teuthology.orchestra.run.smithi130.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-12T16:21:18.513 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-12T16:21:18.513 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.514 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.514 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.514 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.514 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.514 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.514 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.515 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-12T16:21:18.515 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.515 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.515 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.516 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.516 INFO:teuthology.orchestra.run.smithi130.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-12T16:21:18.516 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-12T16:21:18.516 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.516 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.517 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.517 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "*", 2023-09-12T16:21:18.517 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.517 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.517 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.517 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-12T16:21:18.518 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.518 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.518 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.518 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.518 INFO:teuthology.orchestra.run.smithi130.stdout: "max_count_per_host": { 2023-09-12T16:21:18.519 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_count_per_host", 2023-09-12T16:21:18.519 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.519 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.519 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.520 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:18.520 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.520 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.520 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.520 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "max number of daemons per service per host", 2023-09-12T16:21:18.520 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.521 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.521 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.521 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.521 INFO:teuthology.orchestra.run.smithi130.stdout: "max_osd_draining_count": { 2023-09-12T16:21:18.521 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_osd_draining_count", 2023-09-12T16:21:18.522 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.522 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.522 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.522 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:18.522 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.523 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.523 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.523 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-12T16:21:18.523 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.523 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.523 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.524 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.524 INFO:teuthology.orchestra.run.smithi130.stdout: "migration_current": { 2023-09-12T16:21:18.524 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "migration_current", 2023-09-12T16:21:18.524 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.524 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.525 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.525 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.525 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.525 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.525 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.526 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "internal - do not modify", 2023-09-12T16:21:18.526 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.526 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.526 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.526 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.527 INFO:teuthology.orchestra.run.smithi130.stdout: "mode": { 2023-09-12T16:21:18.527 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mode", 2023-09-12T16:21:18.527 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.527 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.527 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.527 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "root", 2023-09-12T16:21:18.528 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.528 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.528 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.528 INFO:teuthology.orchestra.run.smithi130.stdout: "cephadm-package", 2023-09-12T16:21:18.528 INFO:teuthology.orchestra.run.smithi130.stdout: "root" 2023-09-12T16:21:18.529 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.529 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "mode for remote execution of cephadm", 2023-09-12T16:21:18.529 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.529 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.529 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.530 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.530 INFO:teuthology.orchestra.run.smithi130.stdout: "prometheus_alerts_path": { 2023-09-12T16:21:18.530 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "prometheus_alerts_path", 2023-09-12T16:21:18.530 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.530 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.531 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.531 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-12T16:21:18.531 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.531 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.531 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.531 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-12T16:21:18.532 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.532 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.532 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.532 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.532 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_insecure": { 2023-09-12T16:21:18.533 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_insecure", 2023-09-12T16:21:18.533 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.533 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.533 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.533 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.534 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.534 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.534 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.534 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-12T16:21:18.534 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.534 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.535 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.535 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.535 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_password": { 2023-09-12T16:21:18.535 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_password", 2023-09-12T16:21:18.535 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.536 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.536 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.536 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.536 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.536 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.536 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.537 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-12T16:21:18.537 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.537 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.537 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.537 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.538 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_url": { 2023-09-12T16:21:18.538 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_url", 2023-09-12T16:21:18.538 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.538 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.538 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.539 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.539 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.539 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.539 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.539 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-12T16:21:18.539 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.540 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.540 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.540 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.540 INFO:teuthology.orchestra.run.smithi130.stdout: "registry_username": { 2023-09-12T16:21:18.540 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "registry_username", 2023-09-12T16:21:18.541 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.541 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.541 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.541 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.541 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.542 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.542 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.542 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-12T16:21:18.542 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.542 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.542 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.543 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.543 INFO:teuthology.orchestra.run.smithi130.stdout: "secure_monitoring_stack": { 2023-09-12T16:21:18.543 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "secure_monitoring_stack", 2023-09-12T16:21:18.543 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.543 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.544 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.544 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.544 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.544 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.544 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.544 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-12T16:21:18.545 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.545 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.545 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.545 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.545 INFO:teuthology.orchestra.run.smithi130.stdout: "service_discovery_port": { 2023-09-12T16:21:18.546 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "service_discovery_port", 2023-09-12T16:21:18.546 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.546 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.546 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.546 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8765", 2023-09-12T16:21:18.547 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.547 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.547 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.547 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "cephadm service discovery port", 2023-09-12T16:21:18.547 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.548 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.548 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.548 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.548 INFO:teuthology.orchestra.run.smithi130.stdout: "ssh_config_file": { 2023-09-12T16:21:18.548 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssh_config_file", 2023-09-12T16:21:18.548 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.549 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.549 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.549 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.549 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.549 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.550 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.550 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-12T16:21:18.550 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.550 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.550 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.551 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.551 INFO:teuthology.orchestra.run.smithi130.stdout: "use_agent": { 2023-09-12T16:21:18.551 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "use_agent", 2023-09-12T16:21:18.551 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.551 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.551 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.552 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.552 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.552 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.552 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.552 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-12T16:21:18.553 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.553 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.553 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.553 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.553 INFO:teuthology.orchestra.run.smithi130.stdout: "use_repo_digest": { 2023-09-12T16:21:18.553 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "use_repo_digest", 2023-09-12T16:21:18.554 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.554 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.554 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.554 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.554 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.555 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.555 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.555 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-12T16:21:18.555 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.555 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.556 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.556 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.556 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_on_failed_host_check": { 2023-09-12T16:21:18.556 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_on_failed_host_check", 2023-09-12T16:21:18.556 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.556 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.557 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.557 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.557 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.557 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.557 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.558 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise a health warning if the host check fails", 2023-09-12T16:21:18.558 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.558 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.558 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.558 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.559 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_on_stray_daemons": { 2023-09-12T16:21:18.559 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_on_stray_daemons", 2023-09-12T16:21:18.559 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.559 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.559 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.560 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.560 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.560 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.560 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.560 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-12T16:21:18.560 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.561 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.561 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.561 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.561 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_on_stray_hosts": { 2023-09-12T16:21:18.561 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_on_stray_hosts", 2023-09-12T16:21:18.562 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.562 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.562 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.562 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.562 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.563 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.563 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.563 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-12T16:21:18.563 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.563 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.563 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.564 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.564 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.564 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.564 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.564 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crash", 2023-09-12T16:21:18.565 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.565 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.565 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.565 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.565 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.566 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.566 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.566 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.566 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.566 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.567 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.567 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.567 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.567 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.567 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.567 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.568 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.568 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.568 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.568 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.568 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.569 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.569 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.569 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.569 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.569 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.569 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.570 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.570 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.570 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.570 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.570 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.571 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.571 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.571 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.571 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.571 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.571 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.572 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.572 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.572 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.572 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.572 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.573 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.573 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.573 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.573 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.573 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.574 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.574 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.574 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.574 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.574 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.575 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.575 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.575 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.575 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.575 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.575 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.576 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.576 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.576 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.576 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.577 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.577 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.577 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.577 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.577 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.578 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.578 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.578 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.578 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.578 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.579 INFO:teuthology.orchestra.run.smithi130.stdout: "retain_interval": { 2023-09-12T16:21:18.579 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "retain_interval", 2023-09-12T16:21:18.579 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.579 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.579 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.580 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "31536000", 2023-09-12T16:21:18.580 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.580 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.580 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.580 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-12T16:21:18.581 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.581 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.581 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.581 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.582 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_recent_interval": { 2023-09-12T16:21:18.582 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_recent_interval", 2023-09-12T16:21:18.582 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.582 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.582 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.583 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "1209600", 2023-09-12T16:21:18.583 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.583 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.583 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.583 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-12T16:21:18.584 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.584 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.584 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.584 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.584 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.584 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.585 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.585 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "dashboard", 2023-09-12T16:21:18.585 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.585 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.585 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.586 INFO:teuthology.orchestra.run.smithi130.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-12T16:21:18.586 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-12T16:21:18.586 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.586 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.586 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.587 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:18.587 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.587 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.587 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.587 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.588 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.588 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.588 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.588 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.588 INFO:teuthology.orchestra.run.smithi130.stdout: "ALERTMANAGER_API_HOST": { 2023-09-12T16:21:18.589 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-12T16:21:18.589 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.589 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.589 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.589 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.590 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.590 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.590 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.590 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.590 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.590 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.591 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.591 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.591 INFO:teuthology.orchestra.run.smithi130.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-12T16:21:18.591 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-12T16:21:18.591 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.592 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.592 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.592 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.592 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.592 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.593 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.593 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.593 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.593 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.593 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.593 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.594 INFO:teuthology.orchestra.run.smithi130.stdout: "AUDIT_API_ENABLED": { 2023-09-12T16:21:18.594 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "AUDIT_API_ENABLED", 2023-09-12T16:21:18.594 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.594 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.594 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.595 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.595 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.595 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.595 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.595 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.595 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.596 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.596 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.596 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.596 INFO:teuthology.orchestra.run.smithi130.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-12T16:21:18.596 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-12T16:21:18.597 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.597 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.597 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.597 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.597 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.598 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.598 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.598 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.598 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.598 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.598 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.599 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.599 INFO:teuthology.orchestra.run.smithi130.stdout: "ENABLE_BROWSABLE_API": { 2023-09-12T16:21:18.599 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-12T16:21:18.599 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.599 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.600 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.600 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.600 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.600 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.600 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.601 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.601 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.601 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.601 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.601 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.601 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-12T16:21:18.602 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-12T16:21:18.602 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.602 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.602 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.602 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.603 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.603 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.603 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.603 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.603 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.604 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.604 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.604 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.604 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-12T16:21:18.604 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-12T16:21:18.605 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.605 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.605 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.605 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.605 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.605 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.606 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.606 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.606 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.606 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.606 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.607 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.607 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-12T16:21:18.607 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-12T16:21:18.607 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.607 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.607 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.608 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.608 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.608 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.608 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.608 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.609 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.609 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.609 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.609 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.609 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-12T16:21:18.610 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-12T16:21:18.610 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.610 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.610 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.610 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.611 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.611 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.611 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.611 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.611 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.611 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.612 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.612 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.612 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-12T16:21:18.612 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-12T16:21:18.612 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.613 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.613 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.613 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.613 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.613 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.613 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.614 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.614 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.614 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.614 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.614 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.615 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-12T16:21:18.615 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-12T16:21:18.615 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.615 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.615 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.616 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.616 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.616 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.616 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.616 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.616 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.617 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.617 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.617 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.617 INFO:teuthology.orchestra.run.smithi130.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-12T16:21:18.617 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-12T16:21:18.618 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.618 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.618 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.618 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.618 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.619 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.619 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.619 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.619 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.619 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.619 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.620 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.620 INFO:teuthology.orchestra.run.smithi130.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-12T16:21:18.620 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-12T16:21:18.620 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.620 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.621 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.621 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.621 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.621 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.621 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.622 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.622 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.622 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.622 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.622 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.622 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_PASSWORD": { 2023-09-12T16:21:18.623 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-12T16:21:18.623 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.623 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.623 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.623 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "admin", 2023-09-12T16:21:18.624 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.624 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.624 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.624 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.624 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.624 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.625 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.625 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.625 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-12T16:21:18.625 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-12T16:21:18.625 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.626 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.626 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.626 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.626 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.626 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.627 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.627 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.627 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.627 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.627 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.627 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.628 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_URL": { 2023-09-12T16:21:18.628 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_URL", 2023-09-12T16:21:18.628 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.628 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.628 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.629 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.629 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.629 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.629 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.629 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.629 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.630 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.630 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.630 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.630 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_API_USERNAME": { 2023-09-12T16:21:18.630 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-12T16:21:18.631 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.631 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.631 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.631 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "admin", 2023-09-12T16:21:18.631 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.632 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.632 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.632 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.632 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.632 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.632 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.633 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.633 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-12T16:21:18.633 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-12T16:21:18.633 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.633 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.634 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.634 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.634 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.634 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.634 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.635 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.635 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.635 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.635 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.635 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.636 INFO:teuthology.orchestra.run.smithi130.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-12T16:21:18.636 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-12T16:21:18.636 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.636 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.636 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.637 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.637 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.637 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.637 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.637 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.637 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.638 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.638 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.638 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.638 INFO:teuthology.orchestra.run.smithi130.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-12T16:21:18.638 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-12T16:21:18.639 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.639 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.639 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.639 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.639 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.640 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.640 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.640 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.640 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.640 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.641 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.641 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.641 INFO:teuthology.orchestra.run.smithi130.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-12T16:21:18.641 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-12T16:21:18.641 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.642 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.642 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.642 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.642 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.642 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.643 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.643 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.643 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.643 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.643 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.643 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.644 INFO:teuthology.orchestra.run.smithi130.stdout: "PROMETHEUS_API_HOST": { 2023-09-12T16:21:18.644 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-12T16:21:18.644 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.644 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.644 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.645 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.645 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.645 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.645 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.645 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.645 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.646 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.646 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.646 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.646 INFO:teuthology.orchestra.run.smithi130.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-12T16:21:18.646 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-12T16:21:18.647 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.647 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.647 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.647 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.647 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.648 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.648 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.648 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.648 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.648 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.648 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.649 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.649 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-12T16:21:18.649 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-12T16:21:18.649 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.649 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.650 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.650 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.650 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.650 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.650 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.651 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.651 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.651 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.652 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.653 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.653 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-12T16:21:18.653 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-12T16:21:18.653 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.653 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.654 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.654 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.654 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.654 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.654 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.655 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.655 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.655 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.655 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.655 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.656 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-12T16:21:18.656 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-12T16:21:18.656 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.656 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.656 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.656 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.657 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.657 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.657 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.657 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.657 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.658 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.658 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.658 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.658 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-12T16:21:18.658 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-12T16:21:18.659 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.659 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.659 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.659 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.659 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.659 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.660 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.660 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.660 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.660 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.660 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.661 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.661 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-12T16:21:18.661 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-12T16:21:18.661 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.661 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.661 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.662 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.662 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.662 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.662 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.662 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.663 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.663 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.663 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.663 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.663 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-12T16:21:18.663 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-12T16:21:18.664 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.664 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.664 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.664 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.664 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.665 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.665 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.665 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.665 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.665 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.665 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.666 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.666 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-12T16:21:18.666 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-12T16:21:18.666 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.666 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.667 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.667 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.667 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.667 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.667 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.668 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.668 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.668 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.668 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.668 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.668 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_ENABLED": { 2023-09-12T16:21:18.669 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-12T16:21:18.669 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.669 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.669 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.669 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.670 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.670 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.670 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.670 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.670 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.671 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.671 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.671 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.671 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-12T16:21:18.671 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-12T16:21:18.671 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.672 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.672 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.672 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-12T16:21:18.672 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.672 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.672 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.673 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.673 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.673 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.673 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.673 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.674 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-12T16:21:18.674 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-12T16:21:18.674 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.674 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.674 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.675 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:18.675 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.675 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.675 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.675 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.675 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.676 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.676 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.676 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.676 INFO:teuthology.orchestra.run.smithi130.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-12T16:21:18.676 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-12T16:21:18.677 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.677 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.677 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.677 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8", 2023-09-12T16:21:18.677 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.677 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.678 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.678 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.678 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.678 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.678 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.679 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.679 INFO:teuthology.orchestra.run.smithi130.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-12T16:21:18.679 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-12T16:21:18.679 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.679 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.680 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.680 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "45", 2023-09-12T16:21:18.680 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.680 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.680 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.680 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.681 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.681 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.681 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.681 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.681 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_ACCESS_KEY": { 2023-09-12T16:21:18.682 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-12T16:21:18.682 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.682 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.682 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.682 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.683 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.683 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.683 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.683 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.683 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.684 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.684 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.684 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.684 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-12T16:21:18.684 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-12T16:21:18.685 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.685 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.685 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.685 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "admin", 2023-09-12T16:21:18.685 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.685 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.686 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.686 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.686 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.686 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.686 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.687 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.687 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_SECRET_KEY": { 2023-09-12T16:21:18.687 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-12T16:21:18.687 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.687 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.688 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.688 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.688 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.688 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.688 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.688 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.689 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.689 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.689 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.689 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.689 INFO:teuthology.orchestra.run.smithi130.stdout: "RGW_API_SSL_VERIFY": { 2023-09-12T16:21:18.690 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-12T16:21:18.690 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.690 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.690 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.690 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.691 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.691 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.691 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.691 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.691 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.692 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.692 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.692 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.692 INFO:teuthology.orchestra.run.smithi130.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-12T16:21:18.692 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-12T16:21:18.692 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.693 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.693 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.693 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:18.693 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.693 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.694 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.694 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.694 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.694 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.694 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.694 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.695 INFO:teuthology.orchestra.run.smithi130.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-12T16:21:18.695 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-12T16:21:18.695 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.695 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.695 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.696 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:18.696 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.696 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.696 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.696 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.696 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.697 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.697 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.697 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.697 INFO:teuthology.orchestra.run.smithi130.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-12T16:21:18.697 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-12T16:21:18.698 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.698 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.698 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.698 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:18.698 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.699 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.699 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.699 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.699 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.699 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.699 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.700 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.700 INFO:teuthology.orchestra.run.smithi130.stdout: "cross_origin_url": { 2023-09-12T16:21:18.700 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cross_origin_url", 2023-09-12T16:21:18.700 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.700 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.701 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.701 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.701 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.701 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.701 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.702 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.702 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.702 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.702 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.702 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.702 INFO:teuthology.orchestra.run.smithi130.stdout: "crt_file": { 2023-09-12T16:21:18.703 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "crt_file", 2023-09-12T16:21:18.703 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.703 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.703 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.703 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.704 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.704 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.704 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.704 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.704 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.705 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.705 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.705 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.705 INFO:teuthology.orchestra.run.smithi130.stdout: "debug": { 2023-09-12T16:21:18.705 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "debug", 2023-09-12T16:21:18.705 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.706 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.706 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.706 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.706 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.706 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.707 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.707 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Enable/disable debug options", 2023-09-12T16:21:18.707 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.707 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.707 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.708 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.708 INFO:teuthology.orchestra.run.smithi130.stdout: "jwt_token_ttl": { 2023-09-12T16:21:18.708 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "jwt_token_ttl", 2023-09-12T16:21:18.708 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.708 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.709 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.709 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "28800", 2023-09-12T16:21:18.709 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.709 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.709 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.710 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.710 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.710 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.710 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.710 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.711 INFO:teuthology.orchestra.run.smithi130.stdout: "key_file": { 2023-09-12T16:21:18.711 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "key_file", 2023-09-12T16:21:18.711 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.711 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.711 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.712 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.712 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.712 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.712 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.712 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.712 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.713 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.713 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.713 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.713 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.713 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.714 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.714 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.714 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.714 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.714 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.715 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.715 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.715 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.715 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.715 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.715 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.716 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.716 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.716 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.716 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.716 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.717 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.717 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.717 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.717 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.717 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.718 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.718 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.718 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.718 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.718 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.718 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.719 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.719 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.719 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.719 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.719 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.720 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.720 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.720 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.720 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.720 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.721 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.721 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.721 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.721 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.721 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.721 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.722 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.722 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.722 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.722 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.722 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.723 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.723 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.723 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.723 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.723 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.724 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.724 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.724 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.724 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.724 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.724 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.725 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.725 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.725 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.725 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.725 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.726 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.726 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.726 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.726 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.726 INFO:teuthology.orchestra.run.smithi130.stdout: "motd": { 2023-09-12T16:21:18.727 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "motd", 2023-09-12T16:21:18.727 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.727 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.727 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.727 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.727 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.728 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.728 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.728 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "The message of the day", 2023-09-12T16:21:18.728 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.728 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.729 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.729 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.729 INFO:teuthology.orchestra.run.smithi130.stdout: "redirect_resolve_ip_addr": { 2023-09-12T16:21:18.729 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "redirect_resolve_ip_addr", 2023-09-12T16:21:18.729 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.730 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.730 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.730 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.730 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.730 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.730 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.731 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.731 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.731 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.731 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.731 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.732 INFO:teuthology.orchestra.run.smithi130.stdout: "server_addr": { 2023-09-12T16:21:18.732 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_addr", 2023-09-12T16:21:18.732 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.732 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.732 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.733 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "::", 2023-09-12T16:21:18.733 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.733 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.733 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.733 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.733 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.734 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.734 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.734 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.734 INFO:teuthology.orchestra.run.smithi130.stdout: "server_port": { 2023-09-12T16:21:18.734 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_port", 2023-09-12T16:21:18.735 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.735 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.735 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.735 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8080", 2023-09-12T16:21:18.735 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.736 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.736 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.736 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.736 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.736 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.736 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.737 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.737 INFO:teuthology.orchestra.run.smithi130.stdout: "ssl": { 2023-09-12T16:21:18.737 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssl", 2023-09-12T16:21:18.737 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.737 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.738 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.738 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.738 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.738 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.738 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.738 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.739 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.739 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.739 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.739 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.739 INFO:teuthology.orchestra.run.smithi130.stdout: "ssl_server_port": { 2023-09-12T16:21:18.740 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssl_server_port", 2023-09-12T16:21:18.740 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.740 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.740 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.740 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8443", 2023-09-12T16:21:18.740 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.741 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.741 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.741 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.741 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.741 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.742 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.742 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.742 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_behaviour": { 2023-09-12T16:21:18.742 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_behaviour", 2023-09-12T16:21:18.742 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.743 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.743 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.743 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "redirect", 2023-09-12T16:21:18.743 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.743 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.743 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.744 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.744 INFO:teuthology.orchestra.run.smithi130.stdout: "redirect" 2023-09-12T16:21:18.744 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.744 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.744 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.745 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.745 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.745 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.745 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_error_status_code": { 2023-09-12T16:21:18.745 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_error_status_code", 2023-09-12T16:21:18.746 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.746 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.746 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.746 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "500", 2023-09-12T16:21:18.746 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "400", 2023-09-12T16:21:18.746 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "599", 2023-09-12T16:21:18.747 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.747 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.747 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.747 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.747 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.748 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.748 INFO:teuthology.orchestra.run.smithi130.stdout: "url_prefix": { 2023-09-12T16:21:18.748 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "url_prefix", 2023-09-12T16:21:18.748 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.748 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.749 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.749 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.749 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.749 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.749 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.750 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.750 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.750 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.750 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.750 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.751 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.751 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.751 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.751 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "devicehealth", 2023-09-12T16:21:18.751 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.751 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.752 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.752 INFO:teuthology.orchestra.run.smithi130.stdout: "enable_monitoring": { 2023-09-12T16:21:18.752 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enable_monitoring", 2023-09-12T16:21:18.752 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.752 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.753 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.753 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.753 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.753 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.753 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.753 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "monitor device health metrics", 2023-09-12T16:21:18.754 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.754 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.754 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.754 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.754 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.755 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.755 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.755 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.755 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.755 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.756 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.756 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.756 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.756 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.756 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.756 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.757 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.757 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.757 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.757 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.757 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.758 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.758 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.758 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.758 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.758 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.758 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.759 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.759 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.759 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.759 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.759 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.760 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.760 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.760 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.760 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.760 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.761 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.761 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.761 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.761 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.761 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.761 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.762 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.762 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.762 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.762 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.762 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.763 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.763 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.763 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.763 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.763 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.764 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.764 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.764 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.764 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.764 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.765 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.765 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.765 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.765 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.765 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.765 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.766 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.766 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.766 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.766 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.766 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.767 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.767 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.767 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.767 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.767 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.767 INFO:teuthology.orchestra.run.smithi130.stdout: "mark_out_threshold": { 2023-09-12T16:21:18.768 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mark_out_threshold", 2023-09-12T16:21:18.768 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.768 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.768 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.768 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "2419200", 2023-09-12T16:21:18.769 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.769 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.769 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.769 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-12T16:21:18.769 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.770 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.770 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.770 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.770 INFO:teuthology.orchestra.run.smithi130.stdout: "pool_name": { 2023-09-12T16:21:18.770 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pool_name", 2023-09-12T16:21:18.770 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.771 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.771 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.771 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "device_health_metrics", 2023-09-12T16:21:18.771 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.771 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.772 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.772 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-12T16:21:18.772 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.772 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.772 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.772 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.773 INFO:teuthology.orchestra.run.smithi130.stdout: "retention_period": { 2023-09-12T16:21:18.773 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "retention_period", 2023-09-12T16:21:18.773 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.773 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.773 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.774 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "15552000", 2023-09-12T16:21:18.774 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.774 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.774 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.774 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how long to retain device health metrics", 2023-09-12T16:21:18.775 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.775 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.775 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.775 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.775 INFO:teuthology.orchestra.run.smithi130.stdout: "scrape_frequency": { 2023-09-12T16:21:18.775 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "scrape_frequency", 2023-09-12T16:21:18.776 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.776 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.776 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.776 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "86400", 2023-09-12T16:21:18.776 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.777 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.777 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.777 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-12T16:21:18.777 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.777 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.778 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.778 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.778 INFO:teuthology.orchestra.run.smithi130.stdout: "self_heal": { 2023-09-12T16:21:18.778 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "self_heal", 2023-09-12T16:21:18.778 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.779 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.779 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.779 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:18.779 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.779 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.780 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.780 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-12T16:21:18.780 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.780 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.780 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.781 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.781 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:18.781 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:18.781 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.781 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.781 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.782 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:18.782 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.782 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.782 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.782 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how frequently to wake up and check device health", 2023-09-12T16:21:18.783 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.783 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.783 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.783 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.783 INFO:teuthology.orchestra.run.smithi130.stdout: "warn_threshold": { 2023-09-12T16:21:18.784 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "warn_threshold", 2023-09-12T16:21:18.784 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.784 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.784 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.784 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "7257600", 2023-09-12T16:21:18.784 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.785 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.785 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.785 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-12T16:21:18.785 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.785 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.786 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.786 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.786 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.786 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.786 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.787 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "diskprediction_local", 2023-09-12T16:21:18.787 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.787 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.787 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.787 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.788 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.788 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.788 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.788 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.788 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.788 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.789 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.789 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.789 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.789 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.789 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.790 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.790 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.790 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.790 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.790 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.790 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.791 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.791 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.791 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.791 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.791 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.792 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.792 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.792 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.792 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.792 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.792 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.793 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.793 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.793 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.793 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.793 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.794 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.794 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.794 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.794 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.794 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.795 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.795 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.795 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.795 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.795 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.795 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.796 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.796 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.796 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.796 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.796 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.797 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.797 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.797 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.797 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.797 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.798 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.798 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.798 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.798 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.798 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.799 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.799 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.799 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.799 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.799 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.799 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.800 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.800 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.800 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.800 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.800 INFO:teuthology.orchestra.run.smithi130.stdout: "predict_interval": { 2023-09-12T16:21:18.801 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "predict_interval", 2023-09-12T16:21:18.801 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.801 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.801 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.801 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "86400", 2023-09-12T16:21:18.802 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.802 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.802 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.802 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.802 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.803 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.803 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.803 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.803 INFO:teuthology.orchestra.run.smithi130.stdout: "predictor_model": { 2023-09-12T16:21:18.803 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "predictor_model", 2023-09-12T16:21:18.804 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.804 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.804 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.804 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "prophetstor", 2023-09-12T16:21:18.804 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.804 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.805 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.805 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.805 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.805 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.805 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.806 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.806 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:18.806 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:18.806 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.806 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.807 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.807 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "600", 2023-09-12T16:21:18.807 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.807 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.807 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.807 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.808 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.808 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.808 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.808 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.808 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.809 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.809 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.809 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "influx", 2023-09-12T16:21:18.809 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": false, 2023-09-12T16:21:18.809 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "influxdb python module not found", 2023-09-12T16:21:18.809 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.810 INFO:teuthology.orchestra.run.smithi130.stdout: "batch_size": { 2023-09-12T16:21:18.810 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "batch_size", 2023-09-12T16:21:18.810 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.810 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.810 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.811 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5000", 2023-09-12T16:21:18.811 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.811 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.811 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.811 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-12T16:21:18.811 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.812 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.812 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.812 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.812 INFO:teuthology.orchestra.run.smithi130.stdout: "database": { 2023-09-12T16:21:18.812 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "database", 2023-09-12T16:21:18.813 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.813 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.813 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.813 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "ceph", 2023-09-12T16:21:18.813 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.814 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.814 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.814 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-12T16:21:18.814 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.814 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.814 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.815 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.815 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": { 2023-09-12T16:21:18.815 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "hostname", 2023-09-12T16:21:18.815 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.815 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.816 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.816 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.816 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.816 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.816 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.817 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "InfluxDB server hostname", 2023-09-12T16:21:18.817 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.817 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.817 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.817 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.817 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:18.818 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:18.818 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:18.818 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.818 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.818 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "30", 2023-09-12T16:21:18.819 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "5", 2023-09-12T16:21:18.819 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.819 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.819 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-12T16:21:18.820 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.820 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.820 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.820 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.821 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.821 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.821 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.821 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.821 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.822 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.822 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.822 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.822 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.822 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.822 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.823 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.823 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.823 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.823 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.823 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.824 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.824 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.824 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.824 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.824 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.824 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.825 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.825 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.825 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.825 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.825 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.826 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.826 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.826 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.826 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.826 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.827 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.827 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.827 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.827 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.827 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.828 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.828 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.828 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.828 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.828 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.829 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.829 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.829 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.829 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.829 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.830 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.830 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.830 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.830 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.830 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.830 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.831 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.831 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.831 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.831 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.831 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.832 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.832 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.832 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.832 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.832 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.833 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.833 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.833 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.833 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.833 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.834 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.834 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.834 INFO:teuthology.orchestra.run.smithi130.stdout: "password": { 2023-09-12T16:21:18.834 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "password", 2023-09-12T16:21:18.835 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.835 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.835 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.835 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.835 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.836 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.836 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.836 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "password of InfluxDB server user", 2023-09-12T16:21:18.836 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.836 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.837 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.837 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.837 INFO:teuthology.orchestra.run.smithi130.stdout: "port": { 2023-09-12T16:21:18.837 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "port", 2023-09-12T16:21:18.837 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.838 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.838 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.838 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "8086", 2023-09-12T16:21:18.838 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.838 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.839 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.839 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "InfluxDB server port", 2023-09-12T16:21:18.839 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.839 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.839 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.840 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.840 INFO:teuthology.orchestra.run.smithi130.stdout: "ssl": { 2023-09-12T16:21:18.840 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ssl", 2023-09-12T16:21:18.840 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.840 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.840 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.841 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "false", 2023-09-12T16:21:18.841 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.841 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.841 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.841 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-12T16:21:18.842 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.842 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.842 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.842 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.842 INFO:teuthology.orchestra.run.smithi130.stdout: "threads": { 2023-09-12T16:21:18.843 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "threads", 2023-09-12T16:21:18.843 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.843 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.843 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.843 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:18.843 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:18.844 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "32", 2023-09-12T16:21:18.844 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.844 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-12T16:21:18.844 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.844 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.845 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.845 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.845 INFO:teuthology.orchestra.run.smithi130.stdout: "username": { 2023-09-12T16:21:18.845 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "username", 2023-09-12T16:21:18.845 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.845 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.846 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.846 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.846 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.846 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.846 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.847 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "username of InfluxDB server user", 2023-09-12T16:21:18.847 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.847 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.847 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.847 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.847 INFO:teuthology.orchestra.run.smithi130.stdout: "verify_ssl": { 2023-09-12T16:21:18.848 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "verify_ssl", 2023-09-12T16:21:18.848 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.848 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.848 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.848 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "true", 2023-09-12T16:21:18.849 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.849 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.849 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.849 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-12T16:21:18.849 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.849 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.850 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.850 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.850 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.850 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.850 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.851 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "insights", 2023-09-12T16:21:18.851 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.851 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.851 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.851 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.852 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.852 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.852 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.852 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.852 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.853 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.853 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.853 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.853 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.853 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.853 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.854 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.854 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.854 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.854 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.854 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.855 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.855 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.856 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.857 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.857 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.857 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.857 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.858 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.858 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.858 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.858 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.858 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.859 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.859 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.859 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.859 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.859 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.860 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.860 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.860 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.860 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.860 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.861 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.861 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.861 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.861 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.861 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.861 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.862 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.862 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.862 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.862 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.862 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.863 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.863 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.863 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.863 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.863 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.864 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.864 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.864 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.864 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.864 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.864 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.865 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.865 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.865 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.865 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.865 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.866 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.866 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.866 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.866 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.866 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.867 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.867 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.867 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "iostat", 2023-09-12T16:21:18.867 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.867 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.867 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.868 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.868 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.868 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.868 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.868 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.869 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.869 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.869 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.869 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.869 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.869 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.870 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.870 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.870 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.870 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.870 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.871 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.871 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.871 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.871 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.871 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.872 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.872 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.872 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.872 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.872 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.872 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.873 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.873 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.873 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.873 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.873 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.874 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.874 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.874 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.874 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.874 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.875 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.875 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.875 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.875 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.875 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.875 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.876 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.876 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.876 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.876 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.876 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.877 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.877 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.877 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.877 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.877 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.877 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.878 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.878 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.878 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.878 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.878 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.879 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.879 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.879 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.879 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.879 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.880 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.880 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.880 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.880 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.880 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.881 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.881 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.881 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.881 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.881 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "k8sevents", 2023-09-12T16:21:18.881 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.882 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.882 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.882 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_event_retention_days": { 2023-09-12T16:21:18.882 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "ceph_event_retention_days", 2023-09-12T16:21:18.882 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.883 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.883 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.883 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "7", 2023-09-12T16:21:18.883 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.883 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.884 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.884 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-12T16:21:18.884 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.884 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.884 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.884 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.885 INFO:teuthology.orchestra.run.smithi130.stdout: "config_check_secs": { 2023-09-12T16:21:18.885 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "config_check_secs", 2023-09-12T16:21:18.885 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.885 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.885 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.886 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:18.886 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "10", 2023-09-12T16:21:18.886 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.886 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.886 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-12T16:21:18.887 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.887 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.887 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.887 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.887 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.888 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.888 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.888 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.888 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.888 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.888 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.889 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.889 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.889 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.889 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.889 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.890 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.890 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.890 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.890 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.890 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.891 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.891 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.891 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.891 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.891 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.892 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.892 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.892 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.892 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.892 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.892 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.893 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.893 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.893 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.893 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.893 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.894 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.894 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.894 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.894 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.894 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.894 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.895 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.895 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.895 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.895 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.895 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.896 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.896 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.896 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.896 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.896 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.897 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.897 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.897 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.897 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.897 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.898 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.898 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.898 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.898 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.898 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.898 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.899 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.899 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.899 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.899 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.900 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.900 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.900 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.900 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.900 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.900 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.901 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.901 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.901 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.901 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "localpool", 2023-09-12T16:21:18.901 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.902 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.902 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.902 INFO:teuthology.orchestra.run.smithi130.stdout: "failure_domain": { 2023-09-12T16:21:18.902 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "failure_domain", 2023-09-12T16:21:18.902 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.903 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.903 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.903 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "host", 2023-09-12T16:21:18.903 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.903 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.904 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.904 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "failure domain for any created local pool", 2023-09-12T16:21:18.904 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-12T16:21:18.904 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.904 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.905 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.905 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.905 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.905 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.905 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.906 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.906 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.906 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.906 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.906 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.907 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.907 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.907 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.907 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.907 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.907 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.908 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.908 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.908 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.908 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.908 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.909 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.909 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.909 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.909 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.909 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.910 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.910 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.910 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.910 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.910 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.911 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.911 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.911 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.911 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.911 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.911 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.912 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.912 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.912 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.912 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.912 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.913 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.913 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.913 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.913 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.913 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.913 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.914 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.914 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.914 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.914 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.914 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.915 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.915 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.915 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.915 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.915 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.915 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.916 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.916 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.916 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.916 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.916 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.917 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.917 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.917 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.917 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.917 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.918 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.918 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.918 INFO:teuthology.orchestra.run.smithi130.stdout: "min_size": { 2023-09-12T16:21:18.918 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "min_size", 2023-09-12T16:21:18.918 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.918 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.919 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.919 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.919 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.919 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.919 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.920 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "default min_size for any created local pool", 2023-09-12T16:21:18.920 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-12T16:21:18.920 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.920 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.920 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.921 INFO:teuthology.orchestra.run.smithi130.stdout: "num_rep": { 2023-09-12T16:21:18.921 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "num_rep", 2023-09-12T16:21:18.921 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.921 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.921 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.921 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "3", 2023-09-12T16:21:18.922 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.922 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.922 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.922 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "default replica count for any created local pool", 2023-09-12T16:21:18.922 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.923 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.923 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.923 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.923 INFO:teuthology.orchestra.run.smithi130.stdout: "pg_num": { 2023-09-12T16:21:18.924 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pg_num", 2023-09-12T16:21:18.924 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:18.924 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.924 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.924 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "128", 2023-09-12T16:21:18.925 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.925 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.925 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.925 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "default pg_num for any created local pool", 2023-09-12T16:21:18.925 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.925 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.926 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.926 INFO:teuthology.orchestra.run.smithi130.stdout: "prefix": { 2023-09-12T16:21:18.926 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "prefix", 2023-09-12T16:21:18.926 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.927 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.927 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.927 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.927 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.927 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.927 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.928 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "name prefix for any created local pool", 2023-09-12T16:21:18.928 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.928 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.928 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.928 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.928 INFO:teuthology.orchestra.run.smithi130.stdout: "subtree": { 2023-09-12T16:21:18.929 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "subtree", 2023-09-12T16:21:18.929 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.929 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.929 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.929 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "rack", 2023-09-12T16:21:18.930 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.930 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.930 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.930 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-12T16:21:18.930 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-12T16:21:18.930 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.931 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.931 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.931 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.931 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.931 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.932 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mds_autoscaler", 2023-09-12T16:21:18.932 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.932 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.932 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.932 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.933 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.933 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.933 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.933 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.933 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.933 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.934 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.934 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.934 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.934 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.934 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.935 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.935 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.935 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.935 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.935 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.935 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.936 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.936 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.936 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.936 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.936 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.936 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.937 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.937 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.937 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.937 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.937 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.938 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.938 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.938 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.938 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.938 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.939 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.939 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.939 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.939 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.939 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.939 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.940 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.940 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.940 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.940 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.940 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.941 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.941 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.941 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.941 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.941 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.941 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.942 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.942 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.942 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.942 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.942 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.943 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.943 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.943 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.943 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.943 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.943 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.944 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.944 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.944 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.944 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.944 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.945 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.945 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.945 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.945 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.945 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.945 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.946 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mirroring", 2023-09-12T16:21:18.946 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.946 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.946 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.946 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.947 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.947 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.947 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.947 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.947 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.947 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.948 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.948 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.948 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.948 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.948 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.949 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.949 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.949 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.949 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.949 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.950 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.950 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.950 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.950 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.950 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.950 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.951 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.951 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.951 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.951 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.951 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.952 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.952 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.952 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.952 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.952 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.952 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.953 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.953 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.953 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.953 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.953 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.954 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.954 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.954 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.954 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.954 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.954 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.955 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.955 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.955 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.955 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.955 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.955 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.956 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.956 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.956 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.956 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.956 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.956 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.957 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.957 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.957 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.957 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.957 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.957 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.958 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.958 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.958 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.958 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.958 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.958 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.959 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.959 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.959 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.959 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.959 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "nfs", 2023-09-12T16:21:18.959 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.960 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.960 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.960 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.960 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.960 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.960 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.961 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.961 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.961 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.961 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.961 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.962 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.962 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.962 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.962 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.962 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.962 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.963 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.963 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.963 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.963 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.963 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.963 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.964 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.964 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.964 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.964 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.964 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.964 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.965 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.965 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.965 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.965 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.965 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.966 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.966 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.966 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.966 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.966 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.966 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.967 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.967 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.967 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.967 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.967 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.967 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.968 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.968 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.968 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.968 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.968 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.968 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.969 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.969 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.969 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.969 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.969 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.969 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.970 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.970 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.970 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.970 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.970 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.971 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.971 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.971 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.971 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.971 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.971 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.972 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.972 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.972 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.972 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.972 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.973 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.973 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "orchestrator", 2023-09-12T16:21:18.973 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.973 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.973 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.973 INFO:teuthology.orchestra.run.smithi130.stdout: "fail_fs": { 2023-09-12T16:21:18.974 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "fail_fs", 2023-09-12T16:21:18.974 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.974 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.974 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:18.974 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.975 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.975 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.975 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.975 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-12T16:21:18.975 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.975 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.976 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.976 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.976 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.976 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.976 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.976 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.977 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.977 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.977 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.977 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.977 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.978 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.978 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.978 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.978 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.978 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.979 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.979 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.979 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.979 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.979 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.979 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.980 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.980 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.980 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.980 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.980 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.980 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.981 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.981 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.981 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.981 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.981 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.981 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.982 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.982 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.982 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.982 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.982 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.983 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.983 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.983 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.983 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:18.983 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.983 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.984 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.984 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.984 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.984 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.984 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.984 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.985 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.985 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.985 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.985 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.985 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.985 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.986 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.986 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:18.986 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:18.986 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.986 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.987 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.987 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.987 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.987 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.987 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.987 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.988 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.988 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.988 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.988 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.988 INFO:teuthology.orchestra.run.smithi130.stdout: "orchestrator": { 2023-09-12T16:21:18.988 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "orchestrator", 2023-09-12T16:21:18.989 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.989 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.989 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.989 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.989 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.989 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.990 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.990 INFO:teuthology.orchestra.run.smithi130.stdout: "cephadm", 2023-09-12T16:21:18.990 INFO:teuthology.orchestra.run.smithi130.stdout: "rook", 2023-09-12T16:21:18.990 INFO:teuthology.orchestra.run.smithi130.stdout: "test_orchestrator" 2023-09-12T16:21:18.990 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.991 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Orchestrator backend", 2023-09-12T16:21:18.991 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.991 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.991 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.991 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.991 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:18.992 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.992 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:18.992 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "osd_perf_query", 2023-09-12T16:21:18.992 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:18.992 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:18.992 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:18.993 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:18.993 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:18.993 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:18.993 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.993 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.993 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:18.994 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.994 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.994 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:18.994 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:18.994 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:18.995 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:18.995 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:18.995 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:18.995 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:18.995 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:18.996 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.996 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.996 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.996 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.996 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.996 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:18.997 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:18.997 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:18.997 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:18.997 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:18.997 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:18.997 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:18.998 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:18.998 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:18.998 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:18.998 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:18.998 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:18.999 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:18.999 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:18.999 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:18.999 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:18.999 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.000 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.000 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.000 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.000 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.000 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.001 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.001 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.001 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.001 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.001 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.002 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.002 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.002 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.002 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.002 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.003 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.003 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.003 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.003 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.003 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.003 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.004 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.004 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.004 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.004 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.004 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.005 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.005 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.005 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.005 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.005 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.006 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.006 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.006 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.006 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.006 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "osd_support", 2023-09-12T16:21:19.007 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.007 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.007 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.007 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.007 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.007 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.008 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.008 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.008 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.008 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.008 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.008 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.009 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.009 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.009 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.009 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.009 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.010 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.010 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.010 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.010 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.010 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.011 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.011 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.011 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.011 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.011 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.012 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.012 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.012 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.012 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.012 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.012 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.013 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.013 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.013 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.013 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.013 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.013 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.014 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.014 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.014 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.014 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.014 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.015 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.015 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.015 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.015 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.015 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.015 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.016 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.016 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.016 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.016 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.016 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.016 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.017 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.017 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.017 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.017 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.017 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.018 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.018 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.018 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.018 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.018 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.019 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.019 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.019 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.019 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.019 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.020 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.020 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.020 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.020 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.020 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.020 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "pg_autoscaler", 2023-09-12T16:21:19.021 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.021 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.021 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.021 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.021 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.022 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.022 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.022 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.022 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.022 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.023 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.023 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.023 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.023 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.023 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.024 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.024 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.024 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.024 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.024 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.024 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.025 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.025 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.025 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.025 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.025 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.025 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.026 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.026 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.026 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.026 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.026 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.027 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.027 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.027 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.027 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.027 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.027 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.028 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.028 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.028 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.028 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.028 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.028 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.029 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.029 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.029 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.029 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.029 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.029 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.030 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.030 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.030 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.030 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.030 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.031 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.031 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.031 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.031 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.031 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.031 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.032 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.032 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.032 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.032 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.032 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.033 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.033 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.033 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.033 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.033 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.033 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.034 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.034 INFO:teuthology.orchestra.run.smithi130.stdout: "noautoscale": { 2023-09-12T16:21:19.034 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "noautoscale", 2023-09-12T16:21:19.034 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.034 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.035 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.035 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.035 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.035 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.035 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.035 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "global autoscale flag", 2023-09-12T16:21:19.036 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-12T16:21:19.036 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.036 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.036 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.036 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:19.036 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:19.037 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:19.037 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.037 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.037 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:19.037 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.037 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.038 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.038 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.038 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.038 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.038 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.039 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.039 INFO:teuthology.orchestra.run.smithi130.stdout: "threshold": { 2023-09-12T16:21:19.039 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "threshold", 2023-09-12T16:21:19.039 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:19.039 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.039 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.040 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "3.0", 2023-09-12T16:21:19.040 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1.0", 2023-09-12T16:21:19.040 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.040 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.040 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "scaling threshold", 2023-09-12T16:21:19.041 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-12T16:21:19.041 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.041 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.041 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.041 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.042 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.042 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.042 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "progress", 2023-09-12T16:21:19.042 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.042 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.042 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.043 INFO:teuthology.orchestra.run.smithi130.stdout: "allow_pg_recovery_event": { 2023-09-12T16:21:19.043 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "allow_pg_recovery_event", 2023-09-12T16:21:19.043 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.043 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.043 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.043 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.044 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.044 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.044 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.044 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-12T16:21:19.044 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.045 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.045 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.045 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.045 INFO:teuthology.orchestra.run.smithi130.stdout: "enabled": { 2023-09-12T16:21:19.045 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enabled", 2023-09-12T16:21:19.046 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.046 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.046 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.046 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:19.046 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.046 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.047 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.047 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.047 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.047 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.047 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.048 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.048 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.048 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.048 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.048 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.048 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.049 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.049 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.049 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.049 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.049 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.050 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.050 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.050 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.050 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.050 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.050 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.051 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.051 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.051 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.051 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.051 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.052 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.052 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.052 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.052 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.052 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.053 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.053 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.053 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.053 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.053 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.053 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.054 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.054 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.054 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.054 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.054 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.055 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.055 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.055 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.055 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.055 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.055 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.056 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.056 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.056 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.056 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.056 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.056 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.057 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.057 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.057 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.057 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.057 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.057 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.058 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.058 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.058 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.058 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.058 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.059 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.059 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.059 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.059 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.059 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.060 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.060 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.060 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.060 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.060 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.061 INFO:teuthology.orchestra.run.smithi130.stdout: "max_completed_events": { 2023-09-12T16:21:19.062 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_completed_events", 2023-09-12T16:21:19.062 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.062 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.062 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.062 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "50", 2023-09-12T16:21:19.063 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.063 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.063 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.063 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "number of past completed events to remember", 2023-09-12T16:21:19.063 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.064 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.064 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.064 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.064 INFO:teuthology.orchestra.run.smithi130.stdout: "sleep_interval": { 2023-09-12T16:21:19.064 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "sleep_interval", 2023-09-12T16:21:19.064 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:19.065 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.065 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.065 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "5", 2023-09-12T16:21:19.065 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.065 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.065 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.066 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "how long the module is going to sleep", 2023-09-12T16:21:19.066 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.066 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.066 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.066 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.067 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.067 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.067 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.067 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "prometheus", 2023-09-12T16:21:19.067 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.068 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.068 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.068 INFO:teuthology.orchestra.run.smithi130.stdout: "cache": { 2023-09-12T16:21:19.068 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "cache", 2023-09-12T16:21:19.068 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.069 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.069 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.069 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:19.069 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.069 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.069 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.070 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.070 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.070 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.070 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.070 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.071 INFO:teuthology.orchestra.run.smithi130.stdout: "exclude_perf_counters": { 2023-09-12T16:21:19.071 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "exclude_perf_counters", 2023-09-12T16:21:19.071 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.071 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.071 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.072 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:19.072 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.072 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.072 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.072 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-12T16:21:19.072 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-12T16:21:19.073 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.073 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.073 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.073 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.073 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.074 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.074 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.074 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.074 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.074 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.075 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.075 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.075 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.075 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.075 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.075 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.076 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.076 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.076 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.076 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.076 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.077 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.077 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.077 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.077 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.077 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.077 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.078 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.078 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.078 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.078 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.078 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.079 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.079 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.079 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.079 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.079 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.079 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.080 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.080 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.080 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.080 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.080 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.080 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.081 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.081 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.081 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.081 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.081 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.081 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.082 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.082 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.082 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.082 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.082 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.083 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.083 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.083 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.083 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.083 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.083 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.084 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.084 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.084 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.084 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.084 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.084 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.085 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.085 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.085 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.085 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.085 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.086 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.086 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_stats_pools": { 2023-09-12T16:21:19.086 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_stats_pools", 2023-09-12T16:21:19.086 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.086 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.087 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.087 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.087 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.087 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.087 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.087 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.088 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.088 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.088 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.088 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.088 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-12T16:21:19.088 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-12T16:21:19.089 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.089 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.089 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.089 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "300", 2023-09-12T16:21:19.089 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.089 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.090 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.090 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.090 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.090 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.090 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.090 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.091 INFO:teuthology.orchestra.run.smithi130.stdout: "scrape_interval": { 2023-09-12T16:21:19.091 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "scrape_interval", 2023-09-12T16:21:19.091 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:19.091 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.091 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.092 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "15.0", 2023-09-12T16:21:19.092 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.092 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.092 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.092 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.092 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.093 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.093 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.093 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.093 INFO:teuthology.orchestra.run.smithi130.stdout: "server_addr": { 2023-09-12T16:21:19.093 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_addr", 2023-09-12T16:21:19.093 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.094 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.094 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.094 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "::", 2023-09-12T16:21:19.094 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.094 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.095 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.095 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-12T16:21:19.095 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.095 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.095 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.096 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.096 INFO:teuthology.orchestra.run.smithi130.stdout: "server_port": { 2023-09-12T16:21:19.096 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_port", 2023-09-12T16:21:19.096 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.096 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.096 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.097 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "9283", 2023-09-12T16:21:19.097 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.097 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.097 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.097 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-12T16:21:19.097 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.098 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.098 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.098 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.098 INFO:teuthology.orchestra.run.smithi130.stdout: "stale_cache_strategy": { 2023-09-12T16:21:19.098 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "stale_cache_strategy", 2023-09-12T16:21:19.099 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.099 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.099 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.099 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "log", 2023-09-12T16:21:19.099 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.099 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.100 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.100 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.100 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.100 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.100 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.100 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.101 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_behaviour": { 2023-09-12T16:21:19.101 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_behaviour", 2023-09-12T16:21:19.101 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.101 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.101 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.101 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "default", 2023-09-12T16:21:19.102 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.102 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.102 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.102 INFO:teuthology.orchestra.run.smithi130.stdout: "default", 2023-09-12T16:21:19.102 INFO:teuthology.orchestra.run.smithi130.stdout: "error" 2023-09-12T16:21:19.103 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.103 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.103 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.103 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.103 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.103 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.104 INFO:teuthology.orchestra.run.smithi130.stdout: "standby_error_status_code": { 2023-09-12T16:21:19.104 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "standby_error_status_code", 2023-09-12T16:21:19.104 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.104 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.104 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.104 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "500", 2023-09-12T16:21:19.105 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "400", 2023-09-12T16:21:19.105 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "599", 2023-09-12T16:21:19.105 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.105 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.105 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.106 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.106 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.106 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.106 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.106 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.106 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.107 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_support", 2023-09-12T16:21:19.107 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.107 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.107 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.107 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.107 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.108 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.108 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.108 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.108 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.108 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.109 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.109 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.109 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.109 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.109 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.109 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.110 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.110 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.110 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.110 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.110 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.111 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.111 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.111 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.111 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.111 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.111 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.112 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.112 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.112 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.112 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.112 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.112 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.113 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.113 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.113 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.113 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.113 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.114 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.114 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.114 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.114 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.114 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.114 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.115 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.115 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.115 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.115 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.115 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.116 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.116 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.116 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.116 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.116 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.116 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.117 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.117 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.117 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.117 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.117 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.117 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.118 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.118 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.118 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.118 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.118 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.119 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.119 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.119 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.119 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.119 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.119 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.120 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.120 INFO:teuthology.orchestra.run.smithi130.stdout: "max_concurrent_snap_create": { 2023-09-12T16:21:19.120 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_concurrent_snap_create", 2023-09-12T16:21:19.120 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.120 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.120 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.121 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10", 2023-09-12T16:21:19.121 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.121 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.121 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.121 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.121 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.122 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.122 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.122 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.122 INFO:teuthology.orchestra.run.smithi130.stdout: "mirror_snapshot_schedule": { 2023-09-12T16:21:19.122 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "mirror_snapshot_schedule", 2023-09-12T16:21:19.123 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.123 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.123 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.123 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.123 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.123 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.124 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.124 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.124 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.124 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.124 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.124 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.125 INFO:teuthology.orchestra.run.smithi130.stdout: "trash_purge_schedule": { 2023-09-12T16:21:19.125 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "trash_purge_schedule", 2023-09-12T16:21:19.125 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.125 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.125 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.125 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.126 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.126 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.126 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.126 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.126 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.127 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.127 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.127 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.127 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.127 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.127 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.128 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "restful", 2023-09-12T16:21:19.128 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.128 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.128 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.128 INFO:teuthology.orchestra.run.smithi130.stdout: "enable_auth": { 2023-09-12T16:21:19.128 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enable_auth", 2023-09-12T16:21:19.129 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.129 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.129 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.129 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:19.129 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.130 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.130 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.130 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.130 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.130 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.131 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.131 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.131 INFO:teuthology.orchestra.run.smithi130.stdout: "key_file": { 2023-09-12T16:21:19.131 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "key_file", 2023-09-12T16:21:19.131 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.132 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.132 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.132 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.132 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.132 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.132 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.133 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.133 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.133 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.133 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.133 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.134 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.134 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.134 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.134 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.134 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.135 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.135 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.135 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.135 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.135 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.135 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.136 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.136 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.136 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.136 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.136 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.136 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.137 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.137 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.137 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.137 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.137 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.138 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.138 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.138 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.138 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.138 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.138 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.139 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.139 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.139 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.139 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.139 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.140 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.140 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.140 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.140 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.140 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.140 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.141 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.141 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.141 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.141 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.141 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.141 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.142 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.142 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.142 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.142 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.142 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.143 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.143 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.143 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.143 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.143 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.143 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.144 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.144 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.144 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.144 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.144 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.144 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.145 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.145 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.145 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.145 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.145 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.145 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.146 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.146 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.146 INFO:teuthology.orchestra.run.smithi130.stdout: "server_addr": { 2023-09-12T16:21:19.146 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_addr", 2023-09-12T16:21:19.146 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.147 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.147 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.147 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.147 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.147 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.147 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.148 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.148 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.148 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.148 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.148 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.149 INFO:teuthology.orchestra.run.smithi130.stdout: "server_port": { 2023-09-12T16:21:19.149 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "server_port", 2023-09-12T16:21:19.149 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.149 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.149 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.149 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.150 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.150 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.150 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.150 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.150 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.150 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.151 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.151 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.151 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.151 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.151 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.151 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rgw", 2023-09-12T16:21:19.152 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.152 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.152 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.152 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.152 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.152 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.153 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.153 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.153 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.153 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.153 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.153 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.154 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.154 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.154 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.154 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.154 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.155 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.155 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.155 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.155 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.155 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.155 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.156 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.156 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.156 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.156 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.156 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.156 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.157 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.157 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.157 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.157 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.157 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.158 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.158 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.158 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.158 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.158 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.158 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.159 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.159 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.159 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.159 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.159 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.160 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.160 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.160 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.160 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.160 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.161 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.161 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.161 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.161 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.161 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.161 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.162 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.162 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.162 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.162 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.162 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.163 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.163 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.163 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.163 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.163 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.163 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.164 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.164 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.164 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.164 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.164 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.164 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.165 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.165 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.165 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.165 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rook", 2023-09-12T16:21:19.165 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.165 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.166 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.166 INFO:teuthology.orchestra.run.smithi130.stdout: "drive_group_interval": { 2023-09-12T16:21:19.166 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "drive_group_interval", 2023-09-12T16:21:19.166 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:19.166 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.167 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.167 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "300.0", 2023-09-12T16:21:19.167 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.167 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.167 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.168 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-12T16:21:19.168 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.168 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.168 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.168 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.168 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.169 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.169 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.169 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.169 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.169 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.170 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.170 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.170 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.170 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.170 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.171 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.171 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.171 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.171 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.171 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.171 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.172 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.172 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.172 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.172 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.172 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.173 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.173 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.173 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.173 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.173 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.173 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.174 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.174 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.174 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.174 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.174 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.175 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.175 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.175 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.175 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.175 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.175 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.176 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.176 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.176 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.176 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.176 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.176 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.177 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.177 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.177 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.177 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.177 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.178 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.178 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.178 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.178 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.178 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.179 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.179 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.179 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.179 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.179 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.179 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.180 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.180 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.180 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.180 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.180 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.180 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.181 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.181 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.181 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.181 INFO:teuthology.orchestra.run.smithi130.stdout: "storage_class": { 2023-09-12T16:21:19.181 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "storage_class", 2023-09-12T16:21:19.182 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.182 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.182 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.182 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "local", 2023-09-12T16:21:19.182 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.182 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.183 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.183 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-12T16:21:19.183 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.183 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.183 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.183 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.184 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.184 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.184 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.184 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "selftest", 2023-09-12T16:21:19.184 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.184 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.185 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.185 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.185 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.185 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.185 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.185 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.186 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.186 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.186 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.186 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.186 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.187 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.187 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.187 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.187 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.187 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.187 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.188 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.188 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.188 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.188 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.188 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.189 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.189 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.189 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.189 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.189 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.189 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.190 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.190 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.190 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.190 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.190 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.190 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.191 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.191 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.191 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.191 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.191 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.192 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.192 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.192 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.192 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.192 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.192 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.193 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.193 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.193 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.193 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.193 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.193 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.193 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.194 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.194 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.194 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.194 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.194 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.195 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.195 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.195 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.195 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.195 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.195 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.196 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.196 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.196 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.196 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.196 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.196 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.197 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.197 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.197 INFO:teuthology.orchestra.run.smithi130.stdout: "roption1": { 2023-09-12T16:21:19.197 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "roption1", 2023-09-12T16:21:19.197 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.197 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.198 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.198 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.198 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.198 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.198 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.199 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.199 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.199 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.199 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.199 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.199 INFO:teuthology.orchestra.run.smithi130.stdout: "roption2": { 2023-09-12T16:21:19.200 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "roption2", 2023-09-12T16:21:19.200 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.200 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.200 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.200 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "xyz", 2023-09-12T16:21:19.200 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.201 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.201 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.201 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.201 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.201 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.202 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.202 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.202 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption1": { 2023-09-12T16:21:19.202 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption1", 2023-09-12T16:21:19.202 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.203 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.203 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.203 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.203 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.203 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.204 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.204 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.204 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.204 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.204 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.204 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.205 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption2": { 2023-09-12T16:21:19.205 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption2", 2023-09-12T16:21:19.205 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.205 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.205 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.205 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.206 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.206 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.206 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.206 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.206 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.207 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.207 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.207 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.207 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption3": { 2023-09-12T16:21:19.207 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption3", 2023-09-12T16:21:19.207 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "float", 2023-09-12T16:21:19.208 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.208 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.208 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.208 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.208 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.209 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.209 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.209 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.209 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.209 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.210 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.210 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption4": { 2023-09-12T16:21:19.210 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption4", 2023-09-12T16:21:19.210 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.210 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.211 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.211 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.211 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.211 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.211 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.211 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.212 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.212 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.212 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.212 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.212 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption5": { 2023-09-12T16:21:19.212 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption5", 2023-09-12T16:21:19.213 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.213 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.213 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.213 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.213 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.213 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.214 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.214 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.214 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.214 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.214 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.215 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.215 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption6": { 2023-09-12T16:21:19.215 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption6", 2023-09-12T16:21:19.215 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.215 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.215 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.215 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:19.216 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.216 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.216 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.216 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.216 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.216 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.217 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.217 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.217 INFO:teuthology.orchestra.run.smithi130.stdout: "rwoption7": { 2023-09-12T16:21:19.217 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rwoption7", 2023-09-12T16:21:19.217 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.217 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.218 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.218 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.218 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "1", 2023-09-12T16:21:19.218 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "42", 2023-09-12T16:21:19.218 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.219 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.219 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.219 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.219 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.219 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.219 INFO:teuthology.orchestra.run.smithi130.stdout: "testkey": { 2023-09-12T16:21:19.220 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "testkey", 2023-09-12T16:21:19.220 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.220 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.220 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.220 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.221 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.221 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.221 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.221 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.221 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.221 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.222 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.222 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.222 INFO:teuthology.orchestra.run.smithi130.stdout: "testlkey": { 2023-09-12T16:21:19.222 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "testlkey", 2023-09-12T16:21:19.222 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.223 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.223 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.223 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.223 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.223 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.223 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.224 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.224 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.224 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.224 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.224 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.224 INFO:teuthology.orchestra.run.smithi130.stdout: "testnewline": { 2023-09-12T16:21:19.225 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "testnewline", 2023-09-12T16:21:19.225 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.225 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.225 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.225 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.226 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.226 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.226 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.226 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.226 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.226 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.227 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.227 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.227 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.227 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.227 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.227 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "snap_schedule", 2023-09-12T16:21:19.228 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.228 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.228 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.228 INFO:teuthology.orchestra.run.smithi130.stdout: "allow_m_granularity": { 2023-09-12T16:21:19.228 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "allow_m_granularity", 2023-09-12T16:21:19.228 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.229 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.229 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.229 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.229 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.229 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.229 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.230 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "allow minute scheduled snapshots", 2023-09-12T16:21:19.230 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.230 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.230 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.230 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.230 INFO:teuthology.orchestra.run.smithi130.stdout: "dump_on_update": { 2023-09-12T16:21:19.231 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "dump_on_update", 2023-09-12T16:21:19.231 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.231 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.231 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.231 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.232 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.232 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.232 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.232 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "dump database to debug log on update", 2023-09-12T16:21:19.232 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.232 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.233 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.233 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.233 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.233 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.233 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.233 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.234 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.234 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.234 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.234 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.234 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.235 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.235 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.235 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.235 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.235 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.235 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.236 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.236 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.236 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.236 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.236 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.236 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.237 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.237 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.237 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.237 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.237 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.237 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.238 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.238 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.238 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.238 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.238 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.239 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.239 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.239 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.239 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.239 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.240 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.240 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.240 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.240 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.240 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.240 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.241 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.241 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.241 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.241 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.241 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.242 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.242 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.242 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.242 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.242 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.243 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.243 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.243 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.243 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.243 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.243 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.244 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.244 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.244 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.244 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.244 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.245 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.245 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.245 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.245 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.245 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.245 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.246 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.246 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.246 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.246 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "stats", 2023-09-12T16:21:19.246 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.247 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.247 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.247 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.247 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.247 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.247 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.248 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.248 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.248 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.248 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.248 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.248 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.249 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.249 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.249 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.249 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.249 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.250 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.250 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.250 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.250 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.250 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.250 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.251 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.251 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.251 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.251 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.251 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.252 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.252 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.252 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.252 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.252 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.252 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.253 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.253 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.253 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.253 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.253 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.253 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.254 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.254 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.254 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.254 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.254 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.255 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.255 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.255 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.255 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.255 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.255 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.256 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.256 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.256 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.256 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.256 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.256 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.257 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.257 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.257 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.257 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.257 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.257 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.258 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.258 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.258 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.258 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.258 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.259 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.259 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.259 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.259 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.259 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.259 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.260 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.260 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "status", 2023-09-12T16:21:19.260 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.260 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.260 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.260 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.261 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.261 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.261 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.261 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.261 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.262 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.262 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.262 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.262 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.262 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.263 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.263 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.263 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.264 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.265 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.265 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.265 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.265 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.265 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.266 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.266 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.266 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.266 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.266 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.267 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.267 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.267 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.267 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.267 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.268 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.268 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.268 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.268 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.268 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.268 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.269 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.269 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.269 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.269 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.269 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.269 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.270 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.270 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.270 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.270 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.270 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.271 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.271 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.271 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.271 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.271 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.272 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.272 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.272 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.272 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.272 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.272 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.273 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.273 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.273 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.273 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.273 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.273 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.274 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.274 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.274 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.274 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.274 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.275 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.275 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.275 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.275 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.275 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "telegraf", 2023-09-12T16:21:19.275 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.276 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.276 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.276 INFO:teuthology.orchestra.run.smithi130.stdout: "address": { 2023-09-12T16:21:19.276 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "address", 2023-09-12T16:21:19.276 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.277 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.277 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.277 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-12T16:21:19.277 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.277 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.277 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.278 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.278 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.278 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.278 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.278 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.279 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:19.279 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:19.279 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:19.279 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.279 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.279 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "15", 2023-09-12T16:21:19.280 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.280 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.280 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.280 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.280 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.281 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.281 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.281 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.281 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.281 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.282 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.282 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.282 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.282 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.282 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.282 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.283 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.283 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.283 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.283 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.283 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.284 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.284 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.284 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.284 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.284 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.284 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.285 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.285 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.285 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.285 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.285 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.286 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.286 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.286 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.286 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.286 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.286 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.287 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.287 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.287 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.287 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.287 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.287 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.288 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.288 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.288 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.288 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.288 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.289 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.289 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.289 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.289 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.289 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.289 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.290 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.290 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.290 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.290 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.290 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.291 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.291 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.291 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.291 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.291 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.291 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.292 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.292 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.292 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.292 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.292 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.292 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.293 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.293 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.293 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.293 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.293 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.293 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.294 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.294 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.294 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.294 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "telemetry", 2023-09-12T16:21:19.294 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.295 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.295 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.295 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_basic": { 2023-09-12T16:21:19.295 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_basic", 2023-09-12T16:21:19.295 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.296 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.296 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.296 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:19.296 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.296 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.296 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.297 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-12T16:21:19.297 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.297 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.297 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.297 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.297 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_crash": { 2023-09-12T16:21:19.298 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_crash", 2023-09-12T16:21:19.298 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.298 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.298 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.298 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:19.299 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.299 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.299 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.299 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-12T16:21:19.299 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.299 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.300 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.300 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.300 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_device": { 2023-09-12T16:21:19.300 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_device", 2023-09-12T16:21:19.300 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.300 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.301 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.301 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "True", 2023-09-12T16:21:19.301 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.301 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.301 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.301 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-12T16:21:19.302 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.302 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.302 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.302 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.302 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_ident": { 2023-09-12T16:21:19.303 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_ident", 2023-09-12T16:21:19.303 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.303 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.303 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.303 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.303 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.304 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.304 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.304 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-12T16:21:19.304 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.304 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.304 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.305 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.305 INFO:teuthology.orchestra.run.smithi130.stdout: "channel_perf": { 2023-09-12T16:21:19.305 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "channel_perf", 2023-09-12T16:21:19.305 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.305 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.305 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.306 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.306 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.306 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.306 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.306 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-12T16:21:19.307 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.307 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.307 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.307 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.307 INFO:teuthology.orchestra.run.smithi130.stdout: "contact": { 2023-09-12T16:21:19.307 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "contact", 2023-09-12T16:21:19.308 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.308 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.308 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.308 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.308 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.308 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.309 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.309 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.309 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.309 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.309 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.310 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.310 INFO:teuthology.orchestra.run.smithi130.stdout: "description": { 2023-09-12T16:21:19.310 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "description", 2023-09-12T16:21:19.310 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.310 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.311 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.311 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.311 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.311 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.311 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.311 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.312 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.312 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.312 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.312 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.312 INFO:teuthology.orchestra.run.smithi130.stdout: "device_url": { 2023-09-12T16:21:19.313 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "device_url", 2023-09-12T16:21:19.313 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.313 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.313 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.313 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-12T16:21:19.314 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.314 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.314 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.314 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.314 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.314 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.315 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.315 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.315 INFO:teuthology.orchestra.run.smithi130.stdout: "enabled": { 2023-09-12T16:21:19.315 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "enabled", 2023-09-12T16:21:19.315 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.315 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.316 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.316 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.316 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.316 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.316 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.317 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.317 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.317 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.317 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.317 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.317 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:19.318 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:19.318 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.318 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.318 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.318 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "24", 2023-09-12T16:21:19.319 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "8", 2023-09-12T16:21:19.319 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.319 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.319 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.319 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.319 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.320 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.320 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.320 INFO:teuthology.orchestra.run.smithi130.stdout: "last_opt_revision": { 2023-09-12T16:21:19.320 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "last_opt_revision", 2023-09-12T16:21:19.320 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.320 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.321 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.321 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "1", 2023-09-12T16:21:19.321 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.321 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.321 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.321 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.322 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.322 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.322 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.322 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.322 INFO:teuthology.orchestra.run.smithi130.stdout: "leaderboard": { 2023-09-12T16:21:19.323 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "leaderboard", 2023-09-12T16:21:19.323 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.323 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.323 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.323 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.323 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.324 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.324 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.324 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.324 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.324 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.324 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.325 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.325 INFO:teuthology.orchestra.run.smithi130.stdout: "leaderboard_description": { 2023-09-12T16:21:19.325 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "leaderboard_description", 2023-09-12T16:21:19.325 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.325 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.325 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.326 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.326 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.326 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.326 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.326 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.326 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.327 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.327 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.327 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.327 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.327 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.328 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.328 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.328 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.328 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.328 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.328 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.329 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.329 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.329 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.329 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.329 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.329 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.330 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.330 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.330 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.330 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.330 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.330 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.331 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.331 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.331 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.331 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.331 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.332 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.332 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.332 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.332 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.332 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.332 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.333 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.333 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.333 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.333 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.333 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.333 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.334 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.334 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.334 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.334 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.334 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.334 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.335 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.335 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.335 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.335 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.335 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.336 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.336 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.336 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.336 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.336 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.336 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.337 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.337 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.337 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.337 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.337 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.337 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.338 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.338 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.338 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.338 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.338 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.339 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.339 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.339 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.339 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.339 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.340 INFO:teuthology.orchestra.run.smithi130.stdout: "organization": { 2023-09-12T16:21:19.340 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "organization", 2023-09-12T16:21:19.340 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.340 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.340 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.340 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.341 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.341 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.341 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.341 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.341 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.342 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.342 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.342 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.342 INFO:teuthology.orchestra.run.smithi130.stdout: "proxy": { 2023-09-12T16:21:19.342 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "proxy", 2023-09-12T16:21:19.342 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.343 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.343 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.343 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.343 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.343 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.344 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.344 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.344 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.344 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.344 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.344 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.345 INFO:teuthology.orchestra.run.smithi130.stdout: "url": { 2023-09-12T16:21:19.345 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "url", 2023-09-12T16:21:19.345 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.345 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.345 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.346 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-12T16:21:19.346 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.346 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.346 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.346 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.346 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.347 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.347 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.347 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.347 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.347 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.348 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.348 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "test_orchestrator", 2023-09-12T16:21:19.348 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.348 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.348 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.348 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.349 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.349 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.349 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.349 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.349 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.350 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.350 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.350 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.350 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.350 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.351 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.351 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.351 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.351 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.351 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.351 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.352 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.352 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.352 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.352 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.352 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.352 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.353 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.353 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.353 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.353 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.353 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.354 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.354 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.354 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.354 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.354 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.354 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.355 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.355 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.355 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.355 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.355 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.356 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.356 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.356 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.356 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.356 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.356 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.357 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.357 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.357 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.357 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.357 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.357 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.358 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.358 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.358 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.358 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.358 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.358 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.359 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.359 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.359 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.359 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.359 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.360 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.360 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.360 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.360 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.360 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.361 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.361 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.361 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.361 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.361 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.361 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.362 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "volumes", 2023-09-12T16:21:19.362 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.362 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.362 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.362 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.363 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.363 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.363 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.363 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.363 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.363 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.364 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.364 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.364 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.364 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.364 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.364 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.365 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.365 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.365 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.365 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.365 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.365 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.366 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.366 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.366 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.366 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.366 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.367 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.367 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.367 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.367 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.367 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.367 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.368 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.368 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.368 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.368 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.368 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.369 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.369 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.369 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.369 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.369 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.370 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.370 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.370 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.370 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.370 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.370 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.371 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.371 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.371 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.371 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.371 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.372 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.372 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.372 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.372 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.372 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.373 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.373 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.373 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.373 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.373 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.373 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.374 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.374 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.374 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.374 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.374 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.375 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.375 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.375 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.375 INFO:teuthology.orchestra.run.smithi130.stdout: "max_concurrent_clones": { 2023-09-12T16:21:19.375 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "max_concurrent_clones", 2023-09-12T16:21:19.375 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.376 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.376 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.376 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "4", 2023-09-12T16:21:19.376 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.376 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.377 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.377 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-12T16:21:19.377 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.377 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.377 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.377 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.378 INFO:teuthology.orchestra.run.smithi130.stdout: "snapshot_clone_delay": { 2023-09-12T16:21:19.378 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "snapshot_clone_delay", 2023-09-12T16:21:19.378 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.378 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.378 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.379 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "0", 2023-09-12T16:21:19.379 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.379 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.379 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.379 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-12T16:21:19.380 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.380 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.380 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.380 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.380 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.380 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.381 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.381 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix", 2023-09-12T16:21:19.381 INFO:teuthology.orchestra.run.smithi130.stdout: "can_run": true, 2023-09-12T16:21:19.381 INFO:teuthology.orchestra.run.smithi130.stdout: "error_string": "", 2023-09-12T16:21:19.381 INFO:teuthology.orchestra.run.smithi130.stdout: "module_options": { 2023-09-12T16:21:19.381 INFO:teuthology.orchestra.run.smithi130.stdout: "discovery_interval": { 2023-09-12T16:21:19.382 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "discovery_interval", 2023-09-12T16:21:19.382 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "uint", 2023-09-12T16:21:19.382 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.382 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.382 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "100", 2023-09-12T16:21:19.383 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.383 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.383 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.383 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.383 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.384 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.384 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.384 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.384 INFO:teuthology.orchestra.run.smithi130.stdout: "identifier": { 2023-09-12T16:21:19.384 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "identifier", 2023-09-12T16:21:19.384 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.385 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.385 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.385 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.385 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.385 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.385 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.386 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.386 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.386 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.386 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.386 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.387 INFO:teuthology.orchestra.run.smithi130.stdout: "interval": { 2023-09-12T16:21:19.387 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "interval", 2023-09-12T16:21:19.387 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "secs", 2023-09-12T16:21:19.387 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.387 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.387 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "60", 2023-09-12T16:21:19.388 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.388 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.388 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.388 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.388 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.389 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.389 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.389 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.389 INFO:teuthology.orchestra.run.smithi130.stdout: "log_level": { 2023-09-12T16:21:19.389 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_level", 2023-09-12T16:21:19.390 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.390 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.390 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.390 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.390 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.391 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.391 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.391 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.391 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.391 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.392 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.392 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.392 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.392 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.392 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.393 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.393 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.393 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.393 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.393 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster": { 2023-09-12T16:21:19.393 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster", 2023-09-12T16:21:19.394 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.394 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.394 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.394 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.394 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.395 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.395 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.395 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.395 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.395 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.396 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.396 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.396 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_cluster_level": { 2023-09-12T16:21:19.396 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_cluster_level", 2023-09-12T16:21:19.396 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.396 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.397 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.397 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "info", 2023-09-12T16:21:19.397 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.397 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.397 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [ 2023-09-12T16:21:19.398 INFO:teuthology.orchestra.run.smithi130.stdout: "", 2023-09-12T16:21:19.398 INFO:teuthology.orchestra.run.smithi130.stdout: "critical", 2023-09-12T16:21:19.398 INFO:teuthology.orchestra.run.smithi130.stdout: "debug", 2023-09-12T16:21:19.398 INFO:teuthology.orchestra.run.smithi130.stdout: "error", 2023-09-12T16:21:19.398 INFO:teuthology.orchestra.run.smithi130.stdout: "info", 2023-09-12T16:21:19.399 INFO:teuthology.orchestra.run.smithi130.stdout: "warning" 2023-09-12T16:21:19.399 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.399 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.399 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.399 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.399 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.400 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.400 INFO:teuthology.orchestra.run.smithi130.stdout: "log_to_file": { 2023-09-12T16:21:19.400 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "log_to_file", 2023-09-12T16:21:19.400 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "bool", 2023-09-12T16:21:19.400 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.401 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 1, 2023-09-12T16:21:19.401 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "False", 2023-09-12T16:21:19.401 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.401 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.401 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.401 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.402 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.402 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.402 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.402 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.402 INFO:teuthology.orchestra.run.smithi130.stdout: "zabbix_host": { 2023-09-12T16:21:19.403 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix_host", 2023-09-12T16:21:19.403 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.403 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.403 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.403 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "", 2023-09-12T16:21:19.403 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.404 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.404 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.404 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.404 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.404 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.404 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.405 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.405 INFO:teuthology.orchestra.run.smithi130.stdout: "zabbix_port": { 2023-09-12T16:21:19.405 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix_port", 2023-09-12T16:21:19.405 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "int", 2023-09-12T16:21:19.405 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.405 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.406 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "10051", 2023-09-12T16:21:19.406 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.406 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.406 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.406 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.407 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.407 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.407 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.407 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.407 INFO:teuthology.orchestra.run.smithi130.stdout: "zabbix_sender": { 2023-09-12T16:21:19.407 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "zabbix_sender", 2023-09-12T16:21:19.408 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "str", 2023-09-12T16:21:19.408 INFO:teuthology.orchestra.run.smithi130.stdout: "level": "advanced", 2023-09-12T16:21:19.408 INFO:teuthology.orchestra.run.smithi130.stdout: "flags": 0, 2023-09-12T16:21:19.408 INFO:teuthology.orchestra.run.smithi130.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-12T16:21:19.408 INFO:teuthology.orchestra.run.smithi130.stdout: "min": "", 2023-09-12T16:21:19.408 INFO:teuthology.orchestra.run.smithi130.stdout: "max": "", 2023-09-12T16:21:19.409 INFO:teuthology.orchestra.run.smithi130.stdout: "enum_allowed": [], 2023-09-12T16:21:19.409 INFO:teuthology.orchestra.run.smithi130.stdout: "desc": "", 2023-09-12T16:21:19.409 INFO:teuthology.orchestra.run.smithi130.stdout: "long_desc": "", 2023-09-12T16:21:19.409 INFO:teuthology.orchestra.run.smithi130.stdout: "tags": [], 2023-09-12T16:21:19.409 INFO:teuthology.orchestra.run.smithi130.stdout: "see_also": [] 2023-09-12T16:21:19.410 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.410 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.410 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.410 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.410 INFO:teuthology.orchestra.run.smithi130.stdout: "services": { 2023-09-12T16:21:19.411 INFO:teuthology.orchestra.run.smithi130.stdout: "dashboard": "https://172.21.15.130:8443/" 2023-09-12T16:21:19.411 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.411 INFO:teuthology.orchestra.run.smithi130.stdout: "always_on_modules": { 2023-09-12T16:21:19.411 INFO:teuthology.orchestra.run.smithi130.stdout: "octopus": [ 2023-09-12T16:21:19.411 INFO:teuthology.orchestra.run.smithi130.stdout: "balancer", 2023-09-12T16:21:19.412 INFO:teuthology.orchestra.run.smithi130.stdout: "crash", 2023-09-12T16:21:19.412 INFO:teuthology.orchestra.run.smithi130.stdout: "devicehealth", 2023-09-12T16:21:19.412 INFO:teuthology.orchestra.run.smithi130.stdout: "orchestrator", 2023-09-12T16:21:19.412 INFO:teuthology.orchestra.run.smithi130.stdout: "pg_autoscaler", 2023-09-12T16:21:19.412 INFO:teuthology.orchestra.run.smithi130.stdout: "progress", 2023-09-12T16:21:19.412 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_support", 2023-09-12T16:21:19.413 INFO:teuthology.orchestra.run.smithi130.stdout: "status", 2023-09-12T16:21:19.413 INFO:teuthology.orchestra.run.smithi130.stdout: "telemetry", 2023-09-12T16:21:19.413 INFO:teuthology.orchestra.run.smithi130.stdout: "volumes" 2023-09-12T16:21:19.413 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.413 INFO:teuthology.orchestra.run.smithi130.stdout: "pacific": [ 2023-09-12T16:21:19.414 INFO:teuthology.orchestra.run.smithi130.stdout: "balancer", 2023-09-12T16:21:19.414 INFO:teuthology.orchestra.run.smithi130.stdout: "crash", 2023-09-12T16:21:19.414 INFO:teuthology.orchestra.run.smithi130.stdout: "devicehealth", 2023-09-12T16:21:19.414 INFO:teuthology.orchestra.run.smithi130.stdout: "orchestrator", 2023-09-12T16:21:19.414 INFO:teuthology.orchestra.run.smithi130.stdout: "pg_autoscaler", 2023-09-12T16:21:19.415 INFO:teuthology.orchestra.run.smithi130.stdout: "progress", 2023-09-12T16:21:19.415 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_support", 2023-09-12T16:21:19.415 INFO:teuthology.orchestra.run.smithi130.stdout: "status", 2023-09-12T16:21:19.415 INFO:teuthology.orchestra.run.smithi130.stdout: "telemetry", 2023-09-12T16:21:19.415 INFO:teuthology.orchestra.run.smithi130.stdout: "volumes" 2023-09-12T16:21:19.415 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.416 INFO:teuthology.orchestra.run.smithi130.stdout: "quincy": [ 2023-09-12T16:21:19.416 INFO:teuthology.orchestra.run.smithi130.stdout: "balancer", 2023-09-12T16:21:19.416 INFO:teuthology.orchestra.run.smithi130.stdout: "crash", 2023-09-12T16:21:19.416 INFO:teuthology.orchestra.run.smithi130.stdout: "devicehealth", 2023-09-12T16:21:19.416 INFO:teuthology.orchestra.run.smithi130.stdout: "orchestrator", 2023-09-12T16:21:19.416 INFO:teuthology.orchestra.run.smithi130.stdout: "pg_autoscaler", 2023-09-12T16:21:19.417 INFO:teuthology.orchestra.run.smithi130.stdout: "progress", 2023-09-12T16:21:19.417 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_support", 2023-09-12T16:21:19.417 INFO:teuthology.orchestra.run.smithi130.stdout: "status", 2023-09-12T16:21:19.417 INFO:teuthology.orchestra.run.smithi130.stdout: "telemetry", 2023-09-12T16:21:19.417 INFO:teuthology.orchestra.run.smithi130.stdout: "volumes" 2023-09-12T16:21:19.418 INFO:teuthology.orchestra.run.smithi130.stdout: ], 2023-09-12T16:21:19.418 INFO:teuthology.orchestra.run.smithi130.stdout: "reef": [ 2023-09-12T16:21:19.418 INFO:teuthology.orchestra.run.smithi130.stdout: "balancer", 2023-09-12T16:21:19.418 INFO:teuthology.orchestra.run.smithi130.stdout: "crash", 2023-09-12T16:21:19.418 INFO:teuthology.orchestra.run.smithi130.stdout: "devicehealth", 2023-09-12T16:21:19.418 INFO:teuthology.orchestra.run.smithi130.stdout: "orchestrator", 2023-09-12T16:21:19.419 INFO:teuthology.orchestra.run.smithi130.stdout: "pg_autoscaler", 2023-09-12T16:21:19.419 INFO:teuthology.orchestra.run.smithi130.stdout: "progress", 2023-09-12T16:21:19.419 INFO:teuthology.orchestra.run.smithi130.stdout: "rbd_support", 2023-09-12T16:21:19.419 INFO:teuthology.orchestra.run.smithi130.stdout: "status", 2023-09-12T16:21:19.419 INFO:teuthology.orchestra.run.smithi130.stdout: "telemetry", 2023-09-12T16:21:19.420 INFO:teuthology.orchestra.run.smithi130.stdout: "volumes" 2023-09-12T16:21:19.420 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T16:21:19.420 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.420 INFO:teuthology.orchestra.run.smithi130.stdout: "last_failure_osd_epoch": 3, 2023-09-12T16:21:19.420 INFO:teuthology.orchestra.run.smithi130.stdout: "active_clients": [ 2023-09-12T16:21:19.420 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.421 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "devicehealth", 2023-09-12T16:21:19.421 INFO:teuthology.orchestra.run.smithi130.stdout: "addrvec": [ 2023-09-12T16:21:19.421 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.421 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v2", 2023-09-12T16:21:19.421 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2023-09-12T16:21:19.422 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 609656151 2023-09-12T16:21:19.422 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.422 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T16:21:19.422 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.422 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.423 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "libcephsqlite", 2023-09-12T16:21:19.423 INFO:teuthology.orchestra.run.smithi130.stdout: "addrvec": [ 2023-09-12T16:21:19.423 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.423 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v2", 2023-09-12T16:21:19.423 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2023-09-12T16:21:19.423 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 2101559487 2023-09-12T16:21:19.424 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.424 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T16:21:19.424 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.424 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.424 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "rbd_support", 2023-09-12T16:21:19.425 INFO:teuthology.orchestra.run.smithi130.stdout: "addrvec": [ 2023-09-12T16:21:19.425 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.425 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v2", 2023-09-12T16:21:19.425 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2023-09-12T16:21:19.425 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1633987371 2023-09-12T16:21:19.426 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.426 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T16:21:19.426 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:19.426 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.426 INFO:teuthology.orchestra.run.smithi130.stdout: "name": "volumes", 2023-09-12T16:21:19.426 INFO:teuthology.orchestra.run.smithi130.stdout: "addrvec": [ 2023-09-12T16:21:19.427 INFO:teuthology.orchestra.run.smithi130.stdout: { 2023-09-12T16:21:19.427 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v2", 2023-09-12T16:21:19.427 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2023-09-12T16:21:19.427 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 3484494959 2023-09-12T16:21:19.427 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.427 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T16:21:19.428 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:19.428 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2023-09-12T16:21:19.428 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:21:19.430 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:15 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2599812154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:21:19.430 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:15 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/519079611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:21:19.431 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:15 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/4237592491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:21:19.431 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:15 smithi203 ceph-mon[103431]: pgmap v842: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.2 MiB/s wr, 419 op/s 2023-09-12T16:21:19.431 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:16 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1254178117' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-12T16:21:19.431 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:18 smithi203 ceph-mon[103431]: pgmap v843: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 241 op/s 2023-09-12T16:21:19.431 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:15 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2599812154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:21:19.432 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:15 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/519079611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:21:19.432 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:15 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/4237592491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:21:19.432 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:15 smithi159 ceph-mon[102522]: pgmap v842: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.2 MiB/s wr, 419 op/s 2023-09-12T16:21:19.432 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:16 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1254178117' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-12T16:21:19.432 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:18 smithi159 ceph-mon[102522]: pgmap v843: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 241 op/s 2023-09-12T16:21:19.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fb40a0690 msgr2=0x7f1fb40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:19.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fb40a0690 0x7f1fb40a2b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009990 tx=0x7f1fc4005fb0 comp rx=0 tx=0).stop 2023-09-12T16:21:19.433 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc81017f0 msgr2=0x7f1fc8196800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:19.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc81017f0 0x7f1fc8196800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00e9a0 tx=0x7f1fbc00ecb0 comp rx=0 tx=0).stop 2023-09-12T16:21:19.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 shutdown_connections 2023-09-12T16:21:19.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f1fb40a0690 0x7f1fb40a2b50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:19.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fc8102190 0x7f1fc819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:19.434 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc8100600 0x7f1fc81962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:19.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 --2- 172.21.15.130:0/1254178117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc81017f0 0x7f1fc8196800 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:19.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 >> 172.21.15.130:0/1254178117 conn(0x7f1fc80fbb50 msgr2=0x7f1fc8104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:19.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.063+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 shutdown_connections 2023-09-12T16:21:19.435 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:16.064+0000 7f1fcfa8f700 1 -- 172.21.15.130:0/1254178117 wait complete. 2023-09-12T16:21:19.441 DEBUG:tasks.check_counter:Getting stats from a 2023-09-12T16:21:19.441 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.a perf dump 2023-09-12T16:21:19.442 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:16 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1254178117' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-12T16:21:19.442 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:18 smithi130 ceph-mon[98670]: pgmap v843: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 241 op/s 2023-09-12T16:21:20.056 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:21:20.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:19 smithi130 ceph-mon[98670]: pgmap v844: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 476 KiB/s wr, 70 op/s 2023-09-12T16:21:20.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:19 smithi159 ceph-mon[102522]: pgmap v844: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 476 KiB/s wr, 70 op/s 2023-09-12T16:21:20.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:19 smithi203 ceph-mon[103431]: pgmap v844: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 476 KiB/s wr, 70 op/s 2023-09-12T16:21:22.094 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:21:22.095 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:22.095 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 0, 2023-09-12T16:21:22.095 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 2, 2023-09-12T16:21:22.095 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 0, 2023-09-12T16:21:22.096 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 0, 2023-09-12T16:21:22.096 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:22.096 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 0, 2023-09-12T16:21:22.096 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.001361075, 2023-09-12T16:21:22.096 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.000000000, 2023-09-12T16:21:22.097 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.000281159, 2023-09-12T16:21:22.097 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:22.097 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:22.098 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:22.098 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:22.098 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:22.098 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.099 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:22.099 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:22.099 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:22.099 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:22.100 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.100 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-12T16:21:22.100 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2023-09-12T16:21:22.101 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.101 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:22.101 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 2, 2023-09-12T16:21:22.102 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 237, 2023-09-12T16:21:22.102 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 320, 2023-09-12T16:21:22.102 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 194576, 2023-09-12T16:21:22.102 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:22.103 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2023-09-12T16:21:22.103 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.015808687, 2023-09-12T16:21:22.103 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.012367491, 2023-09-12T16:21:22.104 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.001579939, 2023-09-12T16:21:22.104 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:22.104 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:22.105 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 235, 2023-09-12T16:21:22.105 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.103497023, 2023-09-12T16:21:22.105 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000440412 2023-09-12T16:21:22.105 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.106 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:22.106 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:22.106 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:22.107 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:22.107 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.107 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-12T16:21:22.108 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 194576 2023-09-12T16:21:22.108 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.108 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:22.108 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 318, 2023-09-12T16:21:22.109 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 313, 2023-09-12T16:21:22.109 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 500704, 2023-09-12T16:21:22.109 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 187664, 2023-09-12T16:21:22.110 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 2, 2023-09-12T16:21:22.110 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2023-09-12T16:21:22.111 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.044967320, 2023-09-12T16:21:22.111 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.017027905, 2023-09-12T16:21:22.111 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.017053203, 2023-09-12T16:21:22.111 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.009224844, 2023-09-12T16:21:22.112 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:22.112 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 313, 2023-09-12T16:21:22.113 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.148090387, 2023-09-12T16:21:22.113 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000473132 2023-09-12T16:21:22.113 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.114 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:22.114 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:22.114 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:22.115 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:22.115 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.116 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 500704, 2023-09-12T16:21:22.116 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 187664 2023-09-12T16:21:22.116 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.117 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2023-09-12T16:21:22.117 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 0, 2023-09-12T16:21:22.117 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2023-09-12T16:21:22.118 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.118 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2023-09-12T16:21:22.118 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:22.118 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:22.119 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:22.119 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:22.119 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:22.120 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:22.120 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:22.120 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:22.121 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:22.121 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:22.121 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:22.121 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:22.122 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:22.122 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:22.122 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:22.122 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:22.123 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:22.123 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:22.123 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:22.123 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:22.124 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:22.124 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:22.124 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:22.125 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:22.125 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:22.125 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:22.126 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:22.126 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:22.126 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:22.127 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:22.127 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:22.127 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2023-09-12T16:21:22.129 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:22.129 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:22.130 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:22.130 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:22.131 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 148761, 2023-09-12T16:21:22.131 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 31, 2023-09-12T16:21:22.131 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:22.132 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:22.132 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2023-09-12T16:21:22.132 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2023-09-12T16:21:22.133 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:22.133 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:22.133 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:22.134 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:22.134 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 0, 2023-09-12T16:21:22.134 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 0, 2023-09-12T16:21:22.135 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:22.135 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:22.135 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:22.136 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2023-09-12T16:21:22.136 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 928, 2023-09-12T16:21:22.136 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2, 2023-09-12T16:21:22.137 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:22.137 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2023-09-12T16:21:22.137 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:22.138 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2023-09-12T16:21:22.138 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:22.138 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:22.138 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2023-09-12T16:21:22.139 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2023-09-12T16:21:22.139 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2023-09-12T16:21:22.139 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 316, 2023-09-12T16:21:22.139 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 453341, 2023-09-12T16:21:22.140 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:22.141 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 316, 2023-09-12T16:21:22.141 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2023-09-12T16:21:22.141 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2023-09-12T16:21:22.142 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 316, 2023-09-12T16:21:22.142 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 453341, 2023-09-12T16:21:22.142 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2023-09-12T16:21:22.142 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:22.143 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:22.143 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:22.143 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:22.144 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:22.144 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:21:22.198 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:21 smithi130 ceph-mon[98670]: pgmap v845: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 472 KiB/s wr, 72 op/s 2023-09-12T16:21:22.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:21 smithi159 ceph-mon[102522]: pgmap v845: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 472 KiB/s wr, 72 op/s 2023-09-12T16:21:22.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:21 smithi203 ceph-mon[103431]: pgmap v845: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 472 KiB/s wr, 72 op/s 2023-09-12T16:21:22.408 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-09-12T16:21:22.409 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-09-12T16:21:22.409 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-09-12T16:21:22.409 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-09-12T16:21:22.410 DEBUG:tasks.check_counter:Getting stats from d 2023-09-12T16:21:22.410 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.d perf dump 2023-09-12T16:21:22.971 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:21:24.197 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:21:24.197 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:24.197 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 4, 2023-09-12T16:21:24.197 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 4, 2023-09-12T16:21:24.198 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 2992, 2023-09-12T16:21:24.198 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 448, 2023-09-12T16:21:24.198 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:24.198 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 0, 2023-09-12T16:21:24.198 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.001884777, 2023-09-12T16:21:24.199 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.000049678, 2023-09-12T16:21:24.199 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.000738095, 2023-09-12T16:21:24.199 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:24.199 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:24.199 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3, 2023-09-12T16:21:24.200 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000683718, 2023-09-12T16:21:24.200 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000227906 2023-09-12T16:21:24.200 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.200 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:24.200 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:24.200 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:24.201 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:24.201 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.201 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-09-12T16:21:24.201 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2023-09-12T16:21:24.201 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.202 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:24.202 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 318, 2023-09-12T16:21:24.202 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 315, 2023-09-12T16:21:24.202 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 498800, 2023-09-12T16:21:24.202 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 189712, 2023-09-12T16:21:24.203 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 2, 2023-09-12T16:21:24.203 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2023-09-12T16:21:24.203 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.044346951, 2023-09-12T16:21:24.203 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.017180372, 2023-09-12T16:21:24.203 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.016641292, 2023-09-12T16:21:24.203 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.008773926, 2023-09-12T16:21:24.204 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:24.204 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 314, 2023-09-12T16:21:24.204 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.113272114, 2023-09-12T16:21:24.204 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000360739 2023-09-12T16:21:24.204 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.205 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:24.205 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:24.205 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:24.205 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:24.205 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.206 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 498800, 2023-09-12T16:21:24.206 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 189712 2023-09-12T16:21:24.206 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.206 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:24.206 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 2, 2023-09-12T16:21:24.207 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 241, 2023-09-12T16:21:24.207 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 320, 2023-09-12T16:21:24.207 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 195280, 2023-09-12T16:21:24.207 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:24.207 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2023-09-12T16:21:24.207 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.016188444, 2023-09-12T16:21:24.208 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.012628434, 2023-09-12T16:21:24.208 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.001647207, 2023-09-12T16:21:24.208 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:24.208 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:24.208 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 239, 2023-09-12T16:21:24.209 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.083710697, 2023-09-12T16:21:24.209 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000350253 2023-09-12T16:21:24.209 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.209 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:24.209 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:24.209 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:24.210 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:24.210 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.210 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-12T16:21:24.210 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 195280 2023-09-12T16:21:24.210 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.211 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2023-09-12T16:21:24.211 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 0, 2023-09-12T16:21:24.211 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2023-09-12T16:21:24.211 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.211 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2023-09-12T16:21:24.212 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:24.212 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:24.212 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:24.212 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:24.212 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:24.213 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:24.213 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:24.213 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:24.213 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:24.213 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:24.213 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:24.214 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:24.214 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:24.214 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:24.214 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:24.214 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:24.215 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:24.215 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:24.215 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:24.215 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:24.215 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:24.215 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:24.216 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:24.216 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:24.216 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:24.216 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:24.216 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:24.217 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:24.217 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:24.217 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:24.217 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:24.217 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2023-09-12T16:21:24.217 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:24.218 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:24.218 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:24.218 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:24.218 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 148704, 2023-09-12T16:21:24.218 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 29, 2023-09-12T16:21:24.219 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:24.219 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:24.219 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2023-09-12T16:21:24.219 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2023-09-12T16:21:24.219 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:24.220 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:24.220 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:24.220 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:24.220 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 0, 2023-09-12T16:21:24.220 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 0, 2023-09-12T16:21:24.220 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:24.221 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:24.221 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:24.221 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2023-09-12T16:21:24.221 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 928, 2023-09-12T16:21:24.221 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2, 2023-09-12T16:21:24.222 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:24.222 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2023-09-12T16:21:24.222 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:24.222 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2023-09-12T16:21:24.222 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:24.223 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:24.223 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2023-09-12T16:21:24.223 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2023-09-12T16:21:24.223 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2023-09-12T16:21:24.223 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 320, 2023-09-12T16:21:24.223 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 453861, 2023-09-12T16:21:24.224 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:24.224 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 320, 2023-09-12T16:21:24.224 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2023-09-12T16:21:24.224 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2023-09-12T16:21:24.224 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 320, 2023-09-12T16:21:24.225 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 453861, 2023-09-12T16:21:24.225 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2023-09-12T16:21:24.225 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:24.225 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:24.225 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:24.225 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:24.226 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:24.226 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:21:24.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:23 smithi159 ceph-mon[102522]: pgmap v846: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 17 op/s 2023-09-12T16:21:24.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:23 smithi130 ceph-mon[98670]: pgmap v846: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 17 op/s 2023-09-12T16:21:24.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:23 smithi203 ceph-mon[103431]: pgmap v846: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 17 op/s 2023-09-12T16:21:24.532 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-09-12T16:21:24.533 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-09-12T16:21:24.533 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-09-12T16:21:24.533 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-09-12T16:21:24.533 DEBUG:tasks.check_counter:Getting stats from g 2023-09-12T16:21:24.534 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.g perf dump 2023-09-12T16:21:25.090 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:21:26.158 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:25 smithi130 ceph-mon[98670]: pgmap v847: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 15 op/s 2023-09-12T16:21:26.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:25 smithi159 ceph-mon[102522]: pgmap v847: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 15 op/s 2023-09-12T16:21:26.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:25 smithi203 ceph-mon[103431]: pgmap v847: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 15 op/s 2023-09-12T16:21:26.326 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:21:26.326 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:26.327 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 0, 2023-09-12T16:21:26.327 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 2, 2023-09-12T16:21:26.327 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 0, 2023-09-12T16:21:26.327 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 0, 2023-09-12T16:21:26.327 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:26.327 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 0, 2023-09-12T16:21:26.328 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.001273131, 2023-09-12T16:21:26.328 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.000000000, 2023-09-12T16:21:26.328 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.000279778, 2023-09-12T16:21:26.328 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:26.328 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:26.329 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:26.329 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:26.329 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:26.329 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.329 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:26.330 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:26.330 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:26.330 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:26.330 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.330 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-12T16:21:26.331 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2023-09-12T16:21:26.331 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.331 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:26.331 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 315, 2023-09-12T16:21:26.331 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 312, 2023-09-12T16:21:26.331 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 497984, 2023-09-12T16:21:26.332 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 187840, 2023-09-12T16:21:26.332 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 2, 2023-09-12T16:21:26.332 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2023-09-12T16:21:26.332 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.043806358, 2023-09-12T16:21:26.332 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.017019599, 2023-09-12T16:21:26.333 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.016928991, 2023-09-12T16:21:26.333 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.008486318, 2023-09-12T16:21:26.333 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:26.333 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 311, 2023-09-12T16:21:26.333 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.129498785, 2023-09-12T16:21:26.333 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000416394 2023-09-12T16:21:26.334 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.334 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:26.334 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:26.334 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:26.334 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:26.335 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.335 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 497984, 2023-09-12T16:21:26.335 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 187840 2023-09-12T16:21:26.335 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.335 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:26.336 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 6, 2023-09-12T16:21:26.336 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 241, 2023-09-12T16:21:26.336 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 3312, 2023-09-12T16:21:26.336 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 195376, 2023-09-12T16:21:26.336 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:26.336 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2023-09-12T16:21:26.337 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.016609787, 2023-09-12T16:21:26.337 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.012647203, 2023-09-12T16:21:26.337 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.002141785, 2023-09-12T16:21:26.337 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:26.337 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:26.338 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 240, 2023-09-12T16:21:26.338 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.096150437, 2023-09-12T16:21:26.338 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000400626 2023-09-12T16:21:26.338 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.338 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:26.339 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:26.339 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:26.339 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:26.339 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.339 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-09-12T16:21:26.339 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 195376 2023-09-12T16:21:26.340 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.340 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2023-09-12T16:21:26.340 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 0, 2023-09-12T16:21:26.340 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2023-09-12T16:21:26.340 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.341 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2023-09-12T16:21:26.341 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:26.341 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:26.341 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:26.341 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:26.342 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:26.342 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:26.342 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:26.342 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:26.342 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:26.343 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:26.343 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:26.343 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:26.343 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:26.343 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:26.343 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:26.344 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:26.344 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:26.344 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:26.344 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:26.344 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:26.345 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:26.345 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:26.345 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:26.345 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:26.345 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:26.345 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:26.346 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:26.346 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:26.346 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:26.346 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:26.346 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:26.347 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2023-09-12T16:21:26.347 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:26.347 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:26.347 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:26.347 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:26.347 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 148744, 2023-09-12T16:21:26.348 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 30, 2023-09-12T16:21:26.348 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:26.348 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:26.348 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2023-09-12T16:21:26.348 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2023-09-12T16:21:26.349 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:26.349 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:26.349 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:26.349 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:26.349 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 0, 2023-09-12T16:21:26.350 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 0, 2023-09-12T16:21:26.350 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:26.350 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:26.350 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:26.350 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2023-09-12T16:21:26.350 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 928, 2023-09-12T16:21:26.351 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2, 2023-09-12T16:21:26.351 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:26.351 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2023-09-12T16:21:26.351 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:26.351 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2023-09-12T16:21:26.352 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:26.352 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:26.352 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2023-09-12T16:21:26.352 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2023-09-12T16:21:26.352 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2023-09-12T16:21:26.352 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 317, 2023-09-12T16:21:26.353 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 453471, 2023-09-12T16:21:26.353 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:26.353 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 317, 2023-09-12T16:21:26.353 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2023-09-12T16:21:26.353 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2023-09-12T16:21:26.354 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 317, 2023-09-12T16:21:26.354 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 453471, 2023-09-12T16:21:26.354 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2023-09-12T16:21:26.354 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:26.354 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:26.355 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:26.355 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:26.355 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:26.355 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:21:26.741 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-09-12T16:21:26.742 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-09-12T16:21:26.742 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-09-12T16:21:26.742 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-09-12T16:21:26.743 DEBUG:tasks.check_counter:Getting stats from j 2023-09-12T16:21:26.743 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.j perf dump 2023-09-12T16:21:27.299 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-12T16:21:28.517 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2023-09-12T16:21:28.517 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:28.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 0, 2023-09-12T16:21:28.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 2, 2023-09-12T16:21:28.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 0, 2023-09-12T16:21:28.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 0, 2023-09-12T16:21:28.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:28.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 0, 2023-09-12T16:21:28.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.001305469, 2023-09-12T16:21:28.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.000000000, 2023-09-12T16:21:28.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.000350630, 2023-09-12T16:21:28.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:28.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:28.519 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:28.519 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:28.520 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:28.520 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.520 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:28.520 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:28.520 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:28.521 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:28.521 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.521 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-12T16:21:28.521 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2023-09-12T16:21:28.521 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.522 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:28.522 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 324, 2023-09-12T16:21:28.522 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 319, 2023-09-12T16:21:28.522 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 502336, 2023-09-12T16:21:28.522 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 191408, 2023-09-12T16:21:28.522 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 2, 2023-09-12T16:21:28.523 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2023-09-12T16:21:28.523 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.045098666, 2023-09-12T16:21:28.523 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.017800891, 2023-09-12T16:21:28.523 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.016714838, 2023-09-12T16:21:28.523 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.008694544, 2023-09-12T16:21:28.524 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:28.524 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 319, 2023-09-12T16:21:28.524 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.127489752, 2023-09-12T16:21:28.524 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000399654 2023-09-12T16:21:28.524 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.524 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:28.525 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:28.525 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:28.525 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:28.525 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.525 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 502336, 2023-09-12T16:21:28.526 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 191408 2023-09-12T16:21:28.526 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.526 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:28.526 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 2, 2023-09-12T16:21:28.526 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 242, 2023-09-12T16:21:28.526 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 320, 2023-09-12T16:21:28.527 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 195456, 2023-09-12T16:21:28.527 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:28.527 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2023-09-12T16:21:28.527 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.016415562, 2023-09-12T16:21:28.527 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.012752928, 2023-09-12T16:21:28.528 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.001731984, 2023-09-12T16:21:28.528 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:28.528 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:28.528 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 240, 2023-09-12T16:21:28.528 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.106861896, 2023-09-12T16:21:28.529 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000445257 2023-09-12T16:21:28.529 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.529 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:28.529 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:28.529 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:28.529 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:28.530 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.530 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-12T16:21:28.530 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 195456 2023-09-12T16:21:28.530 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.530 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2023-09-12T16:21:28.531 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 0, 2023-09-12T16:21:28.531 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2023-09-12T16:21:28.531 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.531 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2023-09-12T16:21:28.531 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:28.531 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:28.532 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:28.532 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:28.532 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:28.532 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:28.532 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:28.533 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:28.533 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:28.533 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:28.533 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:28.533 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:28.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:28.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:28.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:28.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:28.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:28.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:28.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:28.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:28.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:28.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:28.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:28.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:28.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:28.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:28.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:28.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:28.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:28.537 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:28.537 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:28.537 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2023-09-12T16:21:28.537 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:28.537 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:28.538 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:28.538 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:28.538 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 148784, 2023-09-12T16:21:28.538 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 31, 2023-09-12T16:21:28.538 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:28.539 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:28.539 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2023-09-12T16:21:28.539 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2023-09-12T16:21:28.539 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:28.539 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:28.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:28.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:28.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 0, 2023-09-12T16:21:28.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 0, 2023-09-12T16:21:28.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:28.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:28.541 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:28.541 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2023-09-12T16:21:28.541 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 928, 2023-09-12T16:21:28.541 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2, 2023-09-12T16:21:28.541 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:28.542 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2023-09-12T16:21:28.542 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:28.542 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2023-09-12T16:21:28.542 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2023-09-12T16:21:28.542 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:28.543 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2023-09-12T16:21:28.543 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2023-09-12T16:21:28.543 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2023-09-12T16:21:28.543 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 322, 2023-09-12T16:21:28.543 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 454121, 2023-09-12T16:21:28.543 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:28.544 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 322, 2023-09-12T16:21:28.544 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2023-09-12T16:21:28.544 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2023-09-12T16:21:28.544 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 322, 2023-09-12T16:21:28.544 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 454121, 2023-09-12T16:21:28.544 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2023-09-12T16:21:28.545 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2023-09-12T16:21:28.545 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2023-09-12T16:21:28.545 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2023-09-12T16:21:28.545 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:28.545 INFO:teuthology.orchestra.run.smithi130.stdout: } 2023-09-12T16:21:28.546 INFO:teuthology.orchestra.run.smithi130.stdout:} 2023-09-12T16:21:28.849 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:28 smithi130 ceph-mon[98670]: pgmap v848: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-12T16:21:28.851 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-09-12T16:21:28.851 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-09-12T16:21:28.852 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-09-12T16:21:28.852 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-09-12T16:21:28.852 DEBUG:tasks.check_counter:Getting stats from b 2023-09-12T16:21:28.853 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.b perf dump 2023-09-12T16:21:28.965 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:28 smithi159 ceph-mon[102522]: pgmap v848: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-12T16:21:28.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:28 smithi203 ceph-mon[103431]: pgmap v848: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-12T16:21:29.580 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T16:21:30.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:29 smithi159 ceph-mon[102522]: pgmap v849: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-09-12T16:21:30.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:29 smithi130 ceph-mon[98670]: pgmap v849: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-09-12T16:21:30.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:29 smithi203 ceph-mon[103431]: pgmap v849: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-09-12T16:21:31.310 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-09-12T16:21:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 38724, 2023-09-12T16:21:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 25461, 2023-09-12T16:21:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 10081670, 2023-09-12T16:21:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 13260177, 2023-09-12T16:21:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 10, 2023-09-12T16:21:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2023-09-12T16:21:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 1.495537221, 2023-09-12T16:21:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.511468112, 2023-09-12T16:21:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 1.140805059, 2023-09-12T16:21:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.116074347, 2023-09-12T16:21:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 25462, 2023-09-12T16:21:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.683226899, 2023-09-12T16:21:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000144655 2023-09-12T16:21:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 57567, 2023-09-12T16:21:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.008231859, 2023-09-12T16:21:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000142 2023-09-12T16:21:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-12T16:21:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2023-09-12T16:21:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1859, 2023-09-12T16:21:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2022, 2023-09-12T16:21:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 514354, 2023-09-12T16:21:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 6220945, 2023-09-12T16:21:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 80, 2023-09-12T16:21:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2023-09-12T16:21:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.171229769, 2023-09-12T16:21:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.049085306, 2023-09-12T16:21:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.056513426, 2023-09-12T16:21:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.055361155, 2023-09-12T16:21:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2021, 2023-09-12T16:21:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.686476585, 2023-09-12T16:21:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000339671 2023-09-12T16:21:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 267, 2023-09-12T16:21:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000055451, 2023-09-12T16:21:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000207 2023-09-12T16:21:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 60208, 2023-09-12T16:21:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 627488 2023-09-12T16:21:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 3513333, 2023-09-12T16:21:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 3516968, 2023-09-12T16:21:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 558198163, 2023-09-12T16:21:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 580727534, 2023-09-12T16:21:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 15, 2023-09-12T16:21:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2023-09-12T16:21:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 115.425110409, 2023-09-12T16:21:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 41.734918443, 2023-09-12T16:21:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 85.229414150, 2023-09-12T16:21:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.166927516, 2023-09-12T16:21:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3516964, 2023-09-12T16:21:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 112.383819125, 2023-09-12T16:21:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000031954 2023-09-12T16:21:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 7021719, 2023-09-12T16:21:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.846217724, 2023-09-12T16:21:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000120 2023-09-12T16:21:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 4659104, 2023-09-12T16:21:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 7923520 2023-09-12T16:21:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-09-12T16:21:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-09-12T16:21:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-09-12T16:21:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-09-12T16:21:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-09-12T16:21:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-09-12T16:21:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4828, 2023-09-12T16:21:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.601847544, 2023-09-12T16:21:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000746033 2023-09-12T16:21:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-09-12T16:21:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-09-12T16:21:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-09-12T16:21:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1329, 2023-09-12T16:21:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.062450590, 2023-09-12T16:21:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000046990 2023-09-12T16:21:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-09-12T16:21:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 5155, 2023-09-12T16:21:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 4103, 2023-09-12T16:21:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-09-12T16:21:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4103, 2023-09-12T16:21:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 66.182967899, 2023-09-12T16:21:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.016130384 2023-09-12T16:21:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-09-12T16:21:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 1052, 2023-09-12T16:21:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2023-09-12T16:21:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-09-12T16:21:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 39, 2023-09-12T16:21:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 3, 2023-09-12T16:21:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2023-09-12T16:21:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 822, 2023-09-12T16:21:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 18, 2023-09-12T16:21:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 740, 2023-09-12T16:21:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 64, 2023-09-12T16:21:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 796, 2023-09-12T16:21:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 1248, 2023-09-12T16:21:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 703, 2023-09-12T16:21:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 703, 2023-09-12T16:21:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 11, 2023-09-12T16:21:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 9262, 2023-09-12T16:21:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 6608, 2023-09-12T16:21:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 929, 2023-09-12T16:21:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 395, 2023-09-12T16:21:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2023-09-12T16:21:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 1, 2023-09-12T16:21:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 290, 2023-09-12T16:21:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 8288, 2023-09-12T16:21:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-09-12T16:21:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 17, 2023-09-12T16:21:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 469, 2023-09-12T16:21:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 9, 2023-09-12T16:21:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 121, 2023-09-12T16:21:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-09-12T16:21:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 706, 2023-09-12T16:21:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 1, 2023-09-12T16:21:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 1046, 2023-09-12T16:21:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 9079358200, 2023-09-12T16:21:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2023-09-12T16:21:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 706, 2023-09-12T16:21:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-09-12T16:21:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-09-12T16:21:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-09-12T16:21:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 41, 2023-09-12T16:21:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 2, 2023-09-12T16:21:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 39, 2023-09-12T16:21:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 665, 2023-09-12T16:21:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 7, 2023-09-12T16:21:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 1140, 2023-09-12T16:21:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 3176, 2023-09-12T16:21:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-09-12T16:21:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-12T16:21:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-12T16:21:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 4219, 2023-09-12T16:21:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 2476, 2023-09-12T16:21:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 264, 2023-09-12T16:21:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 898 2023-09-12T16:21:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-09-12T16:21:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 98, 2023-09-12T16:21:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2023-09-12T16:21:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2023-09-12T16:21:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 396, 2023-09-12T16:21:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 260, 2023-09-12T16:21:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2023-09-12T16:21:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-09-12T16:21:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-09-12T16:21:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 332, 2023-09-12T16:21:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 298, 2023-09-12T16:21:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 136, 2023-09-12T16:21:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-09-12T16:21:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-09-12T16:21:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-09-12T16:21:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-09-12T16:21:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-09-12T16:21:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-09-12T16:21:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 2, 2023-09-12T16:21:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 17, 2023-09-12T16:21:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-09-12T16:21:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 3, 2023-09-12T16:21:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-09-12T16:21:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-09-12T16:21:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-09-12T16:21:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2023-09-12T16:21:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 6436, 2023-09-12T16:21:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-09-12T16:21:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-09-12T16:21:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 6436, 2023-09-12T16:21:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-09-12T16:21:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-09-12T16:21:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 10, 2023-09-12T16:21:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-09-12T16:21:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-09-12T16:21:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 10, 2023-09-12T16:21:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2023-09-12T16:21:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-09-12T16:21:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-09-12T16:21:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2023-09-12T16:21:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 21351441, 2023-09-12T16:21:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2023-09-12T16:21:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-09-12T16:21:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3075, 2023-09-12T16:21:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 180.302943760, 2023-09-12T16:21:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.058635103 2023-09-12T16:21:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2023-09-12T16:21:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-09-12T16:21:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 751, 2023-09-12T16:21:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 1493, 2023-09-12T16:21:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 742, 2023-09-12T16:21:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 78, 2023-09-12T16:21:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 88, 2023-09-12T16:21:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 10, 2023-09-12T16:21:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 822, 2023-09-12T16:21:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 2686, 2023-09-12T16:21:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 1864, 2023-09-12T16:21:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 703, 2023-09-12T16:21:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 2284, 2023-09-12T16:21:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 1581, 2023-09-12T16:21:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 67864, 2023-09-12T16:21:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 207152 2023-09-12T16:21:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-09-12T16:21:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 7268, 2023-09-12T16:21:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 875, 2023-09-12T16:21:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 5155, 2023-09-12T16:21:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 1275, 2023-09-12T16:21:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 1442, 2023-09-12T16:21:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-09-12T16:21:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1313, 2023-09-12T16:21:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 52.413407607, 2023-09-12T16:21:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.039918817 2023-09-12T16:21:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-09-12T16:21:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 574, 2023-09-12T16:21:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.970816398, 2023-09-12T16:21:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005175638 2023-09-12T16:21:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-09-12T16:21:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-09-12T16:21:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 34, 2023-09-12T16:21:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.066030647, 2023-09-12T16:21:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001942077 2023-09-12T16:21:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-09-12T16:21:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 940, 2023-09-12T16:21:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 7.047453458, 2023-09-12T16:21:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.007497290 2023-09-12T16:21:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-09-12T16:21:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-09-12T16:21:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2023-09-12T16:21:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000162349, 2023-09-12T16:21:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000162349 2023-09-12T16:21:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-09-12T16:21:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-09-12T16:21:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-09-12T16:21:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-09-12T16:21:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-09-12T16:21:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 35, 2023-09-12T16:21:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.082175022, 2023-09-12T16:21:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002347857 2023-09-12T16:21:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-09-12T16:21:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-09-12T16:21:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-09-12T16:21:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 612, 2023-09-12T16:21:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.576524647, 2023-09-12T16:21:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000942033 2023-09-12T16:21:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-09-12T16:21:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 103, 2023-09-12T16:21:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.193212459, 2023-09-12T16:21:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001875849 2023-09-12T16:21:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-09-12T16:21:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 408, 2023-09-12T16:21:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.636669905, 2023-09-12T16:21:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.006462426 2023-09-12T16:21:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-09-12T16:21:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-09-12T16:21:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-09-12T16:21:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-09-12T16:21:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-09-12T16:21:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-09-12T16:21:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2023-09-12T16:21:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001744043, 2023-09-12T16:21:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000872021 2023-09-12T16:21:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-09-12T16:21:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-09-12T16:21:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-09-12T16:21:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-09-12T16:21:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 7, 2023-09-12T16:21:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.020345043, 2023-09-12T16:21:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002906434 2023-09-12T16:21:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-09-12T16:21:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-09-12T16:21:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 74, 2023-09-12T16:21:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.174426321, 2023-09-12T16:21:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002357112 2023-09-12T16:21:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-09-12T16:21:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-09-12T16:21:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-09-12T16:21:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-09-12T16:21:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2023-09-12T16:21:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 3, 2023-09-12T16:21:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 2, 2023-09-12T16:21:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2023-09-12T16:21:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-09-12T16:21:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 164, 2023-09-12T16:21:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 164, 2023-09-12T16:21:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 1066, 2023-09-12T16:21:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2023-09-12T16:21:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.388 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-09-12T16:21:31.388 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:31.388 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:31.388 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:31.388 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:31.389 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:31.389 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:31.389 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:31.389 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:31.389 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:31.389 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:31.390 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:31.390 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:31.390 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:31.390 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:31.390 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:31.391 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:31.391 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:31.391 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:31.391 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:31.391 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:31.392 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:31.392 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:31.392 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:31.392 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:31.392 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:31.392 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:31.393 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:31.393 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:31.393 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:31.393 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:31.393 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:31.394 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-09-12T16:21:31.394 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:31.394 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:31.394 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:31.394 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:31.395 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 334364, 2023-09-12T16:21:31.395 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 845, 2023-09-12T16:21:31.395 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 616, 2023-09-12T16:21:31.395 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 7, 2023-09-12T16:21:31.395 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-09-12T16:21:31.396 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-09-12T16:21:31.396 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:31.396 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:31.396 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:31.396 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:31.397 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8984, 2023-09-12T16:21:31.397 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 172, 2023-09-12T16:21:31.397 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:31.397 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:31.397 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:31.398 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-09-12T16:21:31.398 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 2707185, 2023-09-12T16:21:31.398 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 33632, 2023-09-12T16:21:31.398 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:31.398 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-09-12T16:21:31.399 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:31.399 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-09-12T16:21:31.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.399 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-09-12T16:21:31.399 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-09-12T16:21:31.399 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-09-12T16:21:31.400 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 5845, 2023-09-12T16:21:31.400 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 17864668, 2023-09-12T16:21:31.400 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 37, 2023-09-12T16:21:31.400 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 5845, 2023-09-12T16:21:31.400 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-09-12T16:21:31.401 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5845, 2023-09-12T16:21:31.401 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 267.720974163, 2023-09-12T16:21:31.401 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.045803417 2023-09-12T16:21:31.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.401 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-09-12T16:21:31.402 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-09-12T16:21:31.402 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5845, 2023-09-12T16:21:31.402 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 6167 2023-09-12T16:21:31.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.402 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 5845, 2023-09-12T16:21:31.403 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 854, 2023-09-12T16:21:31.403 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 4991, 2023-09-12T16:21:31.403 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-09-12T16:21:31.403 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-09-12T16:21:31.403 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2023-09-12T16:21:31.404 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 11, 2023-09-12T16:21:31.404 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 146, 2023-09-12T16:21:31.404 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 3219, 2023-09-12T16:21:31.404 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 195, 2023-09-12T16:21:31.404 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-09-12T16:21:31.405 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-09-12T16:21:31.405 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-09-12T16:21:31.405 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 3, 2023-09-12T16:21:31.405 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 1350, 2023-09-12T16:21:31.405 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-09-12T16:21:31.406 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-09-12T16:21:31.406 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-09-12T16:21:31.406 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 708, 2023-09-12T16:21:31.406 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 22, 2023-09-12T16:21:31.406 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-09-12T16:21:31.406 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-09-12T16:21:31.407 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-09-12T16:21:31.407 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-09-12T16:21:31.407 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-09-12T16:21:31.407 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-09-12T16:21:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-09-12T16:21:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-09-12T16:21:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-09-12T16:21:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-09-12T16:21:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-09-12T16:21:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-09-12T16:21:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-09-12T16:21:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-09-12T16:21:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-09-12T16:21:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-09-12T16:21:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-09-12T16:21:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-09-12T16:21:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-09-12T16:21:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-09-12T16:21:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-09-12T16:21:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-09-12T16:21:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-09-12T16:21:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-09-12T16:21:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-09-12T16:21:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-09-12T16:21:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 90, 2023-09-12T16:21:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-09-12T16:21:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 10, 2023-09-12T16:21:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2023-09-12T16:21:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2023-09-12T16:21:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2023-09-12T16:21:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-09-12T16:21:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 382, 2023-09-12T16:21:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2023-09-12T16:21:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 131 2023-09-12T16:21:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-09-12T16:21:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2023-09-12T16:21:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-09-12T16:21:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-09-12T16:21:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-09-12T16:21:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-09-12T16:21:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2023-09-12T16:21:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-09-12T16:21:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-09-12T16:21:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 240, 2023-09-12T16:21:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-09-12T16:21:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 8, 2023-09-12T16:21:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 298, 2023-09-12T16:21:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-09-12T16:21:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 68, 2023-09-12T16:21:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-09-12T16:21:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 3553912, 2023-09-12T16:21:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 248352841, 2023-09-12T16:21:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 3553912, 2023-09-12T16:21:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3553911, 2023-09-12T16:21:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 248352773, 2023-09-12T16:21:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-09-12T16:21:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-09-12T16:21:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 5845, 2023-09-12T16:21:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 17897150, 2023-09-12T16:21:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 5845, 2023-09-12T16:21:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 17897150, 2023-09-12T16:21:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-09-12T16:21:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-09-12T16:21:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 5845, 2023-09-12T16:21:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 5845, 2023-09-12T16:21:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 5845, 2023-09-12T16:21:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 5845, 2023-09-12T16:21:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-09-12T16:21:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-09-12T16:21:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 298, 2023-09-12T16:21:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 32482, 2023-09-12T16:21:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 298, 2023-09-12T16:21:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 144, 2023-09-12T16:21:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 32482, 2023-09-12T16:21:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55ac542a46e0": { 2023-09-12T16:21:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-09-12T16:21:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 6436, 2023-09-12T16:21:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 17157137, 2023-09-12T16:21:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 6436, 2023-09-12T16:21:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3075, 2023-09-12T16:21:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 17157137, 2023-09-12T16:21:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:31.434 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-09-12T16:21:31.755 INFO:tasks.check_counter:Daemon mds.b mds.exported=17 2023-09-12T16:21:31.755 INFO:tasks.check_counter:Daemon mds.b mds.imported=9 2023-09-12T16:21:31.755 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=98 2023-09-12T16:21:31.756 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2023-09-12T16:21:31.756 DEBUG:tasks.check_counter:Getting stats from e 2023-09-12T16:21:31.756 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.e perf dump 2023-09-12T16:21:31.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:31 smithi159 ceph-mon[102522]: pgmap v850: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2023-09-12T16:21:32.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:31 smithi130 ceph-mon[98670]: pgmap v850: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2023-09-12T16:21:32.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:31 smithi203 ceph-mon[103431]: pgmap v850: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2023-09-12T16:21:32.372 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T16:21:33.722 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-09-12T16:21:33.723 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:33.723 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1285, 2023-09-12T16:21:33.723 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1285, 2023-09-12T16:21:33.724 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1459691, 2023-09-12T16:21:33.724 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 398903, 2023-09-12T16:21:33.724 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 7, 2023-09-12T16:21:33.724 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2023-09-12T16:21:33.724 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.110833194, 2023-09-12T16:21:33.725 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.034866545, 2023-09-12T16:21:33.725 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.027789586, 2023-09-12T16:21:33.725 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.038874372, 2023-09-12T16:21:33.725 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:33.725 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1284, 2023-09-12T16:21:33.726 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.356598218, 2023-09-12T16:21:33.726 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000277724 2023-09-12T16:21:33.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.726 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:33.726 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.727 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.727 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.727 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-09-12T16:21:33.727 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 448 2023-09-12T16:21:33.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.728 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:33.728 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1364, 2023-09-12T16:21:33.728 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1353, 2023-09-12T16:21:33.728 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 801496, 2023-09-12T16:21:33.728 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 511898, 2023-09-12T16:21:33.729 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:33.729 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2023-09-12T16:21:33.729 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.130940365, 2023-09-12T16:21:33.729 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.033777950, 2023-09-12T16:21:33.729 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.037242477, 2023-09-12T16:21:33.729 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.051698931, 2023-09-12T16:21:33.730 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:33.730 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1352, 2023-09-12T16:21:33.730 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.367933283, 2023-09-12T16:21:33.730 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000272140 2023-09-12T16:21:33.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.731 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:33.731 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.731 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.732 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.732 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 548832, 2023-09-12T16:21:33.732 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 192496 2023-09-12T16:21:33.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.733 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:33.733 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1105, 2023-09-12T16:21:33.733 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1345, 2023-09-12T16:21:33.734 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 370928, 2023-09-12T16:21:33.734 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 919209, 2023-09-12T16:21:33.734 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 5, 2023-09-12T16:21:33.734 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2023-09-12T16:21:33.735 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.115541664, 2023-09-12T16:21:33.735 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.034039456, 2023-09-12T16:21:33.735 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.027702945, 2023-09-12T16:21:33.736 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.043088007, 2023-09-12T16:21:33.736 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:33.736 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1343, 2023-09-12T16:21:33.736 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.491860926, 2023-09-12T16:21:33.736 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000366240 2023-09-12T16:21:33.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.737 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:33.737 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.738 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.738 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.738 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.738 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-12T16:21:33.739 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 576176 2023-09-12T16:21:33.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.740 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-09-12T16:21:33.740 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-09-12T16:21:33.741 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-09-12T16:21:33.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.741 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-09-12T16:21:33.742 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-09-12T16:21:33.742 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-09-12T16:21:33.742 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5742, 2023-09-12T16:21:33.743 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.155951599, 2023-09-12T16:21:33.743 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000027159 2023-09-12T16:21:33.743 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:33.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.744 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-09-12T16:21:33.744 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-09-12T16:21:33.744 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-09-12T16:21:33.744 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.745 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.745 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.745 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:33.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.745 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-09-12T16:21:33.746 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2023-09-12T16:21:33.746 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2023-09-12T16:21:33.746 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-09-12T16:21:33.746 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.746 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.747 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.747 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-09-12T16:21:33.747 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2023-09-12T16:21:33.747 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2023-09-12T16:21:33.747 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-09-12T16:21:33.748 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2023-09-12T16:21:33.748 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2023-09-12T16:21:33.748 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2023-09-12T16:21:33.748 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 323, 2023-09-12T16:21:33.748 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2023-09-12T16:21:33.749 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 0, 2023-09-12T16:21:33.749 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 323, 2023-09-12T16:21:33.749 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 323, 2023-09-12T16:21:33.749 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2023-09-12T16:21:33.749 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2023-09-12T16:21:33.750 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2023-09-12T16:21:33.750 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 9, 2023-09-12T16:21:33.750 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2023-09-12T16:21:33.750 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2023-09-12T16:21:33.750 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2023-09-12T16:21:33.751 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2023-09-12T16:21:33.751 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2023-09-12T16:21:33.751 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2023-09-12T16:21:33.751 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2023-09-12T16:21:33.752 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2023-09-12T16:21:33.752 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-09-12T16:21:33.752 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2023-09-12T16:21:33.752 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2023-09-12T16:21:33.753 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2023-09-12T16:21:33.753 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2023-09-12T16:21:33.753 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-09-12T16:21:33.753 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2023-09-12T16:21:33.753 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2023-09-12T16:21:33.753 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 0, 2023-09-12T16:21:33.754 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 0, 2023-09-12T16:21:33.754 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2023-09-12T16:21:33.754 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2023-09-12T16:21:33.754 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-09-12T16:21:33.754 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-09-12T16:21:33.755 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-09-12T16:21:33.755 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2023-09-12T16:21:33.755 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2023-09-12T16:21:33.755 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2023-09-12T16:21:33.756 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2023-09-12T16:21:33.756 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2023-09-12T16:21:33.756 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2023-09-12T16:21:33.756 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2023-09-12T16:21:33.756 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-09-12T16:21:33.757 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-12T16:21:33.757 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-12T16:21:33.757 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2023-09-12T16:21:33.757 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2023-09-12T16:21:33.757 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2023-09-12T16:21:33.758 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2023-09-12T16:21:33.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.758 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-09-12T16:21:33.758 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2023-09-12T16:21:33.758 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2023-09-12T16:21:33.758 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2023-09-12T16:21:33.759 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2023-09-12T16:21:33.759 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2023-09-12T16:21:33.759 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2023-09-12T16:21:33.759 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-09-12T16:21:33.759 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-09-12T16:21:33.760 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2023-09-12T16:21:33.760 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2023-09-12T16:21:33.760 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2023-09-12T16:21:33.760 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-09-12T16:21:33.760 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-09-12T16:21:33.761 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-09-12T16:21:33.761 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-09-12T16:21:33.761 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-09-12T16:21:33.762 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-09-12T16:21:33.762 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2023-09-12T16:21:33.762 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2023-09-12T16:21:33.762 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-09-12T16:21:33.763 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2023-09-12T16:21:33.763 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-09-12T16:21:33.763 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-09-12T16:21:33.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.763 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-09-12T16:21:33.764 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2023-09-12T16:21:33.764 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2023-09-12T16:21:33.764 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-09-12T16:21:33.765 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-09-12T16:21:33.765 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 693, 2023-09-12T16:21:33.765 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-09-12T16:21:33.765 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-09-12T16:21:33.765 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2023-09-12T16:21:33.766 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-09-12T16:21:33.766 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-09-12T16:21:33.766 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2023-09-12T16:21:33.766 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2023-09-12T16:21:33.767 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-09-12T16:21:33.767 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-09-12T16:21:33.767 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2023-09-12T16:21:33.767 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 5317542, 2023-09-12T16:21:33.768 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 5316968, 2023-09-12T16:21:33.768 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-09-12T16:21:33.768 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.769 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.769 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.770 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 693 2023-09-12T16:21:33.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.770 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-09-12T16:21:33.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 322, 2023-09-12T16:21:33.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 322, 2023-09-12T16:21:33.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 0, 2023-09-12T16:21:33.771 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 16, 2023-09-12T16:21:33.772 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 16, 2023-09-12T16:21:33.772 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 0, 2023-09-12T16:21:33.772 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 323, 2023-09-12T16:21:33.773 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 323, 2023-09-12T16:21:33.773 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 0, 2023-09-12T16:21:33.773 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2023-09-12T16:21:33.773 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2023-09-12T16:21:33.774 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2023-09-12T16:21:33.774 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 50232, 2023-09-12T16:21:33.774 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 207152 2023-09-12T16:21:33.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.775 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-09-12T16:21:33.775 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2023-09-12T16:21:33.776 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2023-09-12T16:21:33.776 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2023-09-12T16:21:33.776 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2023-09-12T16:21:33.777 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2023-09-12T16:21:33.777 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-09-12T16:21:33.777 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.778 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.779 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-09-12T16:21:33.779 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.779 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.780 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-09-12T16:21:33.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.781 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.781 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.782 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-09-12T16:21:33.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.782 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.783 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-09-12T16:21:33.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.784 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.784 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.785 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-09-12T16:21:33.785 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.785 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.785 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.786 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-09-12T16:21:33.786 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-09-12T16:21:33.788 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.788 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.788 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.788 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-09-12T16:21:33.789 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.789 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.789 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.789 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-09-12T16:21:33.790 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.790 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.790 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.791 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-09-12T16:21:33.791 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.791 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.791 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.792 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-09-12T16:21:33.792 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.792 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.792 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.793 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-09-12T16:21:33.793 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.793 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.793 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.794 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-09-12T16:21:33.794 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.794 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.794 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.795 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-09-12T16:21:33.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.795 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.796 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-09-12T16:21:33.796 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.796 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.796 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.797 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-09-12T16:21:33.797 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.797 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.797 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.798 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-09-12T16:21:33.798 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.798 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.798 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.799 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-09-12T16:21:33.799 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.799 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.799 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.800 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-09-12T16:21:33.800 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.800 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.800 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.801 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-09-12T16:21:33.801 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.801 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.802 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.802 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-09-12T16:21:33.802 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.802 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.803 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-09-12T16:21:33.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.803 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.804 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.804 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-09-12T16:21:33.804 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.804 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.805 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.805 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-09-12T16:21:33.805 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.805 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.806 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.806 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-09-12T16:21:33.806 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.806 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.807 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.807 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-09-12T16:21:33.807 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.807 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.808 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.808 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-09-12T16:21:33.808 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.808 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.809 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.809 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-09-12T16:21:33.809 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.809 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.810 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.810 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-09-12T16:21:33.810 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-09-12T16:21:33.811 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-09-12T16:21:33.811 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.811 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.811 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.811 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:33.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.812 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-09-12T16:21:33.812 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2023-09-12T16:21:33.812 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 2, 2023-09-12T16:21:33.812 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 1, 2023-09-12T16:21:33.813 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 2, 2023-09-12T16:21:33.813 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-09-12T16:21:33.813 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2023-09-12T16:21:33.813 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2023-09-12T16:21:33.813 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 3689, 2023-09-12T16:21:33.814 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2023-09-12T16:21:33.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.814 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-09-12T16:21:33.814 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:33.814 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:33.815 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:33.815 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:33.815 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:33.817 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:33.817 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:33.817 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:33.817 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:33.817 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:33.818 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:33.818 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:33.818 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:33.818 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:33.818 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:33.819 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:33.819 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:33.819 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:33.819 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:33.819 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:33.820 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:33.820 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:33.820 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:33.820 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:33.820 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:33.821 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:33.821 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:33.821 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:33.821 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:33.821 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:33.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:33.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-09-12T16:21:33.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:33.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:33.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:33.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:33.823 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 201839, 2023-09-12T16:21:33.823 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 368, 2023-09-12T16:21:33.823 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:33.823 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:33.824 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-09-12T16:21:33.824 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-09-12T16:21:33.824 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:33.824 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:33.824 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:33.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:33.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8984, 2023-09-12T16:21:33.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 172, 2023-09-12T16:21:33.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:33.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:33.826 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:33.826 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-09-12T16:21:33.826 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 973202, 2023-09-12T16:21:33.826 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 9002, 2023-09-12T16:21:33.826 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:33.827 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-09-12T16:21:33.827 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:33.827 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-09-12T16:21:33.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.827 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-09-12T16:21:33.828 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-09-12T16:21:33.828 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-09-12T16:21:33.828 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 3411, 2023-09-12T16:21:33.828 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 21, 2023-09-12T16:21:33.828 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 3, 2023-09-12T16:21:33.829 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 3411, 2023-09-12T16:21:33.829 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-09-12T16:21:33.829 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3411, 2023-09-12T16:21:33.829 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 47.449109211, 2023-09-12T16:21:33.829 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.013910615 2023-09-12T16:21:33.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.830 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-09-12T16:21:33.830 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-09-12T16:21:33.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3411, 2023-09-12T16:21:33.830 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3412 2023-09-12T16:21:33.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.831 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 3411, 2023-09-12T16:21:33.831 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 3411, 2023-09-12T16:21:33.831 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2023-09-12T16:21:33.831 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-09-12T16:21:33.832 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-09-12T16:21:33.832 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 2128, 2023-09-12T16:21:33.832 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2023-09-12T16:21:33.832 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 1281, 2023-09-12T16:21:33.832 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2023-09-12T16:21:33.833 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2023-09-12T16:21:33.833 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-09-12T16:21:33.833 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-09-12T16:21:33.833 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-09-12T16:21:33.833 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2023-09-12T16:21:33.834 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2023-09-12T16:21:33.834 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-09-12T16:21:33.834 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-09-12T16:21:33.834 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-09-12T16:21:33.834 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 1, 2023-09-12T16:21:33.835 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2023-09-12T16:21:33.835 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-09-12T16:21:33.835 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-09-12T16:21:33.835 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-09-12T16:21:33.835 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-09-12T16:21:33.836 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-09-12T16:21:33.836 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-09-12T16:21:33.836 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-09-12T16:21:33.836 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-09-12T16:21:33.836 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-09-12T16:21:33.837 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-09-12T16:21:33.837 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-09-12T16:21:33.837 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-09-12T16:21:33.837 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-09-12T16:21:33.837 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-09-12T16:21:33.838 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-09-12T16:21:33.838 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-09-12T16:21:33.838 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-09-12T16:21:33.838 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-09-12T16:21:33.838 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-09-12T16:21:33.839 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-09-12T16:21:33.839 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-09-12T16:21:33.839 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-09-12T16:21:33.839 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-09-12T16:21:33.839 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-09-12T16:21:33.840 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-09-12T16:21:33.840 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-09-12T16:21:33.840 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 90, 2023-09-12T16:21:33.840 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-09-12T16:21:33.840 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 4, 2023-09-12T16:21:33.841 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 5, 2023-09-12T16:21:33.841 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 6, 2023-09-12T16:21:33.841 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 1, 2023-09-12T16:21:33.841 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-09-12T16:21:33.841 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2023-09-12T16:21:33.842 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2023-09-12T16:21:33.842 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2023-09-12T16:21:33.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.842 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-09-12T16:21:33.842 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2023-09-12T16:21:33.843 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-09-12T16:21:33.843 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-09-12T16:21:33.843 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-09-12T16:21:33.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.843 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-09-12T16:21:33.844 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2023-09-12T16:21:33.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.844 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-09-12T16:21:33.844 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-09-12T16:21:33.844 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2023-09-12T16:21:33.845 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-09-12T16:21:33.845 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2023-09-12T16:21:33.845 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2023-09-12T16:21:33.846 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-09-12T16:21:33.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.846 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:33.846 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:33.846 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-09-12T16:21:33.847 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:33.847 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 3750, 2023-09-12T16:21:33.847 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 2274273, 2023-09-12T16:21:33.847 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:33.847 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 3750, 2023-09-12T16:21:33.848 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:33.848 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:33.848 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3750, 2023-09-12T16:21:33.848 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 2274273, 2023-09-12T16:21:33.848 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:33.849 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.849 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.849 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.849 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-09-12T16:21:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-09-12T16:21:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 3411, 2023-09-12T16:21:33.852 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 1123243, 2023-09-12T16:21:33.852 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3411, 2023-09-12T16:21:33.852 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1123243, 2023-09-12T16:21:33.852 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:33.852 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.853 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.853 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:33.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.853 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-09-12T16:21:33.854 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:33.854 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-09-12T16:21:33.854 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:33.854 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:33.854 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:33.855 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:33.855 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:33.855 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 3411, 2023-09-12T16:21:33.855 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 3411, 2023-09-12T16:21:33.855 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3411, 2023-09-12T16:21:33.856 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 3411, 2023-09-12T16:21:33.856 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:33.856 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.856 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.856 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.857 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:33.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.857 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-09-12T16:21:33.857 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:33.857 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-09-12T16:21:33.858 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:33.858 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:33.858 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:33.858 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:33.859 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:33.859 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:33.859 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:33.859 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-09-12T16:21:33.859 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-09-12T16:21:33.860 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:33.860 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.860 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.860 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.860 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:33.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:33.861 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x5558ee080960": { 2023-09-12T16:21:33.861 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:33.861 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-09-12T16:21:33.861 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:33.862 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:33.862 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:33.862 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:33.862 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:33.862 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:33.863 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:33.863 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-09-12T16:21:33.863 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-09-12T16:21:33.863 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:33.864 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:33.864 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:33.864 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:33.864 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:33.864 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:33.864 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-09-12T16:21:34.098 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:33 smithi159 ceph-mon[102522]: pgmap v851: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-09-12T16:21:34.136 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-09-12T16:21:34.137 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-09-12T16:21:34.137 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-09-12T16:21:34.137 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-09-12T16:21:34.137 DEBUG:tasks.check_counter:Getting stats from h 2023-09-12T16:21:34.138 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.h perf dump 2023-09-12T16:21:34.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:33 smithi130 ceph-mon[98670]: pgmap v851: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-09-12T16:21:34.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:33 smithi203 ceph-mon[103431]: pgmap v851: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-09-12T16:21:34.776 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T16:21:36.125 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-09-12T16:21:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2073, 2023-09-12T16:21:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2075, 2023-09-12T16:21:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 9007727, 2023-09-12T16:21:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 644710, 2023-09-12T16:21:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2023-09-12T16:21:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2023-09-12T16:21:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.176621977, 2023-09-12T16:21:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.048062049, 2023-09-12T16:21:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.048658304, 2023-09-12T16:21:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.065848138, 2023-09-12T16:21:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2073, 2023-09-12T16:21:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.556251237, 2023-09-12T16:21:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000268331 2023-09-12T16:21:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-12T16:21:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2023-09-12T16:21:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1057, 2023-09-12T16:21:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1294, 2023-09-12T16:21:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 4464706, 2023-09-12T16:21:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 904955, 2023-09-12T16:21:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2023-09-12T16:21:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2023-09-12T16:21:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.099761028, 2023-09-12T16:21:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.027181957, 2023-09-12T16:21:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.027326763, 2023-09-12T16:21:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.036867394, 2023-09-12T16:21:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1293, 2023-09-12T16:21:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.334039451, 2023-09-12T16:21:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000258344 2023-09-12T16:21:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-09-12T16:21:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 578032 2023-09-12T16:21:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 890, 2023-09-12T16:21:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 878, 2023-09-12T16:21:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1195753, 2023-09-12T16:21:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 363855, 2023-09-12T16:21:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 5, 2023-09-12T16:21:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2023-09-12T16:21:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.091197031, 2023-09-12T16:21:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.024891999, 2023-09-12T16:21:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.029696745, 2023-09-12T16:21:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.032060891, 2023-09-12T16:21:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 877, 2023-09-12T16:21:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.268039991, 2023-09-12T16:21:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000305632 2023-09-12T16:21:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 552672, 2023-09-12T16:21:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 191872 2023-09-12T16:21:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-09-12T16:21:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-09-12T16:21:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-09-12T16:21:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-09-12T16:21:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-09-12T16:21:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-09-12T16:21:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6256, 2023-09-12T16:21:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.154695585, 2023-09-12T16:21:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000024727 2023-09-12T16:21:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-09-12T16:21:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-09-12T16:21:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-09-12T16:21:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-09-12T16:21:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2023-09-12T16:21:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2023-09-12T16:21:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-09-12T16:21:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-09-12T16:21:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2023-09-12T16:21:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2023-09-12T16:21:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-09-12T16:21:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2023-09-12T16:21:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2023-09-12T16:21:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2023-09-12T16:21:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 1977, 2023-09-12T16:21:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2023-09-12T16:21:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 101, 2023-09-12T16:21:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 1876, 2023-09-12T16:21:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 1977, 2023-09-12T16:21:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2023-09-12T16:21:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2023-09-12T16:21:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2023-09-12T16:21:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 9, 2023-09-12T16:21:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2023-09-12T16:21:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2023-09-12T16:21:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2023-09-12T16:21:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2023-09-12T16:21:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2023-09-12T16:21:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2023-09-12T16:21:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2023-09-12T16:21:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2023-09-12T16:21:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-09-12T16:21:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2023-09-12T16:21:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2023-09-12T16:21:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2023-09-12T16:21:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2023-09-12T16:21:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-09-12T16:21:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2023-09-12T16:21:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2023-09-12T16:21:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 0, 2023-09-12T16:21:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 0, 2023-09-12T16:21:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2023-09-12T16:21:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2023-09-12T16:21:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-09-12T16:21:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-09-12T16:21:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-09-12T16:21:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2023-09-12T16:21:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2023-09-12T16:21:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2023-09-12T16:21:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2023-09-12T16:21:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2023-09-12T16:21:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2023-09-12T16:21:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2023-09-12T16:21:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-09-12T16:21:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-12T16:21:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-12T16:21:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2023-09-12T16:21:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2023-09-12T16:21:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2023-09-12T16:21:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2023-09-12T16:21:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-09-12T16:21:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2023-09-12T16:21:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2023-09-12T16:21:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2023-09-12T16:21:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2023-09-12T16:21:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2023-09-12T16:21:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2023-09-12T16:21:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-09-12T16:21:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-09-12T16:21:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2023-09-12T16:21:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2023-09-12T16:21:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2023-09-12T16:21:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-09-12T16:21:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-09-12T16:21:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-09-12T16:21:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-09-12T16:21:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-09-12T16:21:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-09-12T16:21:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2023-09-12T16:21:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2023-09-12T16:21:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-09-12T16:21:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2023-09-12T16:21:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-09-12T16:21:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-09-12T16:21:36.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.158 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-09-12T16:21:36.158 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2023-09-12T16:21:36.158 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2023-09-12T16:21:36.158 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-09-12T16:21:36.159 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-09-12T16:21:36.159 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 7462, 2023-09-12T16:21:36.159 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-09-12T16:21:36.159 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-09-12T16:21:36.159 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2023-09-12T16:21:36.159 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-09-12T16:21:36.160 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-09-12T16:21:36.160 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 11, 2023-09-12T16:21:36.160 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2023-09-12T16:21:36.160 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-09-12T16:21:36.160 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-09-12T16:21:36.161 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2023-09-12T16:21:36.161 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 17283677, 2023-09-12T16:21:36.161 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 17275436, 2023-09-12T16:21:36.161 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-09-12T16:21:36.161 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.161 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.162 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.162 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 7462 2023-09-12T16:21:36.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.162 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-09-12T16:21:36.162 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 85, 2023-09-12T16:21:36.163 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 227, 2023-09-12T16:21:36.163 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 142, 2023-09-12T16:21:36.163 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 20, 2023-09-12T16:21:36.163 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 34, 2023-09-12T16:21:36.163 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 14, 2023-09-12T16:21:36.164 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 1876, 2023-09-12T16:21:36.164 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 2007, 2023-09-12T16:21:36.164 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 131, 2023-09-12T16:21:36.164 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2023-09-12T16:21:36.164 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2023-09-12T16:21:36.164 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2023-09-12T16:21:36.165 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 53952, 2023-09-12T16:21:36.165 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 207152 2023-09-12T16:21:36.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.165 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-09-12T16:21:36.165 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2023-09-12T16:21:36.165 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2023-09-12T16:21:36.166 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2023-09-12T16:21:36.166 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2023-09-12T16:21:36.166 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2023-09-12T16:21:36.166 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-09-12T16:21:36.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.167 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.167 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.167 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-09-12T16:21:36.167 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.167 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.168 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.168 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-09-12T16:21:36.168 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.168 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.169 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.169 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-09-12T16:21:36.169 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.169 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.169 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.170 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-09-12T16:21:36.170 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.170 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.170 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.171 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-09-12T16:21:36.171 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.171 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.171 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.172 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-09-12T16:21:36.172 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.172 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.172 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.172 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-09-12T16:21:36.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.173 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.173 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-09-12T16:21:36.174 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.174 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.174 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.175 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-09-12T16:21:36.175 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.175 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.175 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.176 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-09-12T16:21:36.176 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.176 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.176 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.177 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-09-12T16:21:36.177 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.177 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.177 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.178 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-09-12T16:21:36.178 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.178 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.178 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.179 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-09-12T16:21:36.179 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.179 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.179 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.179 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-09-12T16:21:36.180 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.180 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.180 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.180 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-09-12T16:21:36.181 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.181 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.181 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.181 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-09-12T16:21:36.181 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.182 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.182 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.182 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-09-12T16:21:36.182 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.183 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.183 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.183 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-09-12T16:21:36.183 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.184 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.184 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.184 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-09-12T16:21:36.184 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.184 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.185 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.185 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-09-12T16:21:36.185 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.185 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.185 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.186 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-09-12T16:21:36.186 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.186 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.186 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.187 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-09-12T16:21:36.187 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.187 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.187 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.188 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-09-12T16:21:36.188 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.188 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.188 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.189 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-09-12T16:21:36.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.189 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.189 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-09-12T16:21:36.190 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.190 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.190 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.190 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-09-12T16:21:36.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.191 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.191 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-09-12T16:21:36.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.192 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.192 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-09-12T16:21:36.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.193 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-09-12T16:21:36.193 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-09-12T16:21:36.193 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-09-12T16:21:36.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.194 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.194 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:36.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.195 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-09-12T16:21:36.195 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2023-09-12T16:21:36.195 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 2, 2023-09-12T16:21:36.195 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 1, 2023-09-12T16:21:36.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2023-09-12T16:21:36.196 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-09-12T16:21:36.196 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2023-09-12T16:21:36.196 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2023-09-12T16:21:36.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 2420, 2023-09-12T16:21:36.196 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2023-09-12T16:21:36.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.197 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-09-12T16:21:36.197 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:36.197 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:36.197 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:36.197 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:36.198 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:36.198 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:36.198 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:36.198 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:36.198 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:36.199 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:36.199 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:36.199 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:36.199 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:36.199 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:36.200 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:36.200 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:36.200 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:36.200 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:36.200 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:36.200 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:36.201 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:36.201 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:36.201 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:36.201 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:36.201 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:36.202 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:36.202 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:36.202 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:36.202 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:36.202 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:36.202 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:36.203 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-09-12T16:21:36.203 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:36.203 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:36.203 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:36.203 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:36.204 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 205032, 2023-09-12T16:21:36.204 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 201, 2023-09-12T16:21:36.204 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:36.204 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:36.204 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-09-12T16:21:36.204 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-09-12T16:21:36.205 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:36.205 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:36.205 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:36.205 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:36.205 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8984, 2023-09-12T16:21:36.206 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 172, 2023-09-12T16:21:36.206 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:36.206 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:36.206 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:36.206 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-09-12T16:21:36.206 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 1809109, 2023-09-12T16:21:36.207 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 49422, 2023-09-12T16:21:36.207 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:36.207 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-09-12T16:21:36.207 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:36.207 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-09-12T16:21:36.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.208 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-09-12T16:21:36.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-09-12T16:21:36.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-09-12T16:21:36.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 3677, 2023-09-12T16:21:36.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 21, 2023-09-12T16:21:36.209 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 3, 2023-09-12T16:21:36.209 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 3677, 2023-09-12T16:21:36.209 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-09-12T16:21:36.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3677, 2023-09-12T16:21:36.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 49.732066249, 2023-09-12T16:21:36.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.013525174 2023-09-12T16:21:36.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.210 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-09-12T16:21:36.210 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-09-12T16:21:36.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3677, 2023-09-12T16:21:36.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3678 2023-09-12T16:21:36.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.211 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 3677, 2023-09-12T16:21:36.211 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 3677, 2023-09-12T16:21:36.211 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2023-09-12T16:21:36.212 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-09-12T16:21:36.212 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-09-12T16:21:36.212 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 2125, 2023-09-12T16:21:36.212 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2023-09-12T16:21:36.212 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 1550, 2023-09-12T16:21:36.212 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2023-09-12T16:21:36.213 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2023-09-12T16:21:36.213 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-09-12T16:21:36.213 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-09-12T16:21:36.213 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-09-12T16:21:36.213 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2023-09-12T16:21:36.214 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2023-09-12T16:21:36.214 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-09-12T16:21:36.214 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-09-12T16:21:36.214 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-09-12T16:21:36.214 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 1, 2023-09-12T16:21:36.214 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2023-09-12T16:21:36.215 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-09-12T16:21:36.215 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-09-12T16:21:36.215 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-09-12T16:21:36.215 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-09-12T16:21:36.215 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-09-12T16:21:36.216 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-09-12T16:21:36.216 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-09-12T16:21:36.216 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-09-12T16:21:36.216 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-09-12T16:21:36.216 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-09-12T16:21:36.216 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-09-12T16:21:36.217 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-09-12T16:21:36.217 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-09-12T16:21:36.217 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-09-12T16:21:36.217 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-09-12T16:21:36.217 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-09-12T16:21:36.217 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-09-12T16:21:36.218 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-09-12T16:21:36.218 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-09-12T16:21:36.218 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-09-12T16:21:36.218 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-09-12T16:21:36.218 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-09-12T16:21:36.219 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-09-12T16:21:36.219 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-09-12T16:21:36.219 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-09-12T16:21:36.219 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-09-12T16:21:36.219 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 90, 2023-09-12T16:21:36.220 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-09-12T16:21:36.220 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 4, 2023-09-12T16:21:36.220 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 6, 2023-09-12T16:21:36.220 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 7, 2023-09-12T16:21:36.220 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 1, 2023-09-12T16:21:36.220 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-09-12T16:21:36.221 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2023-09-12T16:21:36.221 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2023-09-12T16:21:36.221 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2023-09-12T16:21:36.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.221 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-09-12T16:21:36.222 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2023-09-12T16:21:36.222 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-09-12T16:21:36.222 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-09-12T16:21:36.222 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-09-12T16:21:36.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-09-12T16:21:36.223 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2023-09-12T16:21:36.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.223 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-09-12T16:21:36.223 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-09-12T16:21:36.223 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2023-09-12T16:21:36.224 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-09-12T16:21:36.224 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2023-09-12T16:21:36.224 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2023-09-12T16:21:36.224 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-09-12T16:21:36.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.225 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:36.225 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:36.225 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-09-12T16:21:36.225 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:36.225 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 4016, 2023-09-12T16:21:36.225 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 14286409, 2023-09-12T16:21:36.226 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:36.226 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 4016, 2023-09-12T16:21:36.226 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:36.226 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:36.226 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4016, 2023-09-12T16:21:36.226 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 14286409, 2023-09-12T16:21:36.227 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:36.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.227 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.227 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:36.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.228 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-09-12T16:21:36.228 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:36.228 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-09-12T16:21:36.228 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:36.228 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:36.229 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:36.229 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:36.229 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:36.229 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 3677, 2023-09-12T16:21:36.229 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 13089378, 2023-09-12T16:21:36.230 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3677, 2023-09-12T16:21:36.230 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 13089378, 2023-09-12T16:21:36.230 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:36.230 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.230 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.230 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.231 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:36.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.231 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-09-12T16:21:36.231 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:36.231 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-09-12T16:21:36.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:36.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:36.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:36.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:36.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:36.232 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 3677, 2023-09-12T16:21:36.233 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 3677, 2023-09-12T16:21:36.233 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3677, 2023-09-12T16:21:36.233 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 3677, 2023-09-12T16:21:36.233 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:36.233 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.234 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.234 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.234 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:36.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.234 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-09-12T16:21:36.234 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:36.235 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-09-12T16:21:36.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:36.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:36.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:36.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:36.236 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:36.236 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:36.236 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:36.236 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-09-12T16:21:36.236 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-09-12T16:21:36.236 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:36.237 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.237 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.237 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.237 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:36.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:36.238 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55c0d6328960": { 2023-09-12T16:21:36.238 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:36.238 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-09-12T16:21:36.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:36.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-09-12T16:21:36.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-09-12T16:21:36.239 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:36.239 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:36.239 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:36.239 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:36.239 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-09-12T16:21:36.240 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-09-12T16:21:36.240 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:36.240 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:36.240 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:36.240 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:36.240 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:36.241 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:36.241 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-09-12T16:21:36.242 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:35 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:21:36.242 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:35 smithi130 ceph-mon[98670]: pgmap v852: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2023-09-12T16:21:36.242 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:35 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:21:36.242 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:35 smithi159 ceph-mon[102522]: pgmap v852: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2023-09-12T16:21:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:35 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:21:36.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:35 smithi203 ceph-mon[103431]: pgmap v852: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2023-09-12T16:21:36.636 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-09-12T16:21:36.637 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-09-12T16:21:36.637 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-09-12T16:21:36.637 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-09-12T16:21:36.637 DEBUG:tasks.check_counter:Getting stats from k 2023-09-12T16:21:36.638 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.k perf dump 2023-09-12T16:21:37.236 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-12T16:21:38.664 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-09-12T16:21:38.665 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:38.665 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 323, 2023-09-12T16:21:38.665 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 320, 2023-09-12T16:21:38.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 500160, 2023-09-12T16:21:38.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 192832, 2023-09-12T16:21:38.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:38.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 0, 2023-09-12T16:21:38.667 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.043929761, 2023-09-12T16:21:38.667 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.015394377, 2023-09-12T16:21:38.667 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.017400081, 2023-09-12T16:21:38.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.009247213, 2023-09-12T16:21:38.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:38.668 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 319, 2023-09-12T16:21:38.668 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.091977507, 2023-09-12T16:21:38.669 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000288330 2023-09-12T16:21:38.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.669 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:38.669 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:38.670 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:38.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:38.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.670 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 500160, 2023-09-12T16:21:38.671 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 192832 2023-09-12T16:21:38.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.671 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:38.672 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2, 2023-09-12T16:21:38.672 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 245, 2023-09-12T16:21:38.672 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 320, 2023-09-12T16:21:38.672 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 195984, 2023-09-12T16:21:38.672 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:38.673 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2023-09-12T16:21:38.673 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.010823649, 2023-09-12T16:21:38.673 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.007712891, 2023-09-12T16:21:38.673 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.001157574, 2023-09-12T16:21:38.673 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:38.674 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:38.674 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 243, 2023-09-12T16:21:38.674 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.057568375, 2023-09-12T16:21:38.674 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000236906 2023-09-12T16:21:38.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.675 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:38.675 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:38.675 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:38.675 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:38.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.676 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-12T16:21:38.676 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 195984 2023-09-12T16:21:38.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.676 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:38.676 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 4, 2023-09-12T16:21:38.677 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 4, 2023-09-12T16:21:38.677 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 2992, 2023-09-12T16:21:38.677 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 448, 2023-09-12T16:21:38.677 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 2, 2023-09-12T16:21:38.678 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 1, 2023-09-12T16:21:38.679 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.000716038, 2023-09-12T16:21:38.680 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.000041479, 2023-09-12T16:21:38.680 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.000641909, 2023-09-12T16:21:38.680 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:38.680 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:38.681 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3, 2023-09-12T16:21:38.681 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000746227, 2023-09-12T16:21:38.681 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000248742 2023-09-12T16:21:38.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.681 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:38.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:38.682 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:38.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:38.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.683 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-09-12T16:21:38.683 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 448 2023-09-12T16:21:38.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.683 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-09-12T16:21:38.683 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 0, 2023-09-12T16:21:38.683 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-09-12T16:21:38.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.684 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-09-12T16:21:38.684 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:38.684 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:38.684 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:38.685 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:38.685 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:38.685 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:38.685 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:38.685 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:38.686 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:38.686 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:38.686 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:38.686 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:38.686 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:38.686 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:38.687 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:38.687 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:38.687 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:38.687 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:38.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:38.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:38.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:38.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:38.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:38.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:38.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:38.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:38.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:38.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:38.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-09-12T16:21:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 148738, 2023-09-12T16:21:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 31, 2023-09-12T16:21:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-09-12T16:21:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-09-12T16:21:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 0, 2023-09-12T16:21:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 0, 2023-09-12T16:21:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-09-12T16:21:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 928, 2023-09-12T16:21:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 2, 2023-09-12T16:21:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-09-12T16:21:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-09-12T16:21:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-09-12T16:21:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-09-12T16:21:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-09-12T16:21:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-09-12T16:21:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 325, 2023-09-12T16:21:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 454511, 2023-09-12T16:21:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 325, 2023-09-12T16:21:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-09-12T16:21:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-09-12T16:21:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 325, 2023-09-12T16:21:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 454511, 2023-09-12T16:21:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-09-12T16:21:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-09-12T16:21:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-09-12T16:21:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-09-12T16:21:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-09-12T16:21:38.699 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-09-12T16:21:38.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:38 smithi159 ceph-mon[102522]: pgmap v853: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 KiB/s wr, 8 op/s 2023-09-12T16:21:38.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:38 smithi130 ceph-mon[98670]: pgmap v853: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 KiB/s wr, 8 op/s 2023-09-12T16:21:38.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:38 smithi203 ceph-mon[103431]: pgmap v853: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 KiB/s wr, 8 op/s 2023-09-12T16:21:39.092 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-09-12T16:21:39.093 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-09-12T16:21:39.093 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-09-12T16:21:39.093 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-09-12T16:21:39.094 DEBUG:tasks.check_counter:Getting stats from c 2023-09-12T16:21:39.094 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.c perf dump 2023-09-12T16:21:39.764 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T16:21:39.970 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:39 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:21:39.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:39 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:21:39.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:39 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:21:40.976 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:40 smithi130 ceph-mon[98670]: pgmap v854: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-09-12T16:21:40.977 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:40 smithi159 ceph-mon[102522]: pgmap v854: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-09-12T16:21:40.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:40 smithi203 ceph-mon[103431]: pgmap v854: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-09-12T16:21:41.436 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-09-12T16:21:41.436 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:41.436 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 453, 2023-09-12T16:21:41.436 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 457, 2023-09-12T16:21:41.437 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 112675, 2023-09-12T16:21:41.437 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 1064412, 2023-09-12T16:21:41.437 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 13, 2023-09-12T16:21:41.437 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 6, 2023-09-12T16:21:41.438 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.038172607, 2023-09-12T16:21:41.438 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.010305416, 2023-09-12T16:21:41.438 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.010144622, 2023-09-12T16:21:41.438 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.013539306, 2023-09-12T16:21:41.438 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:41.439 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 453, 2023-09-12T16:21:41.439 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.110781647, 2023-09-12T16:21:41.439 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000244551 2023-09-12T16:21:41.439 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.440 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:41.440 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 9, 2023-09-12T16:21:41.440 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000001129, 2023-09-12T16:21:41.440 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000125 2023-09-12T16:21:41.440 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.441 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 912, 2023-09-12T16:21:41.441 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 688 2023-09-12T16:21:41.441 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.441 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:41.442 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1564, 2023-09-12T16:21:41.442 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1989, 2023-09-12T16:21:41.442 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 956910, 2023-09-12T16:21:41.442 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 1147552, 2023-09-12T16:21:41.442 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 12, 2023-09-12T16:21:41.443 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 7, 2023-09-12T16:21:41.443 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.141072159, 2023-09-12T16:21:41.443 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.053399589, 2023-09-12T16:21:41.443 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.092023561, 2023-09-12T16:21:41.444 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.010904876, 2023-09-12T16:21:41.444 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:41.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1987, 2023-09-12T16:21:41.444 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.282322721, 2023-09-12T16:21:41.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000142084 2023-09-12T16:21:41.445 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.445 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:41.445 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2403, 2023-09-12T16:21:41.445 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000454464, 2023-09-12T16:21:41.446 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000189 2023-09-12T16:21:41.446 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.446 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 930816, 2023-09-12T16:21:41.446 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 1045312 2023-09-12T16:21:41.447 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.447 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:41.447 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 3581178, 2023-09-12T16:21:41.447 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 3580051, 2023-09-12T16:21:41.447 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 581070054, 2023-09-12T16:21:41.448 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 566483996, 2023-09-12T16:21:41.448 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 8, 2023-09-12T16:21:41.448 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2023-09-12T16:21:41.448 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 112.553521974, 2023-09-12T16:21:41.449 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 42.387516065, 2023-09-12T16:21:41.449 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 81.057051510, 2023-09-12T16:21:41.449 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.005799077, 2023-09-12T16:21:41.449 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:41.449 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 3580053, 2023-09-12T16:21:41.450 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 96.937212641, 2023-09-12T16:21:41.450 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000027077 2023-09-12T16:21:41.451 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.451 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:41.451 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 7159708, 2023-09-12T16:21:41.451 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.664734394, 2023-09-12T16:21:41.452 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000092 2023-09-12T16:21:41.452 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.452 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 160, 2023-09-12T16:21:41.452 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 500368 2023-09-12T16:21:41.453 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.453 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2023-09-12T16:21:41.453 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2023-09-12T16:21:41.453 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2023-09-12T16:21:41.453 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.454 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2023-09-12T16:21:41.454 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-09-12T16:21:41.454 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-09-12T16:21:41.455 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 685, 2023-09-12T16:21:41.455 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.128032310, 2023-09-12T16:21:41.455 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000186908 2023-09-12T16:21:41.455 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:41.455 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.455 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2023-09-12T16:21:41.456 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-09-12T16:21:41.456 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-09-12T16:21:41.456 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2, 2023-09-12T16:21:41.456 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000016705, 2023-09-12T16:21:41.456 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000008352 2023-09-12T16:21:41.457 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:41.457 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.457 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2023-09-12T16:21:41.457 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 751, 2023-09-12T16:21:41.457 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 265, 2023-09-12T16:21:41.457 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2023-09-12T16:21:41.458 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 265, 2023-09-12T16:21:41.458 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.176933845, 2023-09-12T16:21:41.458 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000667674 2023-09-12T16:21:41.458 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.458 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2023-09-12T16:21:41.459 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 486, 2023-09-12T16:21:41.459 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2023-09-12T16:21:41.459 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2023-09-12T16:21:41.459 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 11, 2023-09-12T16:21:41.459 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2023-09-12T16:21:41.460 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2023-09-12T16:21:41.460 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 346, 2023-09-12T16:21:41.460 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 7, 2023-09-12T16:21:41.460 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 339, 2023-09-12T16:21:41.460 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 0, 2023-09-12T16:21:41.460 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 334, 2023-09-12T16:21:41.461 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2023-09-12T16:21:41.461 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 313, 2023-09-12T16:21:41.461 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 313, 2023-09-12T16:21:41.461 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 9, 2023-09-12T16:21:41.461 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 1119, 2023-09-12T16:21:41.462 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 602, 2023-09-12T16:21:41.462 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 486, 2023-09-12T16:21:41.462 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 24, 2023-09-12T16:21:41.462 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2023-09-12T16:21:41.462 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2023-09-12T16:21:41.462 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2023-09-12T16:21:41.463 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 3411, 2023-09-12T16:21:41.463 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2023-09-12T16:21:41.463 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2023-09-12T16:21:41.463 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2023-09-12T16:21:41.463 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 13, 2023-09-12T16:21:41.464 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 441, 2023-09-12T16:21:41.464 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2023-09-12T16:21:41.464 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 221, 2023-09-12T16:21:41.464 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2023-09-12T16:21:41.464 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 1046, 2023-09-12T16:21:41.464 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 9078222795, 2023-09-12T16:21:41.465 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2023-09-12T16:21:41.465 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 221, 2023-09-12T16:21:41.465 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2023-09-12T16:21:41.466 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2023-09-12T16:21:41.466 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2023-09-12T16:21:41.466 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2023-09-12T16:21:41.466 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2023-09-12T16:21:41.466 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2023-09-12T16:21:41.467 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 221, 2023-09-12T16:21:41.467 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2023-09-12T16:21:41.467 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 1, 2023-09-12T16:21:41.467 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 37, 2023-09-12T16:21:41.467 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2023-09-12T16:21:41.468 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-12T16:21:41.468 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-12T16:21:41.468 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 223, 2023-09-12T16:21:41.468 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 198, 2023-09-12T16:21:41.468 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 9, 2023-09-12T16:21:41.468 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 6 2023-09-12T16:21:41.469 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.469 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2023-09-12T16:21:41.469 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 5, 2023-09-12T16:21:41.469 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 234, 2023-09-12T16:21:41.469 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 127, 2023-09-12T16:21:41.470 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 196, 2023-09-12T16:21:41.470 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 5, 2023-09-12T16:21:41.470 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2023-09-12T16:21:41.470 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2023-09-12T16:21:41.470 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2023-09-12T16:21:41.470 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2023-09-12T16:21:41.471 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2023-09-12T16:21:41.471 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2023-09-12T16:21:41.471 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2023-09-12T16:21:41.471 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2023-09-12T16:21:41.472 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2023-09-12T16:21:41.472 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2023-09-12T16:21:41.472 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2023-09-12T16:21:41.472 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2023-09-12T16:21:41.472 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2023-09-12T16:21:41.473 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2023-09-12T16:21:41.473 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2023-09-12T16:21:41.473 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2023-09-12T16:21:41.473 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2023-09-12T16:21:41.473 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2023-09-12T16:21:41.474 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.474 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2023-09-12T16:21:41.474 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2023-09-12T16:21:41.474 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 705, 2023-09-12T16:21:41.474 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2023-09-12T16:21:41.475 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2023-09-12T16:21:41.475 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 705, 2023-09-12T16:21:41.475 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2023-09-12T16:21:41.475 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2023-09-12T16:21:41.475 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 2, 2023-09-12T16:21:41.475 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2023-09-12T16:21:41.476 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2023-09-12T16:21:41.476 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 2, 2023-09-12T16:21:41.476 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2023-09-12T16:21:41.476 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2023-09-12T16:21:41.476 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2023-09-12T16:21:41.476 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2023-09-12T16:21:41.477 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 5438584, 2023-09-12T16:21:41.477 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 0, 2023-09-12T16:21:41.477 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2023-09-12T16:21:41.477 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 243, 2023-09-12T16:21:41.477 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 10.834576262, 2023-09-12T16:21:41.478 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.044586733 2023-09-12T16:21:41.478 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.478 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 0 2023-09-12T16:21:41.478 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.478 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2023-09-12T16:21:41.479 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 472, 2023-09-12T16:21:41.479 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 472, 2023-09-12T16:21:41.479 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 0, 2023-09-12T16:21:41.479 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 49, 2023-09-12T16:21:41.479 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 127, 2023-09-12T16:21:41.479 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 78, 2023-09-12T16:21:41.480 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 474, 2023-09-12T16:21:41.480 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 477, 2023-09-12T16:21:41.480 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 3, 2023-09-12T16:21:41.480 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 442, 2023-09-12T16:21:41.480 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 754, 2023-09-12T16:21:41.480 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 312, 2023-09-12T16:21:41.481 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 57580, 2023-09-12T16:21:41.481 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 198960 2023-09-12T16:21:41.481 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.481 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2023-09-12T16:21:41.481 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 758, 2023-09-12T16:21:41.482 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2023-09-12T16:21:41.482 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 751, 2023-09-12T16:21:41.482 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 18, 2023-09-12T16:21:41.482 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 1, 2023-09-12T16:21:41.482 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2023-09-12T16:21:41.482 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2, 2023-09-12T16:21:41.483 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.009255432, 2023-09-12T16:21:41.483 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004627716 2023-09-12T16:21:41.483 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.483 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2023-09-12T16:21:41.483 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.484 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.484 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.484 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.484 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2023-09-12T16:21:41.485 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.485 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.485 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.485 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.485 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2023-09-12T16:21:41.486 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.486 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.486 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.486 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.486 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2023-09-12T16:21:41.487 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 235, 2023-09-12T16:21:41.487 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.149275975, 2023-09-12T16:21:41.487 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000635216 2023-09-12T16:21:41.487 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.487 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2023-09-12T16:21:41.488 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.488 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.488 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.488 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.488 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2023-09-12T16:21:41.488 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1, 2023-09-12T16:21:41.489 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000190689, 2023-09-12T16:21:41.489 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000190689 2023-09-12T16:21:41.489 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.489 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2023-09-12T16:21:41.489 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.490 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.490 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.490 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.490 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2023-09-12T16:21:41.490 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.491 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.491 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.491 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.492 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2023-09-12T16:21:41.492 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.492 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.492 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.492 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.493 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2023-09-12T16:21:41.493 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.493 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.493 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.493 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.494 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2023-09-12T16:21:41.494 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.494 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.494 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.494 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.494 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2023-09-12T16:21:41.495 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.495 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.495 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.495 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.495 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2023-09-12T16:21:41.496 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.496 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.496 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.496 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.496 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2023-09-12T16:21:41.496 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 20, 2023-09-12T16:21:41.497 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.011484044, 2023-09-12T16:21:41.497 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000574202 2023-09-12T16:21:41.497 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.497 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2023-09-12T16:21:41.498 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 7, 2023-09-12T16:21:41.498 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.006727705, 2023-09-12T16:21:41.498 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000961100 2023-09-12T16:21:41.498 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.499 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2023-09-12T16:21:41.499 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.499 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.499 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.499 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.499 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2023-09-12T16:21:41.500 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.500 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.500 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.500 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.500 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2023-09-12T16:21:41.501 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.501 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.501 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.501 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.501 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2023-09-12T16:21:41.502 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.502 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.502 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.502 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.502 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2023-09-12T16:21:41.502 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.503 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.503 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.503 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.503 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2023-09-12T16:21:41.503 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.504 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.504 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.504 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.504 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2023-09-12T16:21:41.504 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.504 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.505 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.505 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.505 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2023-09-12T16:21:41.505 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.505 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.506 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.506 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.506 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2023-09-12T16:21:41.506 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.506 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.506 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.507 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.507 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2023-09-12T16:21:41.507 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.507 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.507 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.508 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.508 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2023-09-12T16:21:41.508 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.508 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.508 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.508 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.509 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2023-09-12T16:21:41.509 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.509 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.509 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.509 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.510 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2023-09-12T16:21:41.510 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.510 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.510 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.510 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.510 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2023-09-12T16:21:41.511 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2023-09-12T16:21:41.511 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2023-09-12T16:21:41.511 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.511 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.511 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.512 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:41.512 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.512 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2023-09-12T16:21:41.512 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2023-09-12T16:21:41.512 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 2, 2023-09-12T16:21:41.512 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2023-09-12T16:21:41.513 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2023-09-12T16:21:41.513 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2023-09-12T16:21:41.513 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 22, 2023-09-12T16:21:41.513 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 22, 2023-09-12T16:21:41.513 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 1075, 2023-09-12T16:21:41.514 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2023-09-12T16:21:41.514 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.514 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2023-09-12T16:21:41.514 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:41.514 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:41.514 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:41.515 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:41.515 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:41.515 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:41.515 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:41.515 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:41.516 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:41.516 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:41.516 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:41.516 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:41.516 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:41.516 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:41.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:41.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:41.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:41.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:41.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:41.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:41.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:41.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:41.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:41.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:41.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:41.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:41.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:41.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:41.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:41.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:41.520 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:41.520 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2023-09-12T16:21:41.520 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:41.520 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:41.520 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:41.520 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:41.521 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 297874, 2023-09-12T16:21:41.521 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 439, 2023-09-12T16:21:41.521 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:41.521 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:41.521 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2023-09-12T16:21:41.522 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2023-09-12T16:21:41.522 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:41.522 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:41.522 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:41.522 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:41.523 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8984, 2023-09-12T16:21:41.523 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 172, 2023-09-12T16:21:41.523 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:41.523 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:41.523 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:41.524 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2023-09-12T16:21:41.524 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 1220366, 2023-09-12T16:21:41.524 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 12728, 2023-09-12T16:21:41.524 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:41.524 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2023-09-12T16:21:41.525 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:41.525 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2023-09-12T16:21:41.525 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.525 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2023-09-12T16:21:41.525 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2023-09-12T16:21:41.525 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2023-09-12T16:21:41.526 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 567, 2023-09-12T16:21:41.526 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 1285625, 2023-09-12T16:21:41.526 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 28, 2023-09-12T16:21:41.526 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 567, 2023-09-12T16:21:41.526 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2023-09-12T16:21:41.527 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 567, 2023-09-12T16:21:41.527 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 17.880567006, 2023-09-12T16:21:41.527 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.031535391 2023-09-12T16:21:41.527 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.527 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2023-09-12T16:21:41.527 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2023-09-12T16:21:41.528 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 567, 2023-09-12T16:21:41.528 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 606 2023-09-12T16:21:41.528 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.528 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 567, 2023-09-12T16:21:41.528 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 221, 2023-09-12T16:21:41.529 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 346, 2023-09-12T16:21:41.529 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2023-09-12T16:21:41.529 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2023-09-12T16:21:41.529 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 0, 2023-09-12T16:21:41.529 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 10, 2023-09-12T16:21:41.529 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 0, 2023-09-12T16:21:41.530 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 243, 2023-09-12T16:21:41.530 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 58, 2023-09-12T16:21:41.530 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2023-09-12T16:21:41.530 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2023-09-12T16:21:41.530 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2023-09-12T16:21:41.531 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2023-09-12T16:21:41.531 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 6, 2023-09-12T16:21:41.531 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2023-09-12T16:21:41.531 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2023-09-12T16:21:41.531 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2023-09-12T16:21:41.531 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 221, 2023-09-12T16:21:41.532 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 20, 2023-09-12T16:21:41.532 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2023-09-12T16:21:41.532 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2023-09-12T16:21:41.532 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2023-09-12T16:21:41.532 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2023-09-12T16:21:41.533 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2023-09-12T16:21:41.533 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2023-09-12T16:21:41.533 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2023-09-12T16:21:41.533 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2023-09-12T16:21:41.533 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2023-09-12T16:21:41.533 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2023-09-12T16:21:41.534 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2023-09-12T16:21:41.534 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2023-09-12T16:21:41.534 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2023-09-12T16:21:41.534 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2023-09-12T16:21:41.534 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2023-09-12T16:21:41.535 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2023-09-12T16:21:41.535 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2023-09-12T16:21:41.535 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2023-09-12T16:21:41.535 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2023-09-12T16:21:41.535 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2023-09-12T16:21:41.535 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2023-09-12T16:21:41.536 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2023-09-12T16:21:41.536 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2023-09-12T16:21:41.536 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2023-09-12T16:21:41.536 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2023-09-12T16:21:41.536 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2023-09-12T16:21:41.537 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 90, 2023-09-12T16:21:41.537 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2023-09-12T16:21:41.537 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 4, 2023-09-12T16:21:41.537 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 12, 2023-09-12T16:21:41.537 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 12, 2023-09-12T16:21:41.537 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2023-09-12T16:21:41.538 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2023-09-12T16:21:41.538 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 39, 2023-09-12T16:21:41.538 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 0, 2023-09-12T16:21:41.538 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 9 2023-09-12T16:21:41.538 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.539 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2023-09-12T16:21:41.539 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 1, 2023-09-12T16:21:41.539 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2023-09-12T16:21:41.539 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2023-09-12T16:21:41.539 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2023-09-12T16:21:41.539 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.540 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2023-09-12T16:21:41.540 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2023-09-12T16:21:41.540 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.540 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2023-09-12T16:21:41.540 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2023-09-12T16:21:41.540 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2023-09-12T16:21:41.541 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2023-09-12T16:21:41.541 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2023-09-12T16:21:41.541 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2023-09-12T16:21:41.541 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2023-09-12T16:21:41.541 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.542 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:41.542 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 72, 2023-09-12T16:21:41.542 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-09-12T16:21:41.542 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:41.542 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 3583193, 2023-09-12T16:21:41.543 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 259580784, 2023-09-12T16:21:41.543 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:41.543 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 3583193, 2023-09-12T16:21:41.543 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:41.543 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:41.543 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 3583192, 2023-09-12T16:21:41.544 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 259580712, 2023-09-12T16:21:41.544 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:41.544 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.544 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.544 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.544 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:41.545 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.545 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2023-09-12T16:21:41.545 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:41.545 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-09-12T16:21:41.545 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:41.546 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-09-12T16:21:41.546 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-09-12T16:21:41.546 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:41.546 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:41.546 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 567, 2023-09-12T16:21:41.546 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 1285625, 2023-09-12T16:21:41.547 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 567, 2023-09-12T16:21:41.547 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1285625, 2023-09-12T16:21:41.547 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:41.547 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.547 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.548 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.548 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:41.548 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.548 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2023-09-12T16:21:41.548 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:41.548 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2023-09-12T16:21:41.549 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:41.549 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-09-12T16:21:41.549 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-09-12T16:21:41.549 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:41.549 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:41.550 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 567, 2023-09-12T16:21:41.550 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 567, 2023-09-12T16:21:41.550 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 567, 2023-09-12T16:21:41.550 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 567, 2023-09-12T16:21:41.550 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:41.550 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.551 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.551 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.551 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:41.551 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.551 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2023-09-12T16:21:41.552 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:41.552 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-09-12T16:21:41.552 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:41.552 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-09-12T16:21:41.552 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-09-12T16:21:41.552 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:41.553 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:41.553 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:41.553 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:41.553 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2023-09-12T16:21:41.553 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2023-09-12T16:21:41.554 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:41.554 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.554 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.554 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.554 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:41.554 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:41.555 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55bd7c2c46e0": { 2023-09-12T16:21:41.555 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 183, 2023-09-12T16:21:41.555 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-09-12T16:21:41.555 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:41.555 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 705, 2023-09-12T16:21:41.555 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 1244280, 2023-09-12T16:21:41.556 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:41.556 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 705, 2023-09-12T16:21:41.556 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:41.556 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:41.556 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 243, 2023-09-12T16:21:41.557 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1244097, 2023-09-12T16:21:41.557 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:41.557 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:41.557 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:41.557 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:41.557 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:41.558 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:41.558 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-09-12T16:21:41.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:41 smithi203 ceph-mon[103431]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:21:41.924 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-09-12T16:21:41.925 INFO:tasks.check_counter:Daemon mds.c mds.imported=13 2023-09-12T16:21:41.925 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=5 2023-09-12T16:21:41.925 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=234 2023-09-12T16:21:41.926 DEBUG:tasks.check_counter:Getting stats from f 2023-09-12T16:21:41.926 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.f perf dump 2023-09-12T16:21:41.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:41 smithi130 ceph-mon[98670]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:21:41.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:41 smithi159 ceph-mon[102522]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-09-12T16:21:42.510 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T16:21:42.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:42 smithi203 ceph-mon[103431]: pgmap v855: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-12T16:21:42.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:42 smithi130 ceph-mon[98670]: pgmap v855: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-12T16:21:42.975 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:42 smithi159 ceph-mon[102522]: pgmap v855: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-12T16:21:43.895 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-09-12T16:21:43.895 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:43.896 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1687, 2023-09-12T16:21:43.896 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1688, 2023-09-12T16:21:43.896 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 5180474, 2023-09-12T16:21:43.896 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 524356, 2023-09-12T16:21:43.896 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2023-09-12T16:21:43.897 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2023-09-12T16:21:43.897 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.148015193, 2023-09-12T16:21:43.897 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.035441933, 2023-09-12T16:21:43.897 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.039527021, 2023-09-12T16:21:43.897 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.060200160, 2023-09-12T16:21:43.898 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:43.898 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1686, 2023-09-12T16:21:43.898 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.480976147, 2023-09-12T16:21:43.898 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000285276 2023-09-12T16:21:43.898 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.899 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:43.899 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.899 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.899 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.899 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.900 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-12T16:21:43.900 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2023-09-12T16:21:43.900 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.900 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:43.900 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 2002, 2023-09-12T16:21:43.901 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1986, 2023-09-12T16:21:43.901 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 9375961, 2023-09-12T16:21:43.901 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 709123, 2023-09-12T16:21:43.901 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:43.901 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2023-09-12T16:21:43.901 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.181046016, 2023-09-12T16:21:43.902 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.046890550, 2023-09-12T16:21:43.902 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.059677349, 2023-09-12T16:21:43.902 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.064452364, 2023-09-12T16:21:43.902 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:43.902 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1986, 2023-09-12T16:21:43.902 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.531441304, 2023-09-12T16:21:43.903 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000267593 2023-09-12T16:21:43.903 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.903 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:43.903 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.904 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.904 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.904 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.904 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 564864, 2023-09-12T16:21:43.904 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 194352 2023-09-12T16:21:43.905 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.905 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:43.905 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 362, 2023-09-12T16:21:43.905 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 604, 2023-09-12T16:21:43.906 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 4285191, 2023-09-12T16:21:43.906 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 693241, 2023-09-12T16:21:43.906 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2023-09-12T16:21:43.906 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2023-09-12T16:21:43.906 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.042329997, 2023-09-12T16:21:43.906 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.017661836, 2023-09-12T16:21:43.907 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.010020258, 2023-09-12T16:21:43.907 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.010310551, 2023-09-12T16:21:43.907 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:43.907 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 602, 2023-09-12T16:21:43.907 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.141355758, 2023-09-12T16:21:43.908 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000234810 2023-09-12T16:21:43.908 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.908 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:43.908 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.908 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.908 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.909 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.909 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-12T16:21:43.909 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 581280 2023-09-12T16:21:43.909 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.909 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2023-09-12T16:21:43.909 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2023-09-12T16:21:43.910 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2023-09-12T16:21:43.910 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.910 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2023-09-12T16:21:43.910 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-09-12T16:21:43.910 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-09-12T16:21:43.911 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 6319, 2023-09-12T16:21:43.911 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.157844639, 2023-09-12T16:21:43.911 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000024979 2023-09-12T16:21:43.911 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:43.911 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.911 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2023-09-12T16:21:43.912 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-09-12T16:21:43.912 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-09-12T16:21:43.912 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.912 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.912 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.913 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:43.913 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.913 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2023-09-12T16:21:43.913 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2023-09-12T16:21:43.913 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2023-09-12T16:21:43.913 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2023-09-12T16:21:43.914 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.914 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.914 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.914 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.914 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2023-09-12T16:21:43.915 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2023-09-12T16:21:43.915 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2023-09-12T16:21:43.915 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2023-09-12T16:21:43.915 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2023-09-12T16:21:43.915 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2023-09-12T16:21:43.915 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2023-09-12T16:21:43.916 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 1000, 2023-09-12T16:21:43.916 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 0, 2023-09-12T16:21:43.916 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 6, 2023-09-12T16:21:43.916 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 994, 2023-09-12T16:21:43.916 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 1000, 2023-09-12T16:21:43.916 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2023-09-12T16:21:43.917 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2023-09-12T16:21:43.917 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2023-09-12T16:21:43.917 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 15, 2023-09-12T16:21:43.917 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2023-09-12T16:21:43.917 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2023-09-12T16:21:43.918 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2023-09-12T16:21:43.918 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2023-09-12T16:21:43.918 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2023-09-12T16:21:43.918 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2023-09-12T16:21:43.918 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2023-09-12T16:21:43.918 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2023-09-12T16:21:43.919 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2023-09-12T16:21:43.919 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2023-09-12T16:21:43.919 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2023-09-12T16:21:43.919 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2023-09-12T16:21:43.919 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2023-09-12T16:21:43.920 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2023-09-12T16:21:43.920 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2023-09-12T16:21:43.920 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2023-09-12T16:21:43.920 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 1046, 2023-09-12T16:21:43.920 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 9079358200, 2023-09-12T16:21:43.921 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2023-09-12T16:21:43.921 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2023-09-12T16:21:43.921 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2023-09-12T16:21:43.921 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2023-09-12T16:21:43.921 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2023-09-12T16:21:43.921 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2023-09-12T16:21:43.922 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2023-09-12T16:21:43.922 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2023-09-12T16:21:43.922 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2023-09-12T16:21:43.922 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2023-09-12T16:21:43.923 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2023-09-12T16:21:43.923 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2023-09-12T16:21:43.923 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2023-09-12T16:21:43.923 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-12T16:21:43.924 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-12T16:21:43.924 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2023-09-12T16:21:43.924 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2023-09-12T16:21:43.924 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2023-09-12T16:21:43.924 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2023-09-12T16:21:43.925 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.925 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2023-09-12T16:21:43.925 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2023-09-12T16:21:43.925 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2023-09-12T16:21:43.925 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2023-09-12T16:21:43.925 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2023-09-12T16:21:43.926 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2023-09-12T16:21:43.926 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2023-09-12T16:21:43.926 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2023-09-12T16:21:43.926 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2023-09-12T16:21:43.926 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2023-09-12T16:21:43.927 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2023-09-12T16:21:43.927 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2023-09-12T16:21:43.927 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2023-09-12T16:21:43.927 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2023-09-12T16:21:43.927 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2023-09-12T16:21:43.927 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2023-09-12T16:21:43.928 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2023-09-12T16:21:43.928 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2023-09-12T16:21:43.928 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2023-09-12T16:21:43.928 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2023-09-12T16:21:43.928 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2023-09-12T16:21:43.929 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2023-09-12T16:21:43.929 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2023-09-12T16:21:43.929 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2023-09-12T16:21:43.929 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.929 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2023-09-12T16:21:43.929 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2023-09-12T16:21:43.930 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2023-09-12T16:21:43.930 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2023-09-12T16:21:43.930 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2023-09-12T16:21:43.930 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 6452, 2023-09-12T16:21:43.930 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2023-09-12T16:21:43.931 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2023-09-12T16:21:43.931 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2023-09-12T16:21:43.931 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2023-09-12T16:21:43.931 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2023-09-12T16:21:43.931 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 10, 2023-09-12T16:21:43.931 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2023-09-12T16:21:43.932 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2023-09-12T16:21:43.932 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2023-09-12T16:21:43.932 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2023-09-12T16:21:43.932 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 21445219, 2023-09-12T16:21:43.932 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 21443180, 2023-09-12T16:21:43.933 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2023-09-12T16:21:43.933 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.933 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.933 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.933 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.933 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 6452 2023-09-12T16:21:43.934 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.934 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2023-09-12T16:21:43.934 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 523, 2023-09-12T16:21:43.934 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 1476, 2023-09-12T16:21:43.934 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 953, 2023-09-12T16:21:43.935 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 62, 2023-09-12T16:21:43.935 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 74, 2023-09-12T16:21:43.935 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 12, 2023-09-12T16:21:43.935 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 870, 2023-09-12T16:21:43.935 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 1564, 2023-09-12T16:21:43.935 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 694, 2023-09-12T16:21:43.936 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2023-09-12T16:21:43.936 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2023-09-12T16:21:43.936 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2023-09-12T16:21:43.936 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 53864, 2023-09-12T16:21:43.936 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 207152 2023-09-12T16:21:43.937 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.937 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2023-09-12T16:21:43.937 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2023-09-12T16:21:43.937 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2023-09-12T16:21:43.937 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2023-09-12T16:21:43.937 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2023-09-12T16:21:43.938 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2023-09-12T16:21:43.938 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2023-09-12T16:21:43.938 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.938 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.938 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.938 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.939 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2023-09-12T16:21:43.939 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.939 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.939 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.939 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.940 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2023-09-12T16:21:43.940 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.940 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.940 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.940 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.941 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2023-09-12T16:21:43.941 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.941 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.941 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.941 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.942 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2023-09-12T16:21:43.942 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.942 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.942 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.942 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.943 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2023-09-12T16:21:43.943 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.943 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.943 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.944 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.944 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2023-09-12T16:21:43.944 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.944 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.944 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.945 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.945 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2023-09-12T16:21:43.945 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.945 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.945 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.945 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.946 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2023-09-12T16:21:43.946 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.946 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.946 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.946 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.947 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2023-09-12T16:21:43.947 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.947 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.947 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.947 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.947 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2023-09-12T16:21:43.948 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.948 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.948 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.948 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.948 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2023-09-12T16:21:43.949 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.949 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.949 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.949 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.949 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2023-09-12T16:21:43.949 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.950 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.950 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.950 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.950 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2023-09-12T16:21:43.950 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.951 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.951 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.951 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.951 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2023-09-12T16:21:43.951 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.952 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.952 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.952 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.952 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2023-09-12T16:21:43.952 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.952 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.953 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.953 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.953 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2023-09-12T16:21:43.953 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.953 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.954 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.954 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.954 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2023-09-12T16:21:43.954 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.954 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.954 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.955 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.955 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2023-09-12T16:21:43.955 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.955 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.955 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.955 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.956 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2023-09-12T16:21:43.956 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.956 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.956 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.956 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.957 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2023-09-12T16:21:43.957 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.957 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.957 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.957 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.957 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2023-09-12T16:21:43.958 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.958 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.958 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.958 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.958 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2023-09-12T16:21:43.959 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.959 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.959 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.959 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.959 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2023-09-12T16:21:43.959 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.960 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.960 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.960 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.960 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2023-09-12T16:21:43.960 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.961 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.961 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.961 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.961 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2023-09-12T16:21:43.961 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.961 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.962 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.962 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.962 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2023-09-12T16:21:43.962 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.963 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.963 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.963 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.963 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2023-09-12T16:21:43.964 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.964 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.964 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.964 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.964 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2023-09-12T16:21:43.964 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.965 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.965 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.965 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.965 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2023-09-12T16:21:43.965 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2023-09-12T16:21:43.966 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2023-09-12T16:21:43.966 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.966 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.966 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.966 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:43.966 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.967 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2023-09-12T16:21:43.967 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2023-09-12T16:21:43.967 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 3, 2023-09-12T16:21:43.967 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 2, 2023-09-12T16:21:43.967 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 2, 2023-09-12T16:21:43.968 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2023-09-12T16:21:43.968 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2023-09-12T16:21:43.968 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2023-09-12T16:21:43.968 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 3682, 2023-09-12T16:21:43.968 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2023-09-12T16:21:43.968 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.969 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2023-09-12T16:21:43.969 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:43.969 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:43.969 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:43.969 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:43.970 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:43.970 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:43.970 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:43.970 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:43.970 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:43.971 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:43.971 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:43.971 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:43.971 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:43.971 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:43.971 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:43.972 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:43.972 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:43.972 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:43.972 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:43.972 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:43.973 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:43.973 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:43.973 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:43.973 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:43.973 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:43.973 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:43.974 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:43.974 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:43.974 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:43.974 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:43.974 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:43.975 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2023-09-12T16:21:43.975 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:43.975 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:43.975 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:43.975 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:43.975 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 221621, 2023-09-12T16:21:43.976 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 687, 2023-09-12T16:21:43.976 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:43.976 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:43.976 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2023-09-12T16:21:43.976 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2023-09-12T16:21:43.976 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:43.977 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:43.977 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:43.977 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:43.977 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8984, 2023-09-12T16:21:43.977 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 172, 2023-09-12T16:21:43.978 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:43.978 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:43.978 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:43.978 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2023-09-12T16:21:43.978 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 2254501, 2023-09-12T16:21:43.978 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 19788, 2023-09-12T16:21:43.979 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:43.979 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2023-09-12T16:21:43.979 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:43.979 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2023-09-12T16:21:43.979 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.979 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2023-09-12T16:21:43.980 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2023-09-12T16:21:43.980 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2023-09-12T16:21:43.980 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 3701, 2023-09-12T16:21:43.980 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 26, 2023-09-12T16:21:43.980 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 4, 2023-09-12T16:21:43.981 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 3701, 2023-09-12T16:21:43.981 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2023-09-12T16:21:43.981 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 3701, 2023-09-12T16:21:43.981 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 119.427108461, 2023-09-12T16:21:43.981 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.032268875 2023-09-12T16:21:43.981 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.982 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2023-09-12T16:21:43.982 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2023-09-12T16:21:43.982 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 3701, 2023-09-12T16:21:43.982 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 3702 2023-09-12T16:21:43.982 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.983 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 3701, 2023-09-12T16:21:43.983 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 3701, 2023-09-12T16:21:43.983 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2023-09-12T16:21:43.983 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2023-09-12T16:21:43.983 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2023-09-12T16:21:43.983 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 2082, 2023-09-12T16:21:43.984 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2023-09-12T16:21:43.984 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 1616, 2023-09-12T16:21:43.984 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2023-09-12T16:21:43.984 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2023-09-12T16:21:43.984 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2023-09-12T16:21:43.984 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2023-09-12T16:21:43.985 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2023-09-12T16:21:43.985 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2023-09-12T16:21:43.985 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2023-09-12T16:21:43.985 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2023-09-12T16:21:43.986 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2023-09-12T16:21:43.986 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2023-09-12T16:21:43.986 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 2, 2023-09-12T16:21:43.986 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2023-09-12T16:21:43.987 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2023-09-12T16:21:43.987 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2023-09-12T16:21:43.987 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2023-09-12T16:21:43.987 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2023-09-12T16:21:43.987 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2023-09-12T16:21:43.988 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2023-09-12T16:21:43.988 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2023-09-12T16:21:43.988 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2023-09-12T16:21:43.988 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2023-09-12T16:21:43.988 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2023-09-12T16:21:43.988 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2023-09-12T16:21:43.989 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2023-09-12T16:21:43.989 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2023-09-12T16:21:43.989 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2023-09-12T16:21:43.989 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2023-09-12T16:21:43.989 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2023-09-12T16:21:43.990 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2023-09-12T16:21:43.990 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2023-09-12T16:21:43.990 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2023-09-12T16:21:43.990 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2023-09-12T16:21:43.990 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2023-09-12T16:21:43.991 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2023-09-12T16:21:43.991 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2023-09-12T16:21:43.991 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2023-09-12T16:21:43.991 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2023-09-12T16:21:43.991 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2023-09-12T16:21:43.991 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 90, 2023-09-12T16:21:43.992 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2023-09-12T16:21:43.992 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 4, 2023-09-12T16:21:43.992 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 6, 2023-09-12T16:21:43.992 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 6, 2023-09-12T16:21:43.992 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2023-09-12T16:21:43.992 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2023-09-12T16:21:43.993 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2023-09-12T16:21:43.993 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2023-09-12T16:21:43.993 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2023-09-12T16:21:43.993 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.993 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2023-09-12T16:21:43.994 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2023-09-12T16:21:43.994 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2023-09-12T16:21:43.994 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2023-09-12T16:21:43.994 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2023-09-12T16:21:43.994 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.994 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2023-09-12T16:21:43.995 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2023-09-12T16:21:43.995 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.995 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2023-09-12T16:21:43.995 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2023-09-12T16:21:43.995 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2023-09-12T16:21:43.995 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2023-09-12T16:21:43.996 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2023-09-12T16:21:43.996 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2023-09-12T16:21:43.996 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2023-09-12T16:21:43.996 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:43.996 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:43.997 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:43.997 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-09-12T16:21:43.997 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:43.997 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 4047, 2023-09-12T16:21:43.997 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 18456775, 2023-09-12T16:21:43.997 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:43.998 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 4047, 2023-09-12T16:21:43.998 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:43.998 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:43.998 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 4047, 2023-09-12T16:21:43.998 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 18456775, 2023-09-12T16:21:43.999 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:43.999 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:43.999 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:43.999 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:43.999 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:43.999 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:44.000 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2023-09-12T16:21:44.000 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:44.000 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-09-12T16:21:44.000 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:44.000 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-09-12T16:21:44.001 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-09-12T16:21:44.001 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:44.001 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:44.001 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 3701, 2023-09-12T16:21:44.001 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 17253244, 2023-09-12T16:21:44.001 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 3701, 2023-09-12T16:21:44.002 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 17253244, 2023-09-12T16:21:44.002 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:44.002 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:44.002 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:44.002 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:44.002 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:44.003 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:44.003 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2023-09-12T16:21:44.003 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:44.003 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2023-09-12T16:21:44.003 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:44.004 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-09-12T16:21:44.004 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-09-12T16:21:44.004 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:44.004 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:44.004 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 3701, 2023-09-12T16:21:44.004 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 3701, 2023-09-12T16:21:44.005 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 3701, 2023-09-12T16:21:44.005 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 3701, 2023-09-12T16:21:44.005 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:44.005 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:44.005 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:44.005 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:44.006 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:44.006 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:44.006 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2023-09-12T16:21:44.007 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:44.007 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-09-12T16:21:44.007 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:44.007 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-09-12T16:21:44.007 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-09-12T16:21:44.008 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:44.008 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:44.008 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:44.008 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:44.008 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2023-09-12T16:21:44.008 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2023-09-12T16:21:44.009 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:44.009 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:44.009 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:44.009 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:44.009 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:44.010 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:44.010 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55f9cd182960": { 2023-09-12T16:21:44.010 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:44.010 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-09-12T16:21:44.010 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:44.010 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-09-12T16:21:44.011 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-09-12T16:21:44.011 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:44.011 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:44.011 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:44.011 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:44.012 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2023-09-12T16:21:44.012 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2023-09-12T16:21:44.012 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:44.012 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:44.012 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:44.013 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:44.013 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:44.013 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:44.014 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-09-12T16:21:44.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:43 smithi203 ceph-mon[103431]: pgmap v856: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-12T16:21:44.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:43 smithi130 ceph-mon[98670]: pgmap v856: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-12T16:21:44.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:43 smithi159 ceph-mon[102522]: pgmap v856: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-12T16:21:46.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:45 smithi130 ceph-mon[98670]: pgmap v857: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-12T16:21:46.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:45 smithi159 ceph-mon[102522]: pgmap v857: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-12T16:21:46.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:45 smithi203 ceph-mon[103431]: pgmap v857: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-12T16:21:46.374 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-09-12T16:21:46.375 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-09-12T16:21:46.375 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-09-12T16:21:46.375 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-09-12T16:21:46.376 DEBUG:tasks.check_counter:Getting stats from i 2023-09-12T16:21:46.376 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.i perf dump 2023-09-12T16:21:46.985 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T16:21:48.564 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-09-12T16:21:48.565 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:48.565 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 28468, 2023-09-12T16:21:48.565 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 41716, 2023-09-12T16:21:48.565 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 6157040, 2023-09-12T16:21:48.566 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 23806579, 2023-09-12T16:21:48.566 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 14, 2023-09-12T16:21:48.566 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2023-09-12T16:21:48.566 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 1.799724035, 2023-09-12T16:21:48.566 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.705209254, 2023-09-12T16:21:48.567 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 1.025689558, 2023-09-12T16:21:48.567 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.183056932, 2023-09-12T16:21:48.567 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:48.567 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 41714, 2023-09-12T16:21:48.567 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 5.158334421, 2023-09-12T16:21:48.567 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000123659 2023-09-12T16:21:48.568 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.568 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:48.568 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 56967, 2023-09-12T16:21:48.568 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.012267572, 2023-09-12T16:21:48.568 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000215 2023-09-12T16:21:48.569 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.569 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-12T16:21:48.569 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2023-09-12T16:21:48.569 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.569 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:48.570 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 15531, 2023-09-12T16:21:48.570 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 18444, 2023-09-12T16:21:48.570 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 5249175, 2023-09-12T16:21:48.570 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 10691381, 2023-09-12T16:21:48.570 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 11, 2023-09-12T16:21:48.570 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 6, 2023-09-12T16:21:48.571 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 1.013572178, 2023-09-12T16:21:48.571 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.436044328, 2023-09-12T16:21:48.571 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.711467403, 2023-09-12T16:21:48.571 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.053134570, 2023-09-12T16:21:48.571 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:48.572 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 18444, 2023-09-12T16:21:48.572 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.149499717, 2023-09-12T16:21:48.572 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000116541 2023-09-12T16:21:48.572 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.573 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:48.573 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 27738, 2023-09-12T16:21:48.573 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.004431228, 2023-09-12T16:21:48.574 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000159 2023-09-12T16:21:48.574 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.574 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 5235184, 2023-09-12T16:21:48.574 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 10654192 2023-09-12T16:21:48.575 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.575 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:48.575 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 720, 2023-09-12T16:21:48.575 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 955, 2023-09-12T16:21:48.576 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 225762, 2023-09-12T16:21:48.576 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 1519508, 2023-09-12T16:21:48.576 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 9, 2023-09-12T16:21:48.576 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 6, 2023-09-12T16:21:48.577 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.068747086, 2023-09-12T16:21:48.577 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.028450663, 2023-09-12T16:21:48.577 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.023296357, 2023-09-12T16:21:48.577 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.014835774, 2023-09-12T16:21:48.578 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:48.578 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 953, 2023-09-12T16:21:48.578 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.297684708, 2023-09-12T16:21:48.578 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000312365 2023-09-12T16:21:48.578 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.579 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:48.579 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 459, 2023-09-12T16:21:48.579 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000088071, 2023-09-12T16:21:48.579 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000191 2023-09-12T16:21:48.580 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.580 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-12T16:21:48.580 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 589344 2023-09-12T16:21:48.580 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.580 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2023-09-12T16:21:48.581 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2023-09-12T16:21:48.581 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2023-09-12T16:21:48.581 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.581 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2023-09-12T16:21:48.581 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-09-12T16:21:48.582 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-09-12T16:21:48.582 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 10850, 2023-09-12T16:21:48.582 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 7.615271566, 2023-09-12T16:21:48.582 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000701868 2023-09-12T16:21:48.583 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:48.583 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.583 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2023-09-12T16:21:48.583 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-09-12T16:21:48.584 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-09-12T16:21:48.584 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 161, 2023-09-12T16:21:48.584 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.005481367, 2023-09-12T16:21:48.584 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000034045 2023-09-12T16:21:48.584 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:48.585 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.585 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2023-09-12T16:21:48.585 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 12220, 2023-09-12T16:21:48.585 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 12164, 2023-09-12T16:21:48.586 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2023-09-12T16:21:48.586 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 12164, 2023-09-12T16:21:48.586 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 149.939746201, 2023-09-12T16:21:48.586 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.012326516 2023-09-12T16:21:48.587 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.587 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2023-09-12T16:21:48.587 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 56, 2023-09-12T16:21:48.587 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2023-09-12T16:21:48.587 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2023-09-12T16:21:48.588 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 11, 2023-09-12T16:21:48.588 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2023-09-12T16:21:48.588 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2023-09-12T16:21:48.588 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 1967, 2023-09-12T16:21:48.588 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 6, 2023-09-12T16:21:48.588 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 1423, 2023-09-12T16:21:48.589 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 538, 2023-09-12T16:21:48.589 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 1958, 2023-09-12T16:21:48.589 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 224, 2023-09-12T16:21:48.589 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 131, 2023-09-12T16:21:48.589 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 131, 2023-09-12T16:21:48.590 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 15, 2023-09-12T16:21:48.590 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 90000, 2023-09-12T16:21:48.590 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 65689, 2023-09-12T16:21:48.590 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 53, 2023-09-12T16:21:48.590 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 71, 2023-09-12T16:21:48.590 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2023-09-12T16:21:48.591 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2023-09-12T16:21:48.591 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 19714, 2023-09-12T16:21:48.591 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 841, 2023-09-12T16:21:48.591 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2023-09-12T16:21:48.591 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 10, 2023-09-12T16:21:48.592 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 127, 2023-09-12T16:21:48.592 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 16, 2023-09-12T16:21:48.592 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 253, 2023-09-12T16:21:48.592 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2023-09-12T16:21:48.592 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 65, 2023-09-12T16:21:48.593 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2023-09-12T16:21:48.593 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 1046, 2023-09-12T16:21:48.593 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 9078222795, 2023-09-12T16:21:48.593 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2023-09-12T16:21:48.593 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 65, 2023-09-12T16:21:48.593 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2023-09-12T16:21:48.594 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2023-09-12T16:21:48.594 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2023-09-12T16:21:48.594 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 1, 2023-09-12T16:21:48.594 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2023-09-12T16:21:48.594 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 1, 2023-09-12T16:21:48.595 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 64, 2023-09-12T16:21:48.595 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 464, 2023-09-12T16:21:48.595 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 1546, 2023-09-12T16:21:48.595 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 3218, 2023-09-12T16:21:48.595 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2023-09-12T16:21:48.596 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-12T16:21:48.596 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-12T16:21:48.596 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 3882, 2023-09-12T16:21:48.596 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 1582, 2023-09-12T16:21:48.596 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 311, 2023-09-12T16:21:48.596 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 4730 2023-09-12T16:21:48.597 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.597 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2023-09-12T16:21:48.597 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 61, 2023-09-12T16:21:48.597 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 136, 2023-09-12T16:21:48.597 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 79, 2023-09-12T16:21:48.598 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 134, 2023-09-12T16:21:48.598 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 461, 2023-09-12T16:21:48.598 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2023-09-12T16:21:48.598 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2023-09-12T16:21:48.598 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2023-09-12T16:21:48.598 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 1220, 2023-09-12T16:21:48.599 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 15, 2023-09-12T16:21:48.599 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 5766, 2023-09-12T16:21:48.599 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2023-09-12T16:21:48.599 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2023-09-12T16:21:48.599 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2023-09-12T16:21:48.600 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2023-09-12T16:21:48.600 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2023-09-12T16:21:48.600 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2023-09-12T16:21:48.600 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2023-09-12T16:21:48.600 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 10, 2023-09-12T16:21:48.600 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2023-09-12T16:21:48.601 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2023-09-12T16:21:48.601 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2023-09-12T16:21:48.601 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2023-09-12T16:21:48.601 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.601 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2023-09-12T16:21:48.602 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2023-09-12T16:21:48.602 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 7471, 2023-09-12T16:21:48.602 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2023-09-12T16:21:48.602 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2023-09-12T16:21:48.602 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 7471, 2023-09-12T16:21:48.602 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2023-09-12T16:21:48.603 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2023-09-12T16:21:48.603 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 11, 2023-09-12T16:21:48.603 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2023-09-12T16:21:48.603 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2023-09-12T16:21:48.603 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 11, 2023-09-12T16:21:48.604 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2023-09-12T16:21:48.604 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2023-09-12T16:21:48.604 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2023-09-12T16:21:48.604 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2023-09-12T16:21:48.604 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 17297512, 2023-09-12T16:21:48.604 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 0, 2023-09-12T16:21:48.605 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2023-09-12T16:21:48.605 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 5584, 2023-09-12T16:21:48.605 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 43.722546277, 2023-09-12T16:21:48.605 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.007829968 2023-09-12T16:21:48.605 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.606 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 0 2023-09-12T16:21:48.606 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.606 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2023-09-12T16:21:48.606 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 170, 2023-09-12T16:21:48.606 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 353, 2023-09-12T16:21:48.606 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 183, 2023-09-12T16:21:48.607 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 54, 2023-09-12T16:21:48.607 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 87, 2023-09-12T16:21:48.607 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 33, 2023-09-12T16:21:48.607 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 1967, 2023-09-12T16:21:48.608 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 2206, 2023-09-12T16:21:48.608 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 239, 2023-09-12T16:21:48.608 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 131, 2023-09-12T16:21:48.608 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 724, 2023-09-12T16:21:48.608 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 593, 2023-09-12T16:21:48.609 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 60324, 2023-09-12T16:21:48.609 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 198960 2023-09-12T16:21:48.609 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.609 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2023-09-12T16:21:48.609 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 57429, 2023-09-12T16:21:48.610 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 485, 2023-09-12T16:21:48.610 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 12220, 2023-09-12T16:21:48.610 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 2172, 2023-09-12T16:21:48.610 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 1750, 2023-09-12T16:21:48.611 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2023-09-12T16:21:48.611 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 45, 2023-09-12T16:21:48.611 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.092022778, 2023-09-12T16:21:48.611 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002044950 2023-09-12T16:21:48.611 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.612 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2023-09-12T16:21:48.612 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 40, 2023-09-12T16:21:48.612 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.123028132, 2023-09-12T16:21:48.612 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.003075703 2023-09-12T16:21:48.612 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.613 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2023-09-12T16:21:48.613 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.613 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.613 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.613 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.613 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2023-09-12T16:21:48.614 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1205, 2023-09-12T16:21:48.614 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 5.189087389, 2023-09-12T16:21:48.614 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004306296 2023-09-12T16:21:48.614 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.614 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2023-09-12T16:21:48.615 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 3592, 2023-09-12T16:21:48.615 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 13.844909495, 2023-09-12T16:21:48.615 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.003854373 2023-09-12T16:21:48.615 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.615 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2023-09-12T16:21:48.615 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.616 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.616 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.616 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.616 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2023-09-12T16:21:48.617 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.617 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.617 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.617 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.617 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2023-09-12T16:21:48.618 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.618 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.618 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.618 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.619 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2023-09-12T16:21:48.619 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.619 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.619 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.620 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.620 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2023-09-12T16:21:48.620 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.620 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.620 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.621 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.621 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2023-09-12T16:21:48.621 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.621 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.622 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.622 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.622 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2023-09-12T16:21:48.623 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.623 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.623 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.623 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.623 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2023-09-12T16:21:48.624 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.624 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.624 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.624 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.624 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2023-09-12T16:21:48.624 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.625 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.625 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.625 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.625 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2023-09-12T16:21:48.625 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 246, 2023-09-12T16:21:48.626 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.475988660, 2023-09-12T16:21:48.626 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001934913 2023-09-12T16:21:48.626 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.626 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2023-09-12T16:21:48.626 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 50, 2023-09-12T16:21:48.627 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.101881623, 2023-09-12T16:21:48.627 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002037632 2023-09-12T16:21:48.627 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.627 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2023-09-12T16:21:48.627 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 5768, 2023-09-12T16:21:48.627 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 120.861712939, 2023-09-12T16:21:48.628 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.020953833 2023-09-12T16:21:48.628 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.628 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2023-09-12T16:21:48.628 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.628 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.629 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.629 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.629 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2023-09-12T16:21:48.629 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.629 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.629 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.630 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.630 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2023-09-12T16:21:48.630 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.630 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.630 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.631 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.631 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2023-09-12T16:21:48.631 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.631 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.631 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.631 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.632 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2023-09-12T16:21:48.632 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.632 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.632 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.632 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.632 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2023-09-12T16:21:48.633 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.633 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.633 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.633 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.633 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2023-09-12T16:21:48.633 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.634 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.634 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.634 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.634 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2023-09-12T16:21:48.634 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.635 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.635 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.635 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.635 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2023-09-12T16:21:48.635 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.635 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.636 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.636 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.636 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2023-09-12T16:21:48.636 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.636 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.637 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.637 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.637 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2023-09-12T16:21:48.637 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.637 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.637 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.638 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.638 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2023-09-12T16:21:48.638 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1218, 2023-09-12T16:21:48.638 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 9.251115185, 2023-09-12T16:21:48.638 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.007595332 2023-09-12T16:21:48.638 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.639 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2023-09-12T16:21:48.639 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2023-09-12T16:21:48.639 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2023-09-12T16:21:48.639 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.639 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.640 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.640 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:48.640 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.640 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2023-09-12T16:21:48.640 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2023-09-12T16:21:48.640 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 2, 2023-09-12T16:21:48.641 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2023-09-12T16:21:48.641 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2023-09-12T16:21:48.641 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2023-09-12T16:21:48.641 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 497, 2023-09-12T16:21:48.641 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 497, 2023-09-12T16:21:48.642 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 1070, 2023-09-12T16:21:48.642 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2023-09-12T16:21:48.642 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.642 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2023-09-12T16:21:48.642 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:48.643 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:48.643 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:48.643 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:48.643 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:48.644 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:48.644 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:48.644 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:48.644 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:48.644 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:48.644 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:48.645 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:48.645 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:48.645 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:48.645 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:48.646 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:48.646 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:48.646 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:48.646 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:48.646 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:48.647 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:48.647 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:48.647 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:48.647 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:48.648 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:48.648 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:48.648 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:48.648 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:48.648 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:48.649 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:48.649 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:48.649 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2023-09-12T16:21:48.649 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:48.650 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:48.650 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:48.650 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:48.650 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 302491, 2023-09-12T16:21:48.650 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 254, 2023-09-12T16:21:48.651 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:48.651 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:48.651 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2023-09-12T16:21:48.651 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2023-09-12T16:21:48.651 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:48.652 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:48.652 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:48.652 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:48.652 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8984, 2023-09-12T16:21:48.653 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 172, 2023-09-12T16:21:48.653 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:48.653 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:48.653 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:48.654 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2023-09-12T16:21:48.654 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 2016860, 2023-09-12T16:21:48.654 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 62933, 2023-09-12T16:21:48.654 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:48.655 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2023-09-12T16:21:48.655 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:48.655 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2023-09-12T16:21:48.655 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.656 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2023-09-12T16:21:48.656 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2023-09-12T16:21:48.656 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2023-09-12T16:21:48.656 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 6061, 2023-09-12T16:21:48.656 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 13252664, 2023-09-12T16:21:48.656 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 28, 2023-09-12T16:21:48.657 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 6061, 2023-09-12T16:21:48.657 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2023-09-12T16:21:48.657 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 6061, 2023-09-12T16:21:48.657 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 69.785300892, 2023-09-12T16:21:48.657 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.011513826 2023-09-12T16:21:48.658 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.658 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2023-09-12T16:21:48.658 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2023-09-12T16:21:48.658 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 6061, 2023-09-12T16:21:48.658 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 6390 2023-09-12T16:21:48.658 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.659 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 6061, 2023-09-12T16:21:48.659 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 80, 2023-09-12T16:21:48.659 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 5981, 2023-09-12T16:21:48.659 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2023-09-12T16:21:48.659 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2023-09-12T16:21:48.660 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 0, 2023-09-12T16:21:48.660 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 10, 2023-09-12T16:21:48.660 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 15, 2023-09-12T16:21:48.660 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 5599, 2023-09-12T16:21:48.660 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 86, 2023-09-12T16:21:48.660 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2023-09-12T16:21:48.661 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2023-09-12T16:21:48.661 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2023-09-12T16:21:48.661 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2023-09-12T16:21:48.661 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 84, 2023-09-12T16:21:48.661 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2023-09-12T16:21:48.661 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2023-09-12T16:21:48.662 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2023-09-12T16:21:48.662 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 65, 2023-09-12T16:21:48.662 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 20, 2023-09-12T16:21:48.662 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2023-09-12T16:21:48.662 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2023-09-12T16:21:48.663 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2023-09-12T16:21:48.663 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2023-09-12T16:21:48.663 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2023-09-12T16:21:48.663 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2023-09-12T16:21:48.663 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2023-09-12T16:21:48.664 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2023-09-12T16:21:48.664 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2023-09-12T16:21:48.664 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2023-09-12T16:21:48.664 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2023-09-12T16:21:48.664 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2023-09-12T16:21:48.664 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2023-09-12T16:21:48.665 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2023-09-12T16:21:48.665 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2023-09-12T16:21:48.665 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2023-09-12T16:21:48.665 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2023-09-12T16:21:48.665 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2023-09-12T16:21:48.666 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2023-09-12T16:21:48.666 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2023-09-12T16:21:48.666 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2023-09-12T16:21:48.666 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2023-09-12T16:21:48.666 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2023-09-12T16:21:48.666 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2023-09-12T16:21:48.667 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2023-09-12T16:21:48.667 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2023-09-12T16:21:48.667 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 90, 2023-09-12T16:21:48.667 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2023-09-12T16:21:48.667 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 4, 2023-09-12T16:21:48.668 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 12, 2023-09-12T16:21:48.668 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 12, 2023-09-12T16:21:48.668 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2023-09-12T16:21:48.668 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2023-09-12T16:21:48.668 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 362, 2023-09-12T16:21:48.668 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 0, 2023-09-12T16:21:48.669 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 149 2023-09-12T16:21:48.669 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.669 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2023-09-12T16:21:48.669 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 1, 2023-09-12T16:21:48.669 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2023-09-12T16:21:48.670 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2023-09-12T16:21:48.670 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2023-09-12T16:21:48.670 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.670 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2023-09-12T16:21:48.670 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2023-09-12T16:21:48.671 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.671 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2023-09-12T16:21:48.671 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2023-09-12T16:21:48.671 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 5, 2023-09-12T16:21:48.671 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2023-09-12T16:21:48.672 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 1, 2023-09-12T16:21:48.672 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 15, 2023-09-12T16:21:48.672 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2023-09-12T16:21:48.672 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.672 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:48.672 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:48.673 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-09-12T16:21:48.673 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:48.673 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 44715, 2023-09-12T16:21:48.673 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 6938316, 2023-09-12T16:21:48.673 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:48.674 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 44715, 2023-09-12T16:21:48.674 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:48.674 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:48.674 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 44715, 2023-09-12T16:21:48.674 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 6938316, 2023-09-12T16:21:48.675 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:48.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.675 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.675 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:48.676 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.676 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2023-09-12T16:21:48.676 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:48.676 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-09-12T16:21:48.677 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:48.677 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-09-12T16:21:48.677 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-09-12T16:21:48.677 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:48.677 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:48.678 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 6061, 2023-09-12T16:21:48.678 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 13254299, 2023-09-12T16:21:48.678 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 6061, 2023-09-12T16:21:48.679 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 13254299, 2023-09-12T16:21:48.679 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:48.679 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.679 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.679 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.680 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:48.680 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.680 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2023-09-12T16:21:48.680 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:48.680 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2023-09-12T16:21:48.681 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:48.681 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-09-12T16:21:48.681 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-09-12T16:21:48.681 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:48.681 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-09-12T16:21:48.682 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 6061, 2023-09-12T16:21:48.682 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 6061, 2023-09-12T16:21:48.682 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 6061, 2023-09-12T16:21:48.682 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 6061, 2023-09-12T16:21:48.683 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:48.683 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.683 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.683 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.683 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:48.684 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.684 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2023-09-12T16:21:48.684 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:48.684 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-09-12T16:21:48.685 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:48.685 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 15, 2023-09-12T16:21:48.685 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 1635, 2023-09-12T16:21:48.685 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:48.685 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 15, 2023-09-12T16:21:48.685 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:48.686 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:48.686 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 15, 2023-09-12T16:21:48.686 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1635, 2023-09-12T16:21:48.686 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:48.686 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.686 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.687 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.687 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:48.687 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:48.687 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55b5a154e6e0": { 2023-09-12T16:21:48.687 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:48.688 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-09-12T16:21:48.688 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:48.688 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 7471, 2023-09-12T16:21:48.688 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 13103208, 2023-09-12T16:21:48.688 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:48.689 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 7471, 2023-09-12T16:21:48.689 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:48.689 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:48.689 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 5584, 2023-09-12T16:21:48.690 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 13103208, 2023-09-12T16:21:48.690 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:48.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:48.690 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:48.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:48.691 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:48.691 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:48.691 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-09-12T16:21:48.975 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:48 smithi130 ceph-mon[98670]: pgmap v858: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-12T16:21:48.976 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:48 smithi159 ceph-mon[102522]: pgmap v858: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-12T16:21:48.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:48 smithi203 ceph-mon[103431]: pgmap v858: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-12T16:21:49.338 INFO:tasks.check_counter:Daemon mds.i mds.exported=10 2023-09-12T16:21:49.338 INFO:tasks.check_counter:Daemon mds.i mds.imported=16 2023-09-12T16:21:49.338 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=61 2023-09-12T16:21:49.339 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=136 2023-09-12T16:21:49.339 DEBUG:tasks.check_counter:Getting stats from l 2023-09-12T16:21:49.339 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.l perf dump 2023-09-12T16:21:49.938 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-12T16:21:50.177 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:49 smithi203 ceph-mon[103431]: pgmap v859: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 8 op/s 2023-09-12T16:21:50.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:49 smithi130 ceph-mon[98670]: pgmap v859: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 8 op/s 2023-09-12T16:21:50.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:49 smithi159 ceph-mon[102522]: pgmap v859: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 8 op/s 2023-09-12T16:21:51.281 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-09-12T16:21:51.281 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2023-09-12T16:21:51.281 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 0, 2023-09-12T16:21:51.282 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 2, 2023-09-12T16:21:51.282 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 0, 2023-09-12T16:21:51.283 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 0, 2023-09-12T16:21:51.283 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:51.283 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 0, 2023-09-12T16:21:51.283 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.001471151, 2023-09-12T16:21:51.284 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.000000000, 2023-09-12T16:21:51.284 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.000424079, 2023-09-12T16:21:51.284 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:51.284 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:51.285 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:51.285 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:51.285 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:51.285 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.285 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:51.285 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:51.286 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:51.286 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:51.286 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.286 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-12T16:21:51.286 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2023-09-12T16:21:51.287 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.287 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2023-09-12T16:21:51.287 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 331, 2023-09-12T16:21:51.287 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 326, 2023-09-12T16:21:51.287 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 504240, 2023-09-12T16:21:51.288 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 195776, 2023-09-12T16:21:51.288 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 2, 2023-09-12T16:21:51.288 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 1, 2023-09-12T16:21:51.288 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.043566558, 2023-09-12T16:21:51.288 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.015530405, 2023-09-12T16:21:51.288 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.017446599, 2023-09-12T16:21:51.289 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.009172308, 2023-09-12T16:21:51.289 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:51.289 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 326, 2023-09-12T16:21:51.289 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.118300822, 2023-09-12T16:21:51.289 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000362885 2023-09-12T16:21:51.289 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.290 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:51.290 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:51.290 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:51.290 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:51.290 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.291 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 504240, 2023-09-12T16:21:51.291 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 195776 2023-09-12T16:21:51.291 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.291 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2023-09-12T16:21:51.291 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 2, 2023-09-12T16:21:51.292 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 248, 2023-09-12T16:21:51.292 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 320, 2023-09-12T16:21:51.292 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 196512, 2023-09-12T16:21:51.292 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2023-09-12T16:21:51.292 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2023-09-12T16:21:51.292 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.011311538, 2023-09-12T16:21:51.293 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.007865639, 2023-09-12T16:21:51.293 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.001463298, 2023-09-12T16:21:51.293 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-12T16:21:51.293 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-09-12T16:21:51.293 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 246, 2023-09-12T16:21:51.294 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.090437569, 2023-09-12T16:21:51.294 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000367632 2023-09-12T16:21:51.294 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.294 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-09-12T16:21:51.294 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:51.294 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:51.295 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:51.295 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.295 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-12T16:21:51.295 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 196512 2023-09-12T16:21:51.295 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.296 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2023-09-12T16:21:51.296 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 0, 2023-09-12T16:21:51.296 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2023-09-12T16:21:51.296 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.296 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2023-09-12T16:21:51.296 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2023-09-12T16:21:51.297 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2023-09-12T16:21:51.297 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2023-09-12T16:21:51.297 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2023-09-12T16:21:51.297 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2023-09-12T16:21:51.297 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2023-09-12T16:21:51.298 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-12T16:21:51.298 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2023-09-12T16:21:51.298 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-12T16:21:51.298 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2023-09-12T16:21:51.298 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2023-09-12T16:21:51.299 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2023-09-12T16:21:51.299 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-12T16:21:51.299 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2023-09-12T16:21:51.299 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2023-09-12T16:21:51.299 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2023-09-12T16:21:51.299 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2023-09-12T16:21:51.300 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2023-09-12T16:21:51.300 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-12T16:21:51.300 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2023-09-12T16:21:51.300 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-12T16:21:51.300 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2023-09-12T16:21:51.300 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2023-09-12T16:21:51.301 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2023-09-12T16:21:51.301 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2023-09-12T16:21:51.301 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2023-09-12T16:21:51.301 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-12T16:21:51.302 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2023-09-12T16:21:51.302 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2023-09-12T16:21:51.302 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2023-09-12T16:21:51.302 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2023-09-12T16:21:51.302 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2023-09-12T16:21:51.303 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2023-09-12T16:21:51.303 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2023-09-12T16:21:51.303 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2023-09-12T16:21:51.303 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2023-09-12T16:21:51.303 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 148721, 2023-09-12T16:21:51.303 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 30, 2023-09-12T16:21:51.304 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2023-09-12T16:21:51.304 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2023-09-12T16:21:51.304 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2023-09-12T16:21:51.304 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2023-09-12T16:21:51.304 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2023-09-12T16:21:51.305 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2023-09-12T16:21:51.305 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2023-09-12T16:21:51.305 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2023-09-12T16:21:51.305 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 0, 2023-09-12T16:21:51.305 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 0, 2023-09-12T16:21:51.305 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2023-09-12T16:21:51.306 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2023-09-12T16:21:51.306 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2023-09-12T16:21:51.306 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2023-09-12T16:21:51.306 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 928, 2023-09-12T16:21:51.306 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 2, 2023-09-12T16:21:51.307 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2023-09-12T16:21:51.307 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2023-09-12T16:21:51.307 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2023-09-12T16:21:51.307 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2023-09-12T16:21:51.307 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-09-12T16:21:51.307 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-12T16:21:51.308 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-09-12T16:21:51.308 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-09-12T16:21:51.308 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-09-12T16:21:51.308 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 329, 2023-09-12T16:21:51.308 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 455031, 2023-09-12T16:21:51.309 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-09-12T16:21:51.309 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 329, 2023-09-12T16:21:51.309 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-09-12T16:21:51.309 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-09-12T16:21:51.309 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 329, 2023-09-12T16:21:51.309 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 455031, 2023-09-12T16:21:51.310 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-09-12T16:21:51.310 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-09-12T16:21:51.310 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-09-12T16:21:51.310 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-09-12T16:21:51.310 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:51.311 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-09-12T16:21:51.311 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-09-12T16:21:51.706 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-09-12T16:21:51.707 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-09-12T16:21:51.707 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-09-12T16:21:51.707 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-09-12T16:21:51.707 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-09-12T16:21:51.736 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-12T16:21:51.736 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T16:21:51.736 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T16:21:51.780 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T16:21:51.780 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T16:21:51.847 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:21:51.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:51 smithi203 ceph-mon[103431]: pgmap v860: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-12T16:21:52.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.029+0000 7f9489ea5700 1 -- 172.21.15.130:0/176471999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94841006e0 msgr2=0x7f9484100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.029 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.029+0000 7f9489ea5700 1 --2- 172.21.15.130:0/176471999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94841006e0 0x7f9484100b40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f94780099c0 tx=0x7f9478009cd0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.030+0000 7f9489ea5700 1 -- 172.21.15.130:0/176471999 shutdown_connections 2023-09-12T16:21:52.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.030+0000 7f9489ea5700 1 --2- 172.21.15.130:0/176471999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9484101080 0x7f9484107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.036 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.030+0000 7f9489ea5700 1 --2- 172.21.15.130:0/176471999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94840ff4f0 0x7f94840ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.030+0000 7f9489ea5700 1 --2- 172.21.15.130:0/176471999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94841006e0 0x7f9484100b40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.030+0000 7f9489ea5700 1 -- 172.21.15.130:0/176471999 >> 172.21.15.130:0/176471999 conn(0x7f94840faa20 msgr2=0x7f94840fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:52.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.030+0000 7f9489ea5700 1 -- 172.21.15.130:0/176471999 shutdown_connections 2023-09-12T16:21:52.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.030+0000 7f9489ea5700 1 -- 172.21.15.130:0/176471999 wait complete. 2023-09-12T16:21:52.037 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.030+0000 7f9489ea5700 1 Processor -- start 2023-09-12T16:21:52.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.030+0000 7f9489ea5700 1 -- start start 2023-09-12T16:21:52.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9489ea5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94840ff4f0 0x7f9484069320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9489ea5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94841006e0 0x7f948419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9489ea5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9484101080 0x7f948419d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.038 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9489ea5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419ddc0 con 0x7f9484101080 2023-09-12T16:21:52.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9489ea5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419df30 con 0x7f94841006e0 2023-09-12T16:21:52.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9489ea5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419e0f0 con 0x7f94840ff4f0 2023-09-12T16:21:52.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f94837fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94840ff4f0 0x7f9484069320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9483fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9484101080 0x7f948419d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9482ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94841006e0 0x7f948419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.039 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f94837fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94840ff4f0 0x7f9484069320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36562/0 (socket says 172.21.15.130:36562) 2023-09-12T16:21:52.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9483fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9484101080 0x7f948419d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56954/0 (socket says 172.21.15.130:56954) 2023-09-12T16:21:52.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9482ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94841006e0 0x7f948419afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40210/0 (socket says 172.21.15.130:40210) 2023-09-12T16:21:52.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f94837fe700 1 -- 172.21.15.130:0/2276677809 learned_addr learned my addr 172.21.15.130:0/2276677809 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:52.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9483fff700 1 -- 172.21.15.130:0/2276677809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94840ff4f0 msgr2=0x7f9484069320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.040 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9483fff700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94840ff4f0 0x7f9484069320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9483fff700 1 -- 172.21.15.130:0/2276677809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94841006e0 msgr2=0x7f948419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9483fff700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94841006e0 0x7f948419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9483fff700 1 -- 172.21.15.130:0/2276677809 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780095f0 con 0x7f9484101080 2023-09-12T16:21:52.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f94837fe700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94840ff4f0 0x7f9484069320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:52.041 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9483fff700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9484101080 0x7f948419d5f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f947c00e970 tx=0x7f947c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:52.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9480ff9700 1 -- 172.21.15.130:0/2276677809 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c004220 con 0x7f9484101080 2023-09-12T16:21:52.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9480ff9700 1 -- 172.21.15.130:0/2276677809 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f947c004cf0 con 0x7f9484101080 2023-09-12T16:21:52.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948419e3d0 con 0x7f9484101080 2023-09-12T16:21:52.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9480ff9700 1 -- 172.21.15.130:0/2276677809 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c005660 con 0x7f9484101080 2023-09-12T16:21:52.042 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.031+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948419e8b0 con 0x7f9484101080 2023-09-12T16:21:52.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.032+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9484106a30 con 0x7f9484101080 2023-09-12T16:21:52.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.034+0000 7f9480ff9700 1 -- 172.21.15.130:0/2276677809 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f947c004750 con 0x7f9484101080 2023-09-12T16:21:52.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.034+0000 7f9480ff9700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f946c0a0690 0x7f946c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.034+0000 7f94837fe700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f946c0a0690 0x7f946c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.035+0000 7f9480ff9700 1 -- 172.21.15.130:0/2276677809 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f947c014070 con 0x7f9484101080 2023-09-12T16:21:52.043 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.035+0000 7f94837fe700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f946c0a0690 0x7f946c0a2b50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9474003ef0 tx=0x7f947400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:52.044 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.038+0000 7f9480ff9700 1 -- 172.21.15.130:0/2276677809 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f947c095570 con 0x7f9484101080 2023-09-12T16:21:52.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:51 smithi130 ceph-mon[98670]: pgmap v860: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-12T16:21:52.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:51 smithi159 ceph-mon[102522]: pgmap v860: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-12T16:21:52.229 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.229+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f948410ad10 con 0x7f9484101080 2023-09-12T16:21:52.231 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.231+0000 7f9480ff9700 1 -- 172.21.15.130:0/2276677809 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24671 (secure 0 0 0) 0x7f947c004e60 con 0x7f9484101080 2023-09-12T16:21:52.231 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:52.232 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":90,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:03:51.888444+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:21:52.233 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.233+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f946c0a0690 msgr2=0x7f946c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.233+0000 7f9489ea5700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f946c0a0690 0x7f946c0a2b50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9474003ef0 tx=0x7f947400b040 comp rx=0 tx=0).stop 2023-09-12T16:21:52.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.233+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9484101080 msgr2=0x7f948419d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.233+0000 7f9489ea5700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9484101080 0x7f948419d5f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f947c00e970 tx=0x7f947c00ec80 comp rx=0 tx=0).stop 2023-09-12T16:21:52.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.233+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 shutdown_connections 2023-09-12T16:21:52.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.233+0000 7f9489ea5700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f946c0a0690 0x7f946c0a2b50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.234 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.233+0000 7f9489ea5700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9484101080 0x7f948419d5f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.233+0000 7f9489ea5700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94840ff4f0 0x7f9484069320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.233+0000 7f9489ea5700 1 --2- 172.21.15.130:0/2276677809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94841006e0 0x7f948419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.234+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 >> 172.21.15.130:0/2276677809 conn(0x7f94840faa20 msgr2=0x7f9484103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:52.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.234+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 shutdown_connections 2023-09-12T16:21:52.235 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.234+0000 7f9489ea5700 1 -- 172.21.15.130:0/2276677809 wait complete. 2023-09-12T16:21:52.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T15:58:15.008580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T16:02:02.558269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T16:02:03.592552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-12T16:02:07.495861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-12T16:21:52.247 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T16:21:52.374 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.374+0000 7f87e77ea700 1 -- 172.21.15.130:0/2462786176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e01006e0 msgr2=0x7f87e0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.374+0000 7f87e77ea700 1 --2- 172.21.15.130:0/2462786176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e01006e0 0x7f87e0100b40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0099c0 tx=0x7f87dc009cd0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.374+0000 7f87e77ea700 1 -- 172.21.15.130:0/2462786176 shutdown_connections 2023-09-12T16:21:52.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.374+0000 7f87e77ea700 1 --2- 172.21.15.130:0/2462786176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87e0101080 0x7f87e0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.374+0000 7f87e77ea700 1 --2- 172.21.15.130:0/2462786176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87e00ff4f0 0x7f87e00ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.375 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.374+0000 7f87e77ea700 1 --2- 172.21.15.130:0/2462786176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e01006e0 0x7f87e0100b40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.376 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.374+0000 7f87e77ea700 1 -- 172.21.15.130:0/2462786176 >> 172.21.15.130:0/2462786176 conn(0x7f87e00faa60 msgr2=0x7f87e00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:52.376 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.374+0000 7f87e77ea700 1 -- 172.21.15.130:0/2462786176 shutdown_connections 2023-09-12T16:21:52.376 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.374+0000 7f87e77ea700 1 -- 172.21.15.130:0/2462786176 wait complete. 2023-09-12T16:21:52.378 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e77ea700 1 Processor -- start 2023-09-12T16:21:52.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e77ea700 1 -- start start 2023-09-12T16:21:52.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e77ea700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87e00ff4f0 0x7f87e00682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e77ea700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87e01006e0 0x7f87e019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.379 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e77ea700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101080 0x7f87e019f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e77ea700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01a0000 con 0x7f87e0101080 2023-09-12T16:21:52.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e77ea700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01a0170 con 0x7f87e00ff4f0 2023-09-12T16:21:52.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e77ea700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01a0330 con 0x7f87e01006e0 2023-09-12T16:21:52.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5d87700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101080 0x7f87e019f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5d87700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101080 0x7f87e019f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56968/0 (socket says 172.21.15.130:56968) 2023-09-12T16:21:52.380 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5d87700 1 -- 172.21.15.130:0/3227116381 learned_addr learned my addr 172.21.15.130:0/3227116381 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:52.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e4d85700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87e01006e0 0x7f87e019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5586700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87e00ff4f0 0x7f87e00682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5d87700 1 -- 172.21.15.130:0/3227116381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87e01006e0 msgr2=0x7f87e019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5d87700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87e01006e0 0x7f87e019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.381 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5d87700 1 -- 172.21.15.130:0/3227116381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87e00ff4f0 msgr2=0x7f87e00682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5d87700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87e00ff4f0 0x7f87e00682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5d87700 1 -- 172.21.15.130:0/3227116381 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0095f0 con 0x7f87e0101080 2023-09-12T16:21:52.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.375+0000 7f87e5d87700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101080 0x7f87e019f950 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f87d800ef30 tx=0x7f87d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:52.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.376+0000 7f87d27fc700 1 -- 172.21.15.130:0/3227116381 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8018070 con 0x7f87e0101080 2023-09-12T16:21:52.382 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.376+0000 7f87d27fc700 1 -- 172.21.15.130:0/3227116381 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87d800f040 con 0x7f87e0101080 2023-09-12T16:21:52.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.376+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e01a05b0 con 0x7f87e0101080 2023-09-12T16:21:52.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.376+0000 7f87d27fc700 1 -- 172.21.15.130:0/3227116381 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800cd00 con 0x7f87e0101080 2023-09-12T16:21:52.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.376+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e01a0ac0 con 0x7f87e0101080 2023-09-12T16:21:52.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.377+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e0106470 con 0x7f87e0101080 2023-09-12T16:21:52.383 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.378+0000 7f87d27fc700 1 -- 172.21.15.130:0/3227116381 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f87d8003a00 con 0x7f87e0101080 2023-09-12T16:21:52.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.378+0000 7f87d27fc700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87cc0a06c0 0x7f87cc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.378+0000 7f87d27fc700 1 -- 172.21.15.130:0/3227116381 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f87d80cba00 con 0x7f87e0101080 2023-09-12T16:21:52.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.378+0000 7f87e5586700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87cc0a06c0 0x7f87cc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.384 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.379+0000 7f87e5586700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87cc0a06c0 0x7f87cc0a2b80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f87e0100540 tx=0x7f87d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:52.385 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.383+0000 7f87d27fc700 1 -- 172.21.15.130:0/3227116381 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f87d8095250 con 0x7f87e0101080 2023-09-12T16:21:52.574 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.574+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87e002cf70 con 0x7f87e0101080 2023-09-12T16:21:52.575 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.575+0000 7f87d27fc700 1 -- 172.21.15.130:0/3227116381 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87d8098870 con 0x7f87e0101080 2023-09-12T16:21:52.575 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 1 2023-09-12T16:21:52.576 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87cc0a06c0 msgr2=0x7f87cc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87cc0a06c0 0x7f87cc0a2b80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f87e0100540 tx=0x7f87d4009260 comp rx=0 tx=0).stop 2023-09-12T16:21:52.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101080 msgr2=0x7f87e019f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101080 0x7f87e019f950 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f87d800ef30 tx=0x7f87d800c3a0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 shutdown_connections 2023-09-12T16:21:52.577 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f87cc0a06c0 0x7f87cc0a2b80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101080 0x7f87e019f950 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87e00ff4f0 0x7f87e00682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 --2- 172.21.15.130:0/3227116381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87e01006e0 0x7f87e019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 >> 172.21.15.130:0/3227116381 conn(0x7f87e00faa60 msgr2=0x7f87e00fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:52.578 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 shutdown_connections 2023-09-12T16:21:52.579 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.577+0000 7f87e77ea700 1 -- 172.21.15.130:0/3227116381 wait complete. 2023-09-12T16:21:52.590 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T16:21:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.717+0000 7fdba8d10700 1 -- 172.21.15.130:0/1228418214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41039e0 msgr2=0x7fdba4105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.717+0000 7fdba8d10700 1 --2- 172.21.15.130:0/1228418214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41039e0 0x7fdba4105dd0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009990 tx=0x7fdb94009ca0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.717+0000 7fdba8d10700 1 -- 172.21.15.130:0/1228418214 shutdown_connections 2023-09-12T16:21:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.717+0000 7fdba8d10700 1 --2- 172.21.15.130:0/1228418214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba4106310 0x7fdba4108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.717+0000 7fdba8d10700 1 --2- 172.21.15.130:0/1228418214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba41010b0 0x7fdba41034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.717+0000 7fdba8d10700 1 --2- 172.21.15.130:0/1228418214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41039e0 0x7fdba4105dd0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.717+0000 7fdba8d10700 1 -- 172.21.15.130:0/1228418214 >> 172.21.15.130:0/1228418214 conn(0x7fdba40faa80 msgr2=0x7fdba40fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.717+0000 7fdba8d10700 1 -- 172.21.15.130:0/1228418214 shutdown_connections 2023-09-12T16:21:52.719 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.717+0000 7fdba8d10700 1 -- 172.21.15.130:0/1228418214 wait complete. 2023-09-12T16:21:52.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba8d10700 1 Processor -- start 2023-09-12T16:21:52.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba8d10700 1 -- start start 2023-09-12T16:21:52.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba8d10700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41010b0 0x7fdba4100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.721 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba8d10700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba41039e0 0x7fdba40ff060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba8d10700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba4106310 0x7fdba40ff5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba8d10700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40ffd40 con 0x7fdba41010b0 2023-09-12T16:21:52.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba8d10700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40ffeb0 con 0x7fdba41039e0 2023-09-12T16:21:52.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba8d10700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba41a0300 con 0x7fdba4106310 2023-09-12T16:21:52.722 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba259c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41010b0 0x7fdba4100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba259c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41010b0 0x7fdba4100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56978/0 (socket says 172.21.15.130:56978) 2023-09-12T16:21:52.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba259c700 1 -- 172.21.15.130:0/242453731 learned_addr learned my addr 172.21.15.130:0/242453731 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:52.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba1d9b700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba41039e0 0x7fdba40ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba2d9d700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba4106310 0x7fdba40ff5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba259c700 1 -- 172.21.15.130:0/242453731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba4106310 msgr2=0x7fdba40ff5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.723 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba259c700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba4106310 0x7fdba40ff5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba259c700 1 -- 172.21.15.130:0/242453731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba41039e0 msgr2=0x7fdba40ff060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba259c700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba41039e0 0x7fdba40ff060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.718+0000 7fdba259c700 1 -- 172.21.15.130:0/242453731 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb940095f0 con 0x7fdba41010b0 2023-09-12T16:21:52.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.719+0000 7fdba259c700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41010b0 0x7fdba4100a20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800e9a0 tx=0x7fdb9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:52.724 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.719+0000 7fdb937fe700 1 -- 172.21.15.130:0/242453731 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98009790 con 0x7fdba41010b0 2023-09-12T16:21:52.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.719+0000 7fdb937fe700 1 -- 172.21.15.130:0/242453731 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb98010430 con 0x7fdba41010b0 2023-09-12T16:21:52.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.719+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba41a05e0 con 0x7fdba41010b0 2023-09-12T16:21:52.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.719+0000 7fdb937fe700 1 -- 172.21.15.130:0/242453731 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98018680 con 0x7fdba41010b0 2023-09-12T16:21:52.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.719+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba41a0af0 con 0x7fdba41010b0 2023-09-12T16:21:52.725 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.720+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba410be50 con 0x7fdba41010b0 2023-09-12T16:21:52.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.721+0000 7fdb937fe700 1 -- 172.21.15.130:0/242453731 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fdb980105a0 con 0x7fdba41010b0 2023-09-12T16:21:52.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.721+0000 7fdb937fe700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdb8c0a0640 0x7fdb8c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:52.726 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.721+0000 7fdba1d9b700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdb8c0a0640 0x7fdb8c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:52.727 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.721+0000 7fdb937fe700 1 -- 172.21.15.130:0/242453731 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdb98014070 con 0x7fdba41010b0 2023-09-12T16:21:52.727 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.722+0000 7fdba1d9b700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdb8c0a0640 0x7fdb8c0a2b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400b2e0 tx=0x7fdb94005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:52.727 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.725+0000 7fdb937fe700 1 -- 172.21.15.130:0/242453731 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdb98095a50 con 0x7fdba41010b0 2023-09-12T16:21:52.917 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.918+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdba41a0d10 con 0x7fdba41010b0 2023-09-12T16:21:52.918 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.918+0000 7fdb937fe700 1 -- 172.21.15.130:0/242453731 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb98026020 con 0x7fdba41010b0 2023-09-12T16:21:52.918 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:21:52.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdb8c0a0640 msgr2=0x7fdb8c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdb8c0a0640 0x7fdb8c0a2b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400b2e0 tx=0x7fdb94005fb0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.920 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41010b0 msgr2=0x7fdba4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:52.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41010b0 0x7fdba4100a20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800e9a0 tx=0x7fdb9800ecb0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 shutdown_connections 2023-09-12T16:21:52.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fdb8c0a0640 0x7fdb8c0a2b00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba4106310 0x7fdba40ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdba41010b0 0x7fdba4100a20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.921 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 --2- 172.21.15.130:0/242453731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba41039e0 0x7fdba40ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:52.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 >> 172.21.15.130:0/242453731 conn(0x7fdba40faa80 msgr2=0x7fdba41025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:52.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 shutdown_connections 2023-09-12T16:21:52.922 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:52.920+0000 7fdba8d10700 1 -- 172.21.15.130:0/242453731 wait complete. 2023-09-12T16:21:52.932 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T16:21:53.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 -- 172.21.15.130:0/4083850034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60101800 msgr2=0x7fae60101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 --2- 172.21.15.130:0/4083850034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60101800 0x7fae60101c60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae54009990 tx=0x7fae54009ca0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 -- 172.21.15.130:0/4083850034 shutdown_connections 2023-09-12T16:21:53.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 --2- 172.21.15.130:0/4083850034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae601021a0 0x7fae60108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.065 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 --2- 172.21.15.130:0/4083850034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60100610 0x7fae60100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 --2- 172.21.15.130:0/4083850034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60101800 0x7fae60101c60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 -- 172.21.15.130:0/4083850034 >> 172.21.15.130:0/4083850034 conn(0x7fae600fbb80 msgr2=0x7fae600fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:53.066 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 -- 172.21.15.130:0/4083850034 shutdown_connections 2023-09-12T16:21:53.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 -- 172.21.15.130:0/4083850034 wait complete. 2023-09-12T16:21:53.067 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.061+0000 7fae64a44700 1 Processor -- start 2023-09-12T16:21:53.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae64a44700 1 -- start start 2023-09-12T16:21:53.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae64a44700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60100610 0x7fae60196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae64a44700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60101800 0x7fae601968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.068 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae64a44700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae601021a0 0x7fae6019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae64a44700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6019dee0 con 0x7fae601021a0 2023-09-12T16:21:53.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae64a44700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6019e020 con 0x7fae60100610 2023-09-12T16:21:53.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae64a44700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6019e1b0 con 0x7fae60101800 2023-09-12T16:21:53.069 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5ed9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae601021a0 0x7fae6019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60100610 0x7fae60196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5dd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60101800 0x7fae601968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5ed9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae601021a0 0x7fae6019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:56996/0 (socket says 172.21.15.130:56996) 2023-09-12T16:21:53.070 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60100610 0x7fae60196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40250/0 (socket says 172.21.15.130:40250) 2023-09-12T16:21:53.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5ed9d700 1 -- 172.21.15.130:0/912914321 learned_addr learned my addr 172.21.15.130:0/912914321 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:53.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5ed9d700 1 -- 172.21.15.130:0/912914321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60101800 msgr2=0x7fae601968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.071 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5ed9d700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60101800 0x7fae601968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5ed9d700 1 -- 172.21.15.130:0/912914321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60100610 msgr2=0x7fae60196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5ed9d700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60100610 0x7fae60196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5ed9d700 1 -- 172.21.15.130:0/912914321 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58009c40 con 0x7fae601021a0 2023-09-12T16:21:53.072 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5dd9b700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60101800 0x7fae601968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:53.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5e59c700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60100610 0x7fae60196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:53.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae5ed9d700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae601021a0 0x7fae6019d910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fae5800c370 tx=0x7fae5800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:53.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae4f7fe700 1 -- 172.21.15.130:0/912914321 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58017070 con 0x7fae601021a0 2023-09-12T16:21:53.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae540095f0 con 0x7fae601021a0 2023-09-12T16:21:53.073 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae4f7fe700 1 -- 172.21.15.130:0/912914321 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5800f040 con 0x7fae601021a0 2023-09-12T16:21:53.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.062+0000 7fae4f7fe700 1 -- 172.21.15.130:0/912914321 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5800e050 con 0x7fae601021a0 2023-09-12T16:21:53.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.063+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6019e7b0 con 0x7fae601021a0 2023-09-12T16:21:53.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.064+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae60107c50 con 0x7fae601021a0 2023-09-12T16:21:53.074 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.064+0000 7fae4f7fe700 1 -- 172.21.15.130:0/912914321 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fae5800cb60 con 0x7fae601021a0 2023-09-12T16:21:53.075 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.065+0000 7fae4f7fe700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fae480a0640 0x7fae480a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.076 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.065+0000 7fae5e59c700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fae480a0640 0x7fae480a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.065+0000 7fae4f7fe700 1 -- 172.21.15.130:0/912914321 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fae580cb850 con 0x7fae601021a0 2023-09-12T16:21:53.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.065+0000 7fae5e59c700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fae480a0640 0x7fae480a2b00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae60101660 tx=0x7fae50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:53.077 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.069+0000 7fae4f7fe700 1 -- 172.21.15.130:0/912914321 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fae58094fb0 con 0x7fae601021a0 2023-09-12T16:21:53.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:52 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2276677809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:21:53.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:52 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3227116381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:21:53.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:52 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2276677809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:21:53.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:52 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3227116381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:21:53.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:52 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2276677809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:21:53.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:52 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3227116381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-12T16:21:53.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.264+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae6002cf10 con 0x7fae601021a0 2023-09-12T16:21:53.266 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.264+0000 7fae4f7fe700 1 -- 172.21.15.130:0/912914321 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae580985d0 con 0x7fae601021a0 2023-09-12T16:21:53.267 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 64 2023-09-12T16:21:53.267 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fae480a0640 msgr2=0x7fae480a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fae480a0640 0x7fae480a2b00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae60101660 tx=0x7fae50009260 comp rx=0 tx=0).stop 2023-09-12T16:21:53.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae601021a0 msgr2=0x7fae6019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae601021a0 0x7fae6019d910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fae5800c370 tx=0x7fae5800c680 comp rx=0 tx=0).stop 2023-09-12T16:21:53.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 shutdown_connections 2023-09-12T16:21:53.268 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fae480a0640 0x7fae480a2b00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae601021a0 0x7fae6019d910 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60100610 0x7fae60196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 --2- 172.21.15.130:0/912914321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60101800 0x7fae601968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 >> 172.21.15.130:0/912914321 conn(0x7fae600fbb80 msgr2=0x7fae60104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:53.269 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 shutdown_connections 2023-09-12T16:21:53.270 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.267+0000 7fae64a44700 1 -- 172.21.15.130:0/912914321 wait complete. 2023-09-12T16:21:53.279 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-12T16:21:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.404+0000 7f23e618f700 1 -- 172.21.15.130:0/1051052504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 msgr2=0x7f23e0107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.406 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.404+0000 7f23e618f700 1 --2- 172.21.15.130:0/1051052504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 0x7f23e0107930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009f90 tx=0x7f23d800b330 comp rx=0 tx=0).stop 2023-09-12T16:21:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.404+0000 7f23e618f700 1 -- 172.21.15.130:0/1051052504 shutdown_connections 2023-09-12T16:21:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.404+0000 7f23e618f700 1 --2- 172.21.15.130:0/1051052504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 0x7f23e0107930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.404+0000 7f23e618f700 1 --2- 172.21.15.130:0/1051052504 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23e00ff4f0 0x7f23e00ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.404+0000 7f23e618f700 1 --2- 172.21.15.130:0/1051052504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e01006e0 0x7f23e0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.407 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.404+0000 7f23e618f700 1 -- 172.21.15.130:0/1051052504 >> 172.21.15.130:0/1051052504 conn(0x7f23e00faa60 msgr2=0x7f23e00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.404+0000 7f23e618f700 1 -- 172.21.15.130:0/1051052504 shutdown_connections 2023-09-12T16:21:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.405+0000 7f23e618f700 1 -- 172.21.15.130:0/1051052504 wait complete. 2023-09-12T16:21:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.405+0000 7f23e618f700 1 Processor -- start 2023-09-12T16:21:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.405+0000 7f23e618f700 1 -- start start 2023-09-12T16:21:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.405+0000 7f23e618f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23e00ff4f0 0x7f23e01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.408 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.405+0000 7f23e618f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e01006e0 0x7f23e01968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.405+0000 7f23e618f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 0x7f23e019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.405+0000 7f23e618f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e019df30 con 0x7f23e00ff4f0 2023-09-12T16:21:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.405+0000 7f23e618f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e019e070 con 0x7f23e01006e0 2023-09-12T16:21:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.405+0000 7f23e618f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e019e200 con 0x7f23e0101080 2023-09-12T16:21:53.409 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23df7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23e00ff4f0 0x7f23e01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23deffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e01006e0 0x7f23e01968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23dffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 0x7f23e019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23dffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 0x7f23e019d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36622/0 (socket says 172.21.15.130:36622) 2023-09-12T16:21:53.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23dffff700 1 -- 172.21.15.130:0/3569118870 learned_addr learned my addr 172.21.15.130:0/3569118870 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:53.410 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23deffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e01006e0 0x7f23e01968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40266/0 (socket says 172.21.15.130:40266) 2023-09-12T16:21:53.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23df7fe700 1 -- 172.21.15.130:0/3569118870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 msgr2=0x7f23e019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23df7fe700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 0x7f23e019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23df7fe700 1 -- 172.21.15.130:0/3569118870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e01006e0 msgr2=0x7f23e01968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23df7fe700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e01006e0 0x7f23e01968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23df7fe700 1 -- 172.21.15.130:0/3569118870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d00095f0 con 0x7f23e00ff4f0 2023-09-12T16:21:53.411 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23dffff700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 0x7f23e019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:53.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23df7fe700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23e00ff4f0 0x7f23e01963b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f23d0005fd0 tx=0x7f23d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:53.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23dcff9700 1 -- 172.21.15.130:0/3569118870 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d000cbd0 con 0x7f23e00ff4f0 2023-09-12T16:21:53.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23dcff9700 1 -- 172.21.15.130:0/3569118870 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d000cd30 con 0x7f23e00ff4f0 2023-09-12T16:21:53.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d4009520 con 0x7f23e00ff4f0 2023-09-12T16:21:53.412 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23dcff9700 1 -- 172.21.15.130:0/3569118870 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0018710 con 0x7f23e00ff4f0 2023-09-12T16:21:53.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.406+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d8009c40 con 0x7f23e00ff4f0 2023-09-12T16:21:53.413 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.407+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e010ad00 con 0x7f23e00ff4f0 2023-09-12T16:21:53.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.408+0000 7f23dcff9700 1 -- 172.21.15.130:0/3569118870 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f23d0010980 con 0x7f23e00ff4f0 2023-09-12T16:21:53.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.409+0000 7f23dcff9700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23c80a0640 0x7f23c80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.409+0000 7f23deffd700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23c80a0640 0x7f23c80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.409+0000 7f23dcff9700 1 -- 172.21.15.130:0/3569118870 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f23d0014070 con 0x7f23e00ff4f0 2023-09-12T16:21:53.414 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.409+0000 7f23deffd700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23c80a0640 0x7f23c80a2b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009df0 tx=0x7f23d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:53.415 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.412+0000 7f23dcff9700 1 -- 172.21.15.130:0/3569118870 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f23d0096150 con 0x7f23e00ff4f0 2023-09-12T16:21:53.603 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.604+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f23e019e8f0 con 0x7f23e00ff4f0 2023-09-12T16:21:53.604 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.604+0000 7f23dcff9700 1 -- 172.21.15.130:0/3569118870 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f23d0099770 con 0x7f23e00ff4f0 2023-09-12T16:21:53.604 INFO:teuthology.orchestra.run.smithi130.stdout:pg_num: 32 2023-09-12T16:21:53.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23c80a0640 msgr2=0x7f23c80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23c80a0640 0x7f23c80a2b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009df0 tx=0x7f23d4009260 comp rx=0 tx=0).stop 2023-09-12T16:21:53.606 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23e00ff4f0 msgr2=0x7f23e01963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23e00ff4f0 0x7f23e01963b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f23d0005fd0 tx=0x7f23d000ee70 comp rx=0 tx=0).stop 2023-09-12T16:21:53.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 shutdown_connections 2023-09-12T16:21:53.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f23c80a0640 0x7f23c80a2b00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e0101080 0x7f23e019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.607 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23e00ff4f0 0x7f23e01963b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 --2- 172.21.15.130:0/3569118870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e01006e0 0x7f23e01968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 >> 172.21.15.130:0/3569118870 conn(0x7f23e00faa60 msgr2=0x7f23e0067760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:53.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 shutdown_connections 2023-09-12T16:21:53.608 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.606+0000 7f23e618f700 1 -- 172.21.15.130:0/3569118870 wait complete. 2023-09-12T16:21:53.619 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T16:21:53.742 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.742+0000 7f516a652700 1 -- 172.21.15.130:0/351235435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51641062d0 msgr2=0x7f5164108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.742+0000 7f516a652700 1 --2- 172.21.15.130:0/351235435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51641062d0 0x7f5164108740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f515c00b0a0 tx=0x7f515c00b3b0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.742+0000 7f516a652700 1 -- 172.21.15.130:0/351235435 shutdown_connections 2023-09-12T16:21:53.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.742+0000 7f516a652700 1 --2- 172.21.15.130:0/351235435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51641062d0 0x7f5164108740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.743 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.742+0000 7f516a652700 1 --2- 172.21.15.130:0/351235435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5164101070 0x7f5164103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.742+0000 7f516a652700 1 --2- 172.21.15.130:0/351235435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51641039a0 0x7f5164105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.742+0000 7f516a652700 1 -- 172.21.15.130:0/351235435 >> 172.21.15.130:0/351235435 conn(0x7f51640faa60 msgr2=0x7f51640fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:53.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.742+0000 7f516a652700 1 -- 172.21.15.130:0/351235435 shutdown_connections 2023-09-12T16:21:53.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.742+0000 7f516a652700 1 -- 172.21.15.130:0/351235435 wait complete. 2023-09-12T16:21:53.744 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.743+0000 7f516a652700 1 Processor -- start 2023-09-12T16:21:53.745 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.743+0000 7f516a652700 1 -- start start 2023-09-12T16:21:53.747 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.743+0000 7f516a652700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5164101070 0x7f51641963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.747 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.743+0000 7f516a652700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51641039a0 0x7f51641968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.743+0000 7f516a652700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51641062d0 0x7f516419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.743+0000 7f516a652700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419df20 con 0x7f51641062d0 2023-09-12T16:21:53.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.743+0000 7f516a652700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419e060 con 0x7f5164101070 2023-09-12T16:21:53.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.743+0000 7f516a652700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419e1f0 con 0x7f51641039a0 2023-09-12T16:21:53.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5163fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5164101070 0x7f51641963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.748 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5168bef700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51641062d0 0x7f516419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5163fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5164101070 0x7f51641963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40276/0 (socket says 172.21.15.130:40276) 2023-09-12T16:21:53.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f51637fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51641039a0 0x7f51641968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5163fff700 1 -- 172.21.15.130:0/1064842049 learned_addr learned my addr 172.21.15.130:0/1064842049 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:53.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5163fff700 1 -- 172.21.15.130:0/1064842049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51641039a0 msgr2=0x7f51641968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.749 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5163fff700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51641039a0 0x7f51641968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5163fff700 1 -- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51641062d0 msgr2=0x7f516419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5163fff700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51641062d0 0x7f516419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5163fff700 1 -- 172.21.15.130:0/1064842049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5154009520 con 0x7f5164101070 2023-09-12T16:21:53.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5168bef700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51641062d0 0x7f516419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:53.750 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f51637fe700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51641039a0 0x7f51641968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:53.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f5163fff700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5164101070 0x7f51641963a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f515400eb30 tx=0x7f515400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:53.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f51617fa700 1 -- 172.21.15.130:0/1064842049 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154009880 con 0x7f5164101070 2023-09-12T16:21:53.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5158009520 con 0x7f5164101070 2023-09-12T16:21:53.751 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f51617fa700 1 -- 172.21.15.130:0/1064842049 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51540044e0 con 0x7f5164101070 2023-09-12T16:21:53.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f51617fa700 1 -- 172.21.15.130:0/1064842049 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154003d80 con 0x7f5164101070 2023-09-12T16:21:53.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.744+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c009d00 con 0x7f5164101070 2023-09-12T16:21:53.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.745+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51640fc5e0 con 0x7f5164101070 2023-09-12T16:21:53.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.746+0000 7f51617fa700 1 -- 172.21.15.130:0/1064842049 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f5154003f00 con 0x7f5164101070 2023-09-12T16:21:53.752 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.747+0000 7f51617fa700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f514c0a0640 0x7f514c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:53.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.747+0000 7f51637fe700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f514c0a0640 0x7f514c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:53.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.747+0000 7f51617fa700 1 -- 172.21.15.130:0/1064842049 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5154014070 con 0x7f5164101070 2023-09-12T16:21:53.753 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.747+0000 7f51637fe700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f514c0a0640 0x7f514c0a2b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5158009fd0 tx=0x7f5158009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:53.754 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.751+0000 7f51617fa700 1 -- 172.21.15.130:0/1064842049 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5154095d50 con 0x7f5164101070 2023-09-12T16:21:53.965 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.966+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f516410be50 con 0x7f5164101070 2023-09-12T16:21:53.966 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.966+0000 7f51617fa700 1 -- 172.21.15.130:0/1064842049 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+598 (secure 0 0 0) 0x7f515400fe60 con 0x7f5164101070 2023-09-12T16:21:53.967 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/1706265745 2023-09-13T15:55:50.680182+0000 2023-09-12T16:21:53.967 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6800/3042728256 2023-09-13T15:55:50.680182+0000 2023-09-12T16:21:53.967 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/3742416190 2023-09-13T15:55:50.680182+0000 2023-09-12T16:21:53.968 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/2502139426 2023-09-13T15:55:28.728403+0000 2023-09-12T16:21:53.968 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6801/3042728256 2023-09-13T15:55:50.680182+0000 2023-09-12T16:21:53.968 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6800/840569029 2023-09-13T15:55:28.728403+0000 2023-09-12T16:21:53.968 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:6801/840569029 2023-09-13T15:55:28.728403+0000 2023-09-12T16:21:53.969 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/410531865 2023-09-13T15:55:50.680182+0000 2023-09-12T16:21:53.969 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/288931411 2023-09-13T15:55:28.728403+0000 2023-09-12T16:21:53.969 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130:0/1303099884 2023-09-13T15:55:28.728403+0000 2023-09-12T16:21:53.970 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f514c0a0640 msgr2=0x7f514c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f514c0a0640 0x7f514c0a2b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5158009fd0 tx=0x7f5158009260 comp rx=0 tx=0).stop 2023-09-12T16:21:53.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5164101070 msgr2=0x7f51641963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:53.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5164101070 0x7f51641963a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f515400eb30 tx=0x7f515400ee40 comp rx=0 tx=0).stop 2023-09-12T16:21:53.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 shutdown_connections 2023-09-12T16:21:53.971 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f514c0a0640 0x7f514c0a2b00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51641062d0 0x7f516419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5164101070 0x7f51641963a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 --2- 172.21.15.130:0/1064842049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51641039a0 0x7f51641968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:53.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 >> 172.21.15.130:0/1064842049 conn(0x7f51640faa60 msgr2=0x7f5164101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:53.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 shutdown_connections 2023-09-12T16:21:53.972 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:53.969+0000 7f516a652700 1 -- 172.21.15.130:0/1064842049 wait complete. 2023-09-12T16:21:53.973 INFO:teuthology.orchestra.run.smithi130.stderr:listed 10 entries 2023-09-12T16:21:53.981 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-09-12T16:21:53.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:21:53.982 DEBUG:teuthology.orchestra.run.smithi130:> sudo umount /tmp/cephfs 2023-09-12T16:21:54.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi130.front.sepia.ceph.com 2023-09-12T16:21:54.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:21:54.095 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-09-12T16:21:54.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:21:54.192 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list 2023-09-12T16:21:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:53 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/242453731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:21:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:53 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/912914321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:21:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:53 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3569118870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:21:54.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:53 smithi130 ceph-mon[98670]: pgmap v861: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2023-09-12T16:21:54.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:53 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/242453731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:21:54.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:53 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/912914321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:21:54.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:53 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3569118870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:21:54.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:53 smithi159 ceph-mon[102522]: pgmap v861: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2023-09-12T16:21:54.237 INFO:teuthology.orchestra.run.smithi130.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-09-12T16:21:54.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:21:54.238 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-09-12T16:21:54.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:53 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/242453731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-12T16:21:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:53 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/912914321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-12T16:21:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:53 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3569118870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-12T16:21:54.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:53 smithi203 ceph-mon[103431]: pgmap v861: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2023-09-12T16:21:54.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:21:54.314 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link delete ceph-brx 2023-09-12T16:21:54.417 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-12T16:21:54.417 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2023-09-12T16:21:54.417 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T16:21:54.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16027 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/a68feb87423dfea6c843a6fa748abfefdde57323d3838739cac01171353da7dd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/a68feb87423dfea6c843a6fa748abfefdde57323d3838739cac01171353da7dd/diff,workdir=/var/lib/containers/storage/overlay/a68feb87423dfea6c843a6fa748abfefdde57323d3838739cac01171353da7dd/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5e00b2013d58a33e8fa166efac4cd02d89e8d98a6a68f8d2f68210d5572622e2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/5e00b2013d58a33e8fa166efac4cd02d89e8d98a6a68f8d2f68210d5572622e2/diff,workdir=/var/lib/containers/storage/overlay/5e00b2013d58a33e8fa166efac4cd02d89e8d98a6a68f8d2f68210d5572622e2/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/914acc929f8f37496812ce1e0eaab846b720a2f42db7c723660d80e4faafc345/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/914acc929f8f37496812ce1e0eaab846b720a2f42db7c723660d80e4faafc345/diff,workdir=/var/lib/containers/storage/overlay/914acc929f8f37496812ce1e0eaab846b720a2f42db7c723660d80e4faafc345/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/eb3cb3b929d4047b0c3c5df031aae5eb0688287d6a65d27b6d97da385ef5fd95/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/eb3cb3b929d4047b0c3c5df031aae5eb0688287d6a65d27b6d97da385ef5fd95/diff,workdir=/var/lib/containers/storage/overlay/eb3cb3b929d4047b0c3c5df031aae5eb0688287d6a65d27b6d97da385ef5fd95/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/44e2631571b36b0b03872736807242418be6d6a7c7686e7c3d5a2ceaac581d64/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/44e2631571b36b0b03872736807242418be6d6a7c7686e7c3d5a2ceaac581d64/diff,workdir=/var/lib/containers/storage/overlay/44e2631571b36b0b03872736807242418be6d6a7c7686e7c3d5a2ceaac581d64/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b4518ca1cfdf11be56315004e2a5c8584fc352862a3e321dd8474c66f5a18729/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/b4518ca1cfdf11be56315004e2a5c8584fc352862a3e321dd8474c66f5a18729/diff,workdir=/var/lib/containers/storage/overlay/b4518ca1cfdf11be56315004e2a5c8584fc352862a3e321dd8474c66f5a18729/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b38ab2a1a9abf7d995157f5c8a04a009d92ca9481ea0bbef61a84823b1c5ecd7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/b38ab2a1a9abf7d995157f5c8a04a009d92ca9481ea0bbef61a84823b1c5ecd7/diff,workdir=/var/lib/containers/storage/overlay/b38ab2a1a9abf7d995157f5c8a04a009d92ca9481ea0bbef61a84823b1c5ecd7/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c6c566f27b838522ed2da1dd8c51c21ceaf5f595f07edc53e9595243ef6ff28d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/c6c566f27b838522ed2da1dd8c51c21ceaf5f595f07edc53e9595243ef6ff28d/diff,workdir=/var/lib/containers/storage/overlay/c6c566f27b838522ed2da1dd8c51c21ceaf5f595f07edc53e9595243ef6ff28d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a6401907d2bd0f9cfbf1852702c0cf23ae24afcda583c04e0dd72fb7839a17ce/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/a6401907d2bd0f9cfbf1852702c0cf23ae24afcda583c04e0dd72fb7839a17ce/diff,workdir=/var/lib/containers/storage/overlay/a6401907d2bd0f9cfbf1852702c0cf23ae24afcda583c04e0dd72fb7839a17ce/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/113a48cf47c4d42eec843ad72d41e66104b3b386582cc5f61aa4b90fa50c64c8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QDAFP2R6VIO5JRJRLX2P4CFKJE:/var/lib/containers/storage/overlay/l/L2CUNT2WGRUIKGS3GCW7GNLGOJ,upperdir=/var/lib/containers/storage/overlay/113a48cf47c4d42eec843ad72d41e66104b3b386582cc5f61aa4b90fa50c64c8/diff,workdir=/var/lib/containers/storage/overlay/113a48cf47c4d42eec843ad72d41e66104b3b386582cc5f61aa4b90fa50c64c8/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-09-12T16:21:54.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:21:54.438 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list 2023-09-12T16:21:54.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T16:21:54.495 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link delete ceph-brx 2023-09-12T16:21:54.665 INFO:teuthology.orchestra.run.smithi130.stderr:Cannot find device "ceph-brx" 2023-09-12T16:21:54.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T16:21:54.667 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-09-12T16:21:54.677 INFO:tasks.cephadm:Teardown begin 2023-09-12T16:21:54.677 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-12T16:21:54.677 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:21:54.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.808+0000 7febb8391700 1 -- 172.21.15.130:0/1425008581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb01006d0 msgr2=0x7febb0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:54.810 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.808+0000 7febb8391700 1 --2- 172.21.15.130:0/1425008581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb01006d0 0x7febb0100b30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feba0009990 tx=0x7feba0009ca0 comp rx=0 tx=0).stop 2023-09-12T16:21:54.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 -- 172.21.15.130:0/1425008581 shutdown_connections 2023-09-12T16:21:54.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 --2- 172.21.15.130:0/1425008581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febb0101070 0x7febb0107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:54.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 --2- 172.21.15.130:0/1425008581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febb00ff4e0 0x7febb00ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:54.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 --2- 172.21.15.130:0/1425008581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb01006d0 0x7febb0100b30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:54.811 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 -- 172.21.15.130:0/1425008581 >> 172.21.15.130:0/1425008581 conn(0x7febb00faa30 msgr2=0x7febb00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:54.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 -- 172.21.15.130:0/1425008581 shutdown_connections 2023-09-12T16:21:54.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 -- 172.21.15.130:0/1425008581 wait complete. 2023-09-12T16:21:54.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 Processor -- start 2023-09-12T16:21:54.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 -- start start 2023-09-12T16:21:54.812 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febb00ff4e0 0x7febb00682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:54.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb01006d0 0x7febb019b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:54.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febb0101070 0x7febb019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:54.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb019de20 con 0x7febb0101070 2023-09-12T16:21:54.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb019df90 con 0x7febb00ff4e0 2023-09-12T16:21:54.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.809+0000 7febb8391700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb019e150 con 0x7febb01006d0 2023-09-12T16:21:54.813 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb612d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febb00ff4e0 0x7febb00682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:54.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb692e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febb0101070 0x7febb019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:54.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb592c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb01006d0 0x7febb019b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:54.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb612d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febb00ff4e0 0x7febb00682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.130:40290/0 (socket says 172.21.15.130:40290) 2023-09-12T16:21:54.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb612d700 1 -- 172.21.15.130:0/427704649 learned_addr learned my addr 172.21.15.130:0/427704649 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:54.814 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb592c700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb01006d0 0x7febb019b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.130:36646/0 (socket says 172.21.15.130:36646) 2023-09-12T16:21:54.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb692e700 1 -- 172.21.15.130:0/427704649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb01006d0 msgr2=0x7febb019b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:54.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb692e700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb01006d0 0x7febb019b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:54.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb692e700 1 -- 172.21.15.130:0/427704649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febb00ff4e0 msgr2=0x7febb00682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:54.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb692e700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febb00ff4e0 0x7febb00682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:54.815 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb692e700 1 -- 172.21.15.130:0/427704649 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8009c40 con 0x7febb0101070 2023-09-12T16:21:54.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb612d700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febb00ff4e0 0x7febb00682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:54.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb692e700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febb0101070 0x7febb019d680 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7feba800efb0 tx=0x7feba800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:54.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7feba77fe700 1 -- 172.21.15.130:0/427704649 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8003a00 con 0x7febb0101070 2023-09-12T16:21:54.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7feba77fe700 1 -- 172.21.15.130:0/427704649 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feba800f040 con 0x7febb0101070 2023-09-12T16:21:54.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba00095f0 con 0x7febb0101070 2023-09-12T16:21:54.816 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7feba77fe700 1 -- 172.21.15.130:0/427704649 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8004540 con 0x7febb0101070 2023-09-12T16:21:54.817 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.810+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb019e760 con 0x7febb0101070 2023-09-12T16:21:54.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.811+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb010ad50 con 0x7febb0101070 2023-09-12T16:21:54.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.812+0000 7feba77fe700 1 -- 172.21.15.130:0/427704649 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7feba8003bf0 con 0x7febb0101070 2023-09-12T16:21:54.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.813+0000 7feba77fe700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7feb9c0a0690 0x7feb9c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:54.818 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.813+0000 7feba77fe700 1 -- 172.21.15.130:0/427704649 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7feba80cbdb0 con 0x7febb0101070 2023-09-12T16:21:54.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.813+0000 7febb612d700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7feb9c0a0690 0x7feb9c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:54.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.813+0000 7febb612d700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7feb9c0a0690 0x7feb9c0a2b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7febac005950 tx=0x7febac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:54.819 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:54.817+0000 7feba77fe700 1 -- 172.21.15.130:0/427704649 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7feba8095600 con 0x7febb0101070 2023-09-12T16:21:55.058 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.058+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febb0106420 con 0x7febb0101070 2023-09-12T16:21:55.059 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.059+0000 7feba77fe700 1 -- 172.21.15.130:0/427704649 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11346 (secure 0 0 0) 0x7feba8098c20 con 0x7febb0101070 2023-09-12T16:21:55.060 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:55.060 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:21:39.746036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:21:55.061 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.061+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7feb9c0a0690 msgr2=0x7feb9c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.061+0000 7febb8391700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7feb9c0a0690 0x7feb9c0a2b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7febac005950 tx=0x7febac009260 comp rx=0 tx=0).stop 2023-09-12T16:21:55.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.061+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febb0101070 msgr2=0x7febb019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.061+0000 7febb8391700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febb0101070 0x7febb019d680 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7feba800efb0 tx=0x7feba800c3a0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.062 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.062+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 shutdown_connections 2023-09-12T16:21:55.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.062+0000 7febb8391700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7feb9c0a0690 0x7feb9c0a2b50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.062+0000 7febb8391700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febb0101070 0x7febb019d680 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.062+0000 7febb8391700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febb00ff4e0 0x7febb00682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.063 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.062+0000 7febb8391700 1 --2- 172.21.15.130:0/427704649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb01006d0 0x7febb019b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.062+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 >> 172.21.15.130:0/427704649 conn(0x7febb00faa30 msgr2=0x7febb00fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:55.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.062+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 shutdown_connections 2023-09-12T16:21:55.064 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.062+0000 7febb8391700 1 -- 172.21.15.130:0/427704649 wait complete. 2023-09-12T16:21:55.064 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 35 2023-09-12T16:21:55.074 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T16:21:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.196+0000 7f4cfb386700 1 -- 172.21.15.130:0/4183344258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 msgr2=0x7f4cf40ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.196+0000 7f4cfb386700 1 --2- 172.21.15.130:0/4183344258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 0x7f4cf40ff920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40099c0 tx=0x7f4ce4009cd0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.196+0000 7f4cfb386700 1 -- 172.21.15.130:0/4183344258 shutdown_connections 2023-09-12T16:21:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.196+0000 7f4cfb386700 1 --2- 172.21.15.130:0/4183344258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4cf4101090 0x7f4cf4107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.196+0000 7f4cfb386700 1 --2- 172.21.15.130:0/4183344258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 0x7f4cf40ff920 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.196+0000 7f4cfb386700 1 --2- 172.21.15.130:0/4183344258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf41006f0 0x7f4cf4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.196+0000 7f4cfb386700 1 -- 172.21.15.130:0/4183344258 >> 172.21.15.130:0/4183344258 conn(0x7f4cf40faa30 msgr2=0x7f4cf40fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.196+0000 7f4cfb386700 1 -- 172.21.15.130:0/4183344258 shutdown_connections 2023-09-12T16:21:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.196+0000 7f4cfb386700 1 -- 172.21.15.130:0/4183344258 wait complete. 2023-09-12T16:21:55.200 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cfb386700 1 Processor -- start 2023-09-12T16:21:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cfb386700 1 -- start start 2023-09-12T16:21:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cfb386700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 0x7f4cf4195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cfb386700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4cf41006f0 0x7f4cf4195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cfb386700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf4101090 0x7f4cf419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cfb386700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf419ccd0 con 0x7f4cf40ff500 2023-09-12T16:21:55.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cfb386700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf419ce10 con 0x7f4cf41006f0 2023-09-12T16:21:55.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cfb386700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf419cfa0 con 0x7f4cf4101090 2023-09-12T16:21:55.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cf9122700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 0x7f4cf4195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cf9122700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 0x7f4cf4195150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57044/0 (socket says 172.21.15.130:57044) 2023-09-12T16:21:55.202 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cf9122700 1 -- 172.21.15.130:0/827211295 learned_addr learned my addr 172.21.15.130:0/827211295 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:55.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cf9923700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf4101090 0x7f4cf419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.197+0000 7f4cf8921700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4cf41006f0 0x7f4cf4195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cf9122700 1 -- 172.21.15.130:0/827211295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf4101090 msgr2=0x7f4cf419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cf9122700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf4101090 0x7f4cf419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cf9122700 1 -- 172.21.15.130:0/827211295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4cf41006f0 msgr2=0x7f4cf4195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.203 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cf9122700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4cf41006f0 0x7f4cf4195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cf9122700 1 -- 172.21.15.130:0/827211295 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce40095f0 con 0x7f4cf40ff500 2023-09-12T16:21:55.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cf9122700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 0x7f4cf4195150 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009fd0 tx=0x7f4ce4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:55.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cea7fc700 1 -- 172.21.15.130:0/827211295 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce401d070 con 0x7f4cf40ff500 2023-09-12T16:21:55.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cea7fc700 1 -- 172.21.15.130:0/827211295 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce4005030 con 0x7f4cf40ff500 2023-09-12T16:21:55.204 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf419d220 con 0x7f4cf40ff500 2023-09-12T16:21:55.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cea7fc700 1 -- 172.21.15.130:0/827211295 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4004a90 con 0x7f4cf40ff500 2023-09-12T16:21:55.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.198+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf419d640 con 0x7f4cf40ff500 2023-09-12T16:21:55.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.199+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf4106a70 con 0x7f4cf40ff500 2023-09-12T16:21:55.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.200+0000 7f4cea7fc700 1 -- 172.21.15.130:0/827211295 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f4ce40051a0 con 0x7f4cf40ff500 2023-09-12T16:21:55.205 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.200+0000 7f4cea7fc700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4ce00a0690 0x7f4ce00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.201+0000 7f4cea7fc700 1 -- 172.21.15.130:0/827211295 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4ce40cd0a0 con 0x7f4cf40ff500 2023-09-12T16:21:55.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.201+0000 7f4cf8921700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4ce00a0690 0x7f4ce00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.201+0000 7f4cf8921700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4ce00a0690 0x7f4ce00a2b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009c60 tx=0x7f4cf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:55.206 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.204+0000 7f4cea7fc700 1 -- 172.21.15.130:0/827211295 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4ce40968f0 con 0x7f4cf40ff500 2023-09-12T16:21:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1064842049' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-12T16:21:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:21:55.225 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi203 to 196.2M 2023-09-12T16:21:55.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: Unable to set osd_memory_target on smithi203 to 205740953: error parsing value: Value '205740953' is below minimum 939524096 2023-09-12T16:21:55.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.227 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.228 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-09-12T16:21:55.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: Adjusting osd_memory_target on smithi130 to 196.2M 2023-09-12T16:21:55.229 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-12T16:21:55.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: Unable to set osd_memory_target on smithi130 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-09-12T16:21:55.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:21:55.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:21:55.230 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:54 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:21:55.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1064842049' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-12T16:21:55.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:21:55.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi203 to 196.2M 2023-09-12T16:21:55.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: Unable to set osd_memory_target on smithi203 to 205740953: error parsing value: Value '205740953' is below minimum 939524096 2023-09-12T16:21:55.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-09-12T16:21:55.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: Adjusting osd_memory_target on smithi130 to 196.2M 2023-09-12T16:21:55.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-12T16:21:55.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: Unable to set osd_memory_target on smithi130 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-09-12T16:21:55.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:21:55.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:21:55.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:54 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:21:55.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1064842049' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-12T16:21:55.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-12T16:21:55.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi203 to 196.2M 2023-09-12T16:21:55.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: Unable to set osd_memory_target on smithi203 to 205740953: error parsing value: Value '205740953' is below minimum 939524096 2023-09-12T16:21:55.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-12T16:21:55.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-09-12T16:21:55.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: Adjusting osd_memory_target on smithi130 to 196.2M 2023-09-12T16:21:55.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-12T16:21:55.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: Unable to set osd_memory_target on smithi130 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-09-12T16:21:55.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-12T16:21:55.251 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-12T16:21:55.251 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:54 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' 2023-09-12T16:21:55.443 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.443+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cf410af60 con 0x7f4cf40ff500 2023-09-12T16:21:55.444 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.444+0000 7f4cea7fc700 1 -- 172.21.15.130:0/827211295 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f4ce4099f10 con 0x7f4cf40ff500 2023-09-12T16:21:55.444 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:55.445 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2074628565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2074628565},{"type":"v1","addr":"172.21.15.130:6835","nonce":2074628565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/372380878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":372380878},{"type":"v1","addr":"172.21.15.130:6837","nonce":372380878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14772,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2430896629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2430896629},{"type":"v1","addr":"172.21.15.130:6839","nonce":2430896629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6841/1672964981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6840","nonce":1672964981},{"type":"v1","addr":"172.21.15.130:6841","nonce":1672964981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2681608700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2681608700},{"type":"v1","addr":"172.21.15.159:6833","nonce":2681608700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24520,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/2012369878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2012369878},{"type":"v1","addr":"172.21.15.203:6833","nonce":2012369878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-12T16:02:04.422106+0000","modified":"2023-09-12T16:21:39.746036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24592,"mds_2":24574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3667735298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3667735298},{"type":"v1","addr":"172.21.15.159:6835","nonce":3667735298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/3413713845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3413713845},{"type":"v1","addr":"172.21.15.159:6837","nonce":3413713845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/1801270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801270748},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801270748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6837/3950241499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3950241499},{"type":"v1","addr":"172.21.15.203:6837","nonce":3950241499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6839/2671073294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2671073294},{"type":"v1","addr":"172.21.15.203:6839","nonce":2671073294}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/3094783118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3094783118},{"type":"v1","addr":"172.21.15.159:6839","nonce":3094783118}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T16:21:55.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4ce00a0690 msgr2=0x7f4ce00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4ce00a0690 0x7f4ce00a2b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009c60 tx=0x7f4cf0009260 comp rx=0 tx=0).stop 2023-09-12T16:21:55.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 msgr2=0x7f4cf4195150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.447 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 0x7f4cf4195150 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009fd0 tx=0x7f4ce4009580 comp rx=0 tx=0).stop 2023-09-12T16:21:55.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 shutdown_connections 2023-09-12T16:21:55.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f4ce00a0690 0x7f4ce00a2b50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf4101090 0x7f4cf419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.448 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4cf40ff500 0x7f4cf4195150 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 --2- 172.21.15.130:0/827211295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4cf41006f0 0x7f4cf4195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.446+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 >> 172.21.15.130:0/827211295 conn(0x7f4cf40faa30 msgr2=0x7f4cf4103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:55.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.447+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 shutdown_connections 2023-09-12T16:21:55.449 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.447+0000 7f4cfb386700 1 -- 172.21.15.130:0/827211295 wait complete. 2023-09-12T16:21:55.449 INFO:teuthology.orchestra.run.smithi130.stderr:dumped fsmap epoch 35 2023-09-12T16:21:55.459 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T16:21:55.583 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.580+0000 7f0018e65700 1 -- 172.21.15.130:0/3333172480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00141006e0 msgr2=0x7f0014100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.580+0000 7f0018e65700 1 --2- 172.21.15.130:0/3333172480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00141006e0 0x7f0014100b40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f00040099c0 tx=0x7f0004009cd0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.580+0000 7f0018e65700 1 -- 172.21.15.130:0/3333172480 shutdown_connections 2023-09-12T16:21:55.584 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.580+0000 7f0018e65700 1 --2- 172.21.15.130:0/3333172480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0014101080 0x7f0014107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.580+0000 7f0018e65700 1 --2- 172.21.15.130:0/3333172480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00140ff4f0 0x7f00140ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.580+0000 7f0018e65700 1 --2- 172.21.15.130:0/3333172480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00141006e0 0x7f0014100b40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.580+0000 7f0018e65700 1 -- 172.21.15.130:0/3333172480 >> 172.21.15.130:0/3333172480 conn(0x7f00140faa20 msgr2=0x7f00140fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:55.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.580+0000 7f0018e65700 1 -- 172.21.15.130:0/3333172480 shutdown_connections 2023-09-12T16:21:55.585 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.580+0000 7f0018e65700 1 -- 172.21.15.130:0/3333172480 wait complete. 2023-09-12T16:21:55.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0018e65700 1 Processor -- start 2023-09-12T16:21:55.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0018e65700 1 -- start start 2023-09-12T16:21:55.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0018e65700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00140ff4f0 0x7f00141984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0018e65700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00141006e0 0x7f00141989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.586 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0018e65700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0014101080 0x7f001419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0018e65700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00141a0030 con 0x7f00140ff4f0 2023-09-12T16:21:55.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0018e65700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00141a0170 con 0x7f0014101080 2023-09-12T16:21:55.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0018e65700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00141a0300 con 0x7f00141006e0 2023-09-12T16:21:55.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f001259c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00140ff4f0 0x7f00141984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.587 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f001259c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00140ff4f0 0x7f00141984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57058/0 (socket says 172.21.15.130:57058) 2023-09-12T16:21:55.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f001259c700 1 -- 172.21.15.130:0/3563956684 learned_addr learned my addr 172.21.15.130:0/3563956684 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:55.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0011d9b700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00141006e0 0x7f00141989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.581+0000 7f0012d9d700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0014101080 0x7f001419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.588 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f0011d9b700 1 -- 172.21.15.130:0/3563956684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0014101080 msgr2=0x7f001419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f0011d9b700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0014101080 0x7f001419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f0011d9b700 1 -- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00140ff4f0 msgr2=0x7f00141984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f0011d9b700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00140ff4f0 0x7f00141984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.589 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f0011d9b700 1 -- 172.21.15.130:0/3563956684 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00040095f0 con 0x7f00141006e0 2023-09-12T16:21:55.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f001259c700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00140ff4f0 0x7f00141984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:55.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f0012d9d700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0014101080 0x7f001419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:55.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f0011d9b700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00141006e0 0x7f00141989f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00040098e0 tx=0x7f000400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:55.590 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f00037fe700 1 -- 172.21.15.130:0/3563956684 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000401d070 con 0x7f00141006e0 2023-09-12T16:21:55.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00141a0580 con 0x7f00141006e0 2023-09-12T16:21:55.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f00037fe700 1 -- 172.21.15.130:0/3563956684 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0004005020 con 0x7f00141006e0 2023-09-12T16:21:55.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f00037fe700 1 -- 172.21.15.130:0/3563956684 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004004a70 con 0x7f00141006e0 2023-09-12T16:21:55.591 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.582+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00141a09a0 con 0x7f00141006e0 2023-09-12T16:21:55.592 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.583+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001410ad60 con 0x7f00141006e0 2023-09-12T16:21:55.592 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.584+0000 7f00037fe700 1 -- 172.21.15.130:0/3563956684 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f0004005400 con 0x7f00141006e0 2023-09-12T16:21:55.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.584+0000 7f00037fe700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efffc0a0690 0x7efffc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.585+0000 7f00037fe700 1 -- 172.21.15.130:0/3563956684 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f00040ccb20 con 0x7f00141006e0 2023-09-12T16:21:55.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.585+0000 7f001259c700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efffc0a0690 0x7efffc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.593 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.585+0000 7f001259c700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efffc0a0690 0x7efffc0a2b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0008005950 tx=0x7f0008009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:55.594 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.589+0000 7f00037fe700 1 -- 172.21.15.130:0/3563956684 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0004096370 con 0x7f00141006e0 2023-09-12T16:21:55.782 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.783+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f001410af50 con 0x7f00141006e0 2023-09-12T16:21:55.784 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.785+0000 7f00037fe700 1 -- 172.21.15.130:0/3563956684 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24671 (secure 0 0 0) 0x7f0004005190 con 0x7f00141006e0 2023-09-12T16:21:55.785 INFO:teuthology.orchestra.run.smithi130.stdout: 2023-09-12T16:21:55.785 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":90,"fsid":"9ef22ea2-5184-11ee-9ab7-7b867c8bd7da","created":"2023-09-12T15:55:02.305146+0000","modified":"2023-09-12T16:03:51.888444+0000","last_up_change":"2023-09-12T16:01:22.820056+0000","last_in_change":"2023-09-12T16:01:05.151616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T15:58:15.008580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T16:02:02.558269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T16:02:03.592552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-12T16:02:07.495861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10fe9546-2d1e-4ceb-8afb-ccfc1d09a88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6803","nonce":1767854979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6805","nonce":1767854979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6809","nonce":1767854979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":1767854979},{"type":"v1","addr":"172.21.15.130:6807","nonce":1767854979}]},"public_addr":"172.21.15.130:6803/1767854979","cluster_addr":"172.21.15.130:6805/1767854979","heartbeat_back_addr":"172.21.15.130:6809/1767854979","heartbeat_front_addr":"172.21.15.130:6807/1767854979","state":["exists","up"]},{"osd":1,"uuid":"8fe17faa-52a3-4d63-878a-da07c02679b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6801","nonce":1684620878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6803","nonce":1684620878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6807","nonce":1684620878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1684620878},{"type":"v1","addr":"172.21.15.159:6805","nonce":1684620878}]},"public_addr":"172.21.15.159:6801/1684620878","cluster_addr":"172.21.15.159:6803/1684620878","heartbeat_back_addr":"172.21.15.159:6807/1684620878","heartbeat_front_addr":"172.21.15.159:6805/1684620878","state":["exists","up"]},{"osd":2,"uuid":"3c101604-31dd-462a-88ee-7d55829e0d2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6801","nonce":3797136832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6803","nonce":3797136832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6807","nonce":3797136832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3797136832},{"type":"v1","addr":"172.21.15.203:6805","nonce":3797136832}]},"public_addr":"172.21.15.203:6801/3797136832","cluster_addr":"172.21.15.203:6803/3797136832","heartbeat_back_addr":"172.21.15.203:6807/3797136832","heartbeat_front_addr":"172.21.15.203:6805/3797136832","state":["exists","up"]},{"osd":3,"uuid":"7045b414-e695-4f08-9764-72c802812750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6811","nonce":3784588793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6813","nonce":3784588793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6817","nonce":3784588793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3784588793},{"type":"v1","addr":"172.21.15.130:6815","nonce":3784588793}]},"public_addr":"172.21.15.130:6811/3784588793","cluster_addr":"172.21.15.130:6813/3784588793","heartbeat_back_addr":"172.21.15.130:6817/3784588793","heartbeat_front_addr":"172.21.15.130:6815/3784588793","state":["exists","up"]},{"osd":4,"uuid":"8e61d2d4-2479-45ab-a033-28ae1aeb8202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6809","nonce":2237508615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6811","nonce":2237508615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6815","nonce":2237508615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2237508615},{"type":"v1","addr":"172.21.15.159:6813","nonce":2237508615}]},"public_addr":"172.21.15.159:6809/2237508615","cluster_addr":"172.21.15.159:6811/2237508615","heartbeat_back_addr":"172.21.15.159:6815/2237508615","heartbeat_front_addr":"172.21.15.159:6813/2237508615","state":["exists","up"]},{"osd":5,"uuid":"5fc371aa-08c6-49f0-8f7f-0ce25705e987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6809","nonce":2191150970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6811","nonce":2191150970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6815","nonce":2191150970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2191150970},{"type":"v1","addr":"172.21.15.203:6813","nonce":2191150970}]},"public_addr":"172.21.15.203:6809/2191150970","cluster_addr":"172.21.15.203:6811/2191150970","heartbeat_back_addr":"172.21.15.203:6815/2191150970","heartbeat_front_addr":"172.21.15.203:6813/2191150970","state":["exists","up"]},{"osd":6,"uuid":"6cb4de31-02a3-4822-944d-14a9fe33f79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6819","nonce":291320749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6821","nonce":291320749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6825","nonce":291320749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":291320749},{"type":"v1","addr":"172.21.15.130:6823","nonce":291320749}]},"public_addr":"172.21.15.130:6819/291320749","cluster_addr":"172.21.15.130:6821/291320749","heartbeat_back_addr":"172.21.15.130:6825/291320749","heartbeat_front_addr":"172.21.15.130:6823/291320749","state":["exists","up"]},{"osd":7,"uuid":"d8c5305d-d340-496a-8967-461ee63ee0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6817","nonce":315414716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6819","nonce":315414716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6823","nonce":315414716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":315414716},{"type":"v1","addr":"172.21.15.159:6821","nonce":315414716}]},"public_addr":"172.21.15.159:6817/315414716","cluster_addr":"172.21.15.159:6819/315414716","heartbeat_back_addr":"172.21.15.159:6823/315414716","heartbeat_front_addr":"172.21.15.159:6821/315414716","state":["exists","up"]},{"osd":8,"uuid":"52e84740-7c08-4017-b5d4-3a20f0ecad18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6817","nonce":2706023432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6819","nonce":2706023432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6823","nonce":2706023432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2706023432},{"type":"v1","addr":"172.21.15.203:6821","nonce":2706023432}]},"public_addr":"172.21.15.203:6817/2706023432","cluster_addr":"172.21.15.203:6819/2706023432","heartbeat_back_addr":"172.21.15.203:6823/2706023432","heartbeat_front_addr":"172.21.15.203:6821/2706023432","state":["exists","up"]},{"osd":9,"uuid":"9c95841a-2cba-411e-b7e6-f958256adc5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6827","nonce":715792473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6829","nonce":715792473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6833","nonce":715792473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":715792473},{"type":"v1","addr":"172.21.15.130:6831","nonce":715792473}]},"public_addr":"172.21.15.130:6827/715792473","cluster_addr":"172.21.15.130:6829/715792473","heartbeat_back_addr":"172.21.15.130:6833/715792473","heartbeat_front_addr":"172.21.15.130:6831/715792473","state":["exists","up"]},{"osd":10,"uuid":"742fd1e6-2b06-438c-8d3d-d7249d475f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6825","nonce":1888655979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6827","nonce":1888655979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6831","nonce":1888655979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1888655979},{"type":"v1","addr":"172.21.15.159:6829","nonce":1888655979}]},"public_addr":"172.21.15.159:6825/1888655979","cluster_addr":"172.21.15.159:6827/1888655979","heartbeat_back_addr":"172.21.15.159:6831/1888655979","heartbeat_front_addr":"172.21.15.159:6829/1888655979","state":["exists","up"]},{"osd":11,"uuid":"b32e5d8c-4556-4d0e-83ce-9f35138be3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6825","nonce":768600250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6827","nonce":768600250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6831","nonce":768600250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":768600250},{"type":"v1","addr":"172.21.15.203:6829","nonce":768600250}]},"public_addr":"172.21.15.203:6825/768600250","cluster_addr":"172.21.15.203:6827/768600250","heartbeat_back_addr":"172.21.15.203:6831/768600250","heartbeat_front_addr":"172.21.15.203:6829/768600250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:30.316543+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:57:50.644471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:12.331757+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:33.411184+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:58:53.923320+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:15.456167+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:36.221464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T15:59:56.734029+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:18.505511+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:39.247618+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:00:59.727084+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T16:01:21.549896+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:0/1706265745":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/3742416190":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/2502139426":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/3042728256":"2023-09-13T15:55:50.680182+0000","172.21.15.130:6800/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:6801/840569029":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/410531865":"2023-09-13T15:55:50.680182+0000","172.21.15.130:0/288931411":"2023-09-13T15:55:28.728403+0000","172.21.15.130:0/1303099884":"2023-09-13T15:55:28.728403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T16:21:55.787 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efffc0a0690 msgr2=0x7efffc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efffc0a0690 0x7efffc0a2b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0008005950 tx=0x7f0008009260 comp rx=0 tx=0).stop 2023-09-12T16:21:55.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00141006e0 msgr2=0x7f00141989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00141006e0 0x7f00141989f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00040098e0 tx=0x7f000400f690 comp rx=0 tx=0).stop 2023-09-12T16:21:55.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 shutdown_connections 2023-09-12T16:21:55.788 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7efffc0a0690 0x7efffc0a2b50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0014101080 0x7f001419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00140ff4f0 0x7f00141984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.789 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 --2- 172.21.15.130:0/3563956684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00141006e0 0x7f00141989f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 >> 172.21.15.130:0/3563956684 conn(0x7f00140faa20 msgr2=0x7f0014068850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:55.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 shutdown_connections 2023-09-12T16:21:55.790 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.787+0000 7f0018e65700 1 -- 172.21.15.130:0/3563956684 wait complete. 2023-09-12T16:21:55.800 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-12T16:21:55.931 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 -- 172.21.15.130:0/994203951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8941006e0 msgr2=0x7fa894100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.931 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 --2- 172.21.15.130:0/994203951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8941006e0 0x7fa894100b40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa8800098f0 tx=0x7fa880009c00 comp rx=0 tx=0).stop 2023-09-12T16:21:55.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 -- 172.21.15.130:0/994203951 shutdown_connections 2023-09-12T16:21:55.932 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 --2- 172.21.15.130:0/994203951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894101080 0x7fa894107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 --2- 172.21.15.130:0/994203951 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8940ff4f0 0x7fa8940ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 --2- 172.21.15.130:0/994203951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8941006e0 0x7fa894100b40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.933 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 -- 172.21.15.130:0/994203951 >> 172.21.15.130:0/994203951 conn(0x7fa8940faa60 msgr2=0x7fa8940fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:55.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 -- 172.21.15.130:0/994203951 shutdown_connections 2023-09-12T16:21:55.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 -- 172.21.15.130:0/994203951 wait complete. 2023-09-12T16:21:55.934 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 Processor -- start 2023-09-12T16:21:55.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.926+0000 7fa89aafa700 1 -- start start 2023-09-12T16:21:55.935 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa89aafa700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8940ff4f0 0x7fa894195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa89aafa700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8941006e0 0x7fa8941957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa89aafa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa894101080 0x7fa89419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa89aafa700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419ce10 con 0x7fa8940ff4f0 2023-09-12T16:21:55.936 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa89aafa700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419cf50 con 0x7fa8941006e0 2023-09-12T16:21:55.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa89aafa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419d0e0 con 0x7fa894101080 2023-09-12T16:21:55.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa898896700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8940ff4f0 0x7fa894195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.937 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa898896700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8940ff4f0 0x7fa894195290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57072/0 (socket says 172.21.15.130:57072) 2023-09-12T16:21:55.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa898896700 1 -- 172.21.15.130:0/3030682434 learned_addr learned my addr 172.21.15.130:0/3030682434 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:55.938 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa899097700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa894101080 0x7fa89419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa893fff700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8941006e0 0x7fa8941957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.939 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa899097700 1 -- 172.21.15.130:0/3030682434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8941006e0 msgr2=0x7fa8941957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa899097700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8941006e0 0x7fa8941957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.940 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa899097700 1 -- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8940ff4f0 msgr2=0x7fa894195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:55.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa899097700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8940ff4f0 0x7fa894195290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:55.941 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa899097700 1 -- 172.21.15.130:0/3030682434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880095f0 con 0x7fa894101080 2023-09-12T16:21:55.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa898896700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8940ff4f0 0x7fa894195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:55.942 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa893fff700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8941006e0 0x7fa8941957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:55.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa899097700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa894101080 0x7fa89419c840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00efc0 tx=0x7fa88c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:55.943 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa891ffb700 1 -- 172.21.15.130:0/3030682434 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c018070 con 0x7fa894101080 2023-09-12T16:21:55.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa891ffb700 1 -- 172.21.15.130:0/3030682434 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa88c00f040 con 0x7fa894101080 2023-09-12T16:21:55.944 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa880009520 con 0x7fa894101080 2023-09-12T16:21:55.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.927+0000 7fa891ffb700 1 -- 172.21.15.130:0/3030682434 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c00cd40 con 0x7fa894101080 2023-09-12T16:21:55.945 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.928+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa894067b30 con 0x7fa894101080 2023-09-12T16:21:55.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.929+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa894068520 con 0x7fa894101080 2023-09-12T16:21:55.946 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.929+0000 7fa891ffb700 1 -- 172.21.15.130:0/3030682434 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fa88c003a00 con 0x7fa894101080 2023-09-12T16:21:55.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.930+0000 7fa891ffb700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa8840a06c0 0x7fa8840a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:55.947 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.930+0000 7fa898896700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa8840a06c0 0x7fa8840a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:55.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.930+0000 7fa891ffb700 1 -- 172.21.15.130:0/3030682434 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa88c0cb9a0 con 0x7fa894101080 2023-09-12T16:21:55.949 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.930+0000 7fa898896700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa8840a06c0 0x7fa8840a2b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa888005950 tx=0x7fa888010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:55.950 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:55.935+0000 7fa891ffb700 1 -- 172.21.15.130:0/3030682434 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa88c0951f0 con 0x7fa894101080 2023-09-12T16:21:56.169 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.170+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa894105e80 con 0x7fa894101080 2023-09-12T16:21:56.172 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:55 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/427704649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:21:56.172 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:55 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/827211295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:21:56.172 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:55 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3563956684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:21:56.172 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:55 smithi130 ceph-mon[98670]: pgmap v862: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-09-12T16:21:56.177 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.177+0000 7fa891ffb700 1 -- 172.21.15.130:0/3030682434 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa88c01b320 con 0x7fa894101080 2023-09-12T16:21:56.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa8840a06c0 msgr2=0x7fa8840a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:56.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa8840a06c0 0x7fa8840a2b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa888005950 tx=0x7fa888010040 comp rx=0 tx=0).stop 2023-09-12T16:21:56.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa894101080 msgr2=0x7fa89419c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:56.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa894101080 0x7fa89419c840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00efc0 tx=0x7fa88c00c3a0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 shutdown_connections 2023-09-12T16:21:56.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fa8840a06c0 0x7fa8840a2b80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa894101080 0x7fa89419c840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8940ff4f0 0x7fa894195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 --2- 172.21.15.130:0/3030682434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8941006e0 0x7fa8941957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.180+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 >> 172.21.15.130:0/3030682434 conn(0x7fa8940faa60 msgr2=0x7fa8940fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:56.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.181+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 shutdown_connections 2023-09-12T16:21:56.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.181+0000 7fa89aafa700 1 -- 172.21.15.130:0/3030682434 wait complete. 2023-09-12T16:21:56.183 INFO:teuthology.orchestra.run.smithi130.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-12T16:21:56.193 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-12T16:21:56.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:55 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/427704649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:21:56.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:55 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/827211295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:21:56.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:55 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3563956684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:21:56.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:55 smithi159 ceph-mon[102522]: pgmap v862: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-09-12T16:21:56.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:55 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/427704649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:21:56.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:55 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/827211295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-12T16:21:56.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:55 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3563956684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-12T16:21:56.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:55 smithi203 ceph-mon[103431]: pgmap v862: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-09-12T16:21:56.322 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 -- 172.21.15.130:0/4117676945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d41006e0 msgr2=0x7f00d4100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 --2- 172.21.15.130:0/4117676945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d41006e0 0x7f00d4100b40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f00d00099c0 tx=0x7f00d0009cd0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 -- 172.21.15.130:0/4117676945 shutdown_connections 2023-09-12T16:21:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 --2- 172.21.15.130:0/4117676945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d4101080 0x7f00d4107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 --2- 172.21.15.130:0/4117676945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00d40ff4f0 0x7f00d40ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.323 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 --2- 172.21.15.130:0/4117676945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d41006e0 0x7f00d4100b40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 -- 172.21.15.130:0/4117676945 >> 172.21.15.130:0/4117676945 conn(0x7f00d40faa60 msgr2=0x7f00d40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:56.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 -- 172.21.15.130:0/4117676945 shutdown_connections 2023-09-12T16:21:56.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 -- 172.21.15.130:0/4117676945 wait complete. 2023-09-12T16:21:56.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 Processor -- start 2023-09-12T16:21:56.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.321+0000 7f00dbf76700 1 -- start start 2023-09-12T16:21:56.324 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00dbf76700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00d40ff4f0 0x7f00d41951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00dbf76700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41006e0 0x7f00d41956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00dbf76700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d4101080 0x7f00d419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00dbf76700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d419cd20 con 0x7f00d4101080 2023-09-12T16:21:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00dbf76700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d419ce60 con 0x7f00d40ff4f0 2023-09-12T16:21:56.325 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00dbf76700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d419cff0 con 0x7f00d41006e0 2023-09-12T16:21:56.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00da513700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d4101080 0x7f00d419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:56.326 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00da513700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d4101080 0x7f00d419c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57088/0 (socket says 172.21.15.130:57088) 2023-09-12T16:21:56.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00d9511700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41006e0 0x7f00d41956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:56.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00da513700 1 -- 172.21.15.130:0/3124522452 learned_addr learned my addr 172.21.15.130:0/3124522452 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:56.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00d9d12700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00d40ff4f0 0x7f00d41951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:56.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00d9511700 1 -- 172.21.15.130:0/3124522452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00d40ff4f0 msgr2=0x7f00d41951a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:56.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00d9511700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00d40ff4f0 0x7f00d41951a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00d9511700 1 -- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d4101080 msgr2=0x7f00d419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:56.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00d9511700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d4101080 0x7f00d419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:56.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00d9511700 1 -- 172.21.15.130:0/3124522452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d00095f0 con 0x7f00d41006e0 2023-09-12T16:21:56.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00da513700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d4101080 0x7f00d419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T16:21:56.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00d9511700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41006e0 0x7f00d41956e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009fd0 tx=0x7f00d0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:56.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00c6ffd700 1 -- 172.21.15.130:0/3124522452 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d001d070 con 0x7f00d41006e0 2023-09-12T16:21:56.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d4068790 con 0x7f00d41006e0 2023-09-12T16:21:56.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00c6ffd700 1 -- 172.21.15.130:0/3124522452 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00d0005020 con 0x7f00d41006e0 2023-09-12T16:21:56.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00c6ffd700 1 -- 172.21.15.130:0/3124522452 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0004a90 con 0x7f00d41006e0 2023-09-12T16:21:56.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.322+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d4068be0 con 0x7f00d41006e0 2023-09-12T16:21:56.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.323+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d4069630 con 0x7f00d41006e0 2023-09-12T16:21:56.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.324+0000 7f00c6ffd700 1 -- 172.21.15.130:0/3124522452 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f00d0005190 con 0x7f00d41006e0 2023-09-12T16:21:56.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.325+0000 7f00c6ffd700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00c00a0640 0x7f00c00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:56.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.325+0000 7f00c6ffd700 1 -- 172.21.15.130:0/3124522452 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f00d00ccc90 con 0x7f00d41006e0 2023-09-12T16:21:56.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.325+0000 7f00d9d12700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00c00a0640 0x7f00c00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:56.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.325+0000 7f00d9d12700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00c00a0640 0x7f00c00a2b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f00d4100540 tx=0x7f00c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:56.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.329+0000 7f00c6ffd700 1 -- 172.21.15.130:0/3124522452 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f00d00963e0 con 0x7f00d41006e0 2023-09-12T16:21:56.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-a[98646]: 2023-09-12T16:21:56.171+0000 7f5eb80d7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-12T16:21:56.572 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:56.572+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f00d410af20 con 0x7f00d41006e0 2023-09-12T16:21:56.726 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-b[139180]: ignoring --setuser ceph since I am not root 2023-09-12T16:21:56.726 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-b[139180]: ignoring --setgroup ceph since I am not root 2023-09-12T16:21:56.726 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mds[139204]: -- 172.21.15.159:0/2470491758 <== mon.0 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560c84fb91e0 con 0x560c84f8ec00 2023-09-12T16:21:56.726 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-b[139180]: starting mds.b at 2023-09-12T16:21:56.727 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-h[138839]: ignoring --setuser ceph since I am not root 2023-09-12T16:21:56.727 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-h[138839]: ignoring --setgroup ceph since I am not root 2023-09-12T16:21:56.727 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-h[138839]: starting mds.h at 2023-09-12T16:21:56.728 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-e[138498]: ignoring --setuser ceph since I am not root 2023-09-12T16:21:56.728 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-e[138498]: ignoring --setgroup ceph since I am not root 2023-09-12T16:21:56.728 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mds[138523]: -- 172.21.15.159:0/4122938140 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560c224431e0 con 0x560c22418c00 2023-09-12T16:21:56.728 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-e[138498]: starting mds.e at 2023-09-12T16:21:56.745 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-c[140578]: ignoring --setuser ceph since I am not root 2023-09-12T16:21:56.746 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-c[140578]: ignoring --setgroup ceph since I am not root 2023-09-12T16:21:56.746 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mds[140602]: -- 172.21.15.203:0/2968758729 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5606a32b31e0 con 0x5606a3289000 2023-09-12T16:21:56.746 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-c[140578]: starting mds.c at 2023-09-12T16:21:56.747 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-f[140235]: ignoring --setuser ceph since I am not root 2023-09-12T16:21:56.747 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-f[140235]: ignoring --setgroup ceph since I am not root 2023-09-12T16:21:56.747 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mds[140259]: -- 172.21.15.203:0/4239264603 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557d2b22f1e0 con 0x557d2b205000 2023-09-12T16:21:56.747 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-f[140235]: starting mds.f at 2023-09-12T16:21:56.748 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-i[140918]: ignoring --setuser ceph since I am not root 2023-09-12T16:21:56.748 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-i[140918]: ignoring --setgroup ceph since I am not root 2023-09-12T16:21:56.748 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mds[140942]: -- 172.21.15.203:0/444569321 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a9389831e0 con 0x55a938958c00 2023-09-12T16:21:56.748 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-i[140918]: starting mds.i at 2023-09-12T16:21:57.180 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.179+0000 7f00c6ffd700 1 -- 172.21.15.130:0/3124522452 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f00d0099a00 con 0x7f00d41006e0 2023-09-12T16:21:57.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00c00a0640 msgr2=0x7f00c00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:57.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00c00a0640 0x7f00c00a2b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f00d4100540 tx=0x7f00c8009260 comp rx=0 tx=0).stop 2023-09-12T16:21:57.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41006e0 msgr2=0x7f00d41956e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:57.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41006e0 0x7f00d41956e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009fd0 tx=0x7f00d0009580 comp rx=0 tx=0).stop 2023-09-12T16:21:57.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 shutdown_connections 2023-09-12T16:21:57.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f00c00a0640 0x7f00c00a2b00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:57.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00d4101080 0x7f00d419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:57.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00d40ff4f0 0x7f00d41951a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:57.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 --2- 172.21.15.130:0/3124522452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41006e0 0x7f00d41956e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:57.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 >> 172.21.15.130:0/3124522452 conn(0x7f00d40faa60 msgr2=0x7f00d40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:57.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 shutdown_connections 2023-09-12T16:21:57.185 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.182+0000 7f00dbf76700 1 -- 172.21.15.130:0/3124522452 wait complete. 2023-09-12T16:21:57.195 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3030682434' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-12T16:21:57.195 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-mon[98670]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-12T16:21:57.195 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-mon[98670]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-12T16:21:57.195 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-mon[98670]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-12T16:21:57.196 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-mon[98670]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-12T16:21:57.196 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-mon[98670]: osdmap e91: 12 total, 12 up, 12 in 2023-09-12T16:21:57.196 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-mon[98670]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-12T16:21:57.196 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/3124522452' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-12T16:21:57.196 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:56 smithi130 ceph-mon[98670]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-12T16:21:57.197 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-12T16:21:57.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3030682434' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-12T16:21:57.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mon[102522]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-12T16:21:57.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mon[102522]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-12T16:21:57.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mon[102522]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-12T16:21:57.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mon[102522]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-12T16:21:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mon[102522]: osdmap e91: 12 total, 12 up, 12 in 2023-09-12T16:21:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mon[102522]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-12T16:21:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/3124522452' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-12T16:21:57.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:56 smithi159 ceph-mon[102522]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-12T16:21:57.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3030682434' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-12T16:21:57.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mon[103431]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-12T16:21:57.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mon[103431]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-12T16:21:57.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mon[103431]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-12T16:21:57.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mon[103431]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-12T16:21:57.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mon[103431]: osdmap e91: 12 total, 12 up, 12 in 2023-09-12T16:21:57.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mon[103431]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-12T16:21:57.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/3124522452' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-12T16:21:57.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:56 smithi203 ceph-mon[103431]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-12T16:21:57.327 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 -- 172.21.15.130:0/804148439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d400ff520 msgr2=0x7f2d400ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:57.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/804148439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d400ff520 0x7f2d400ff940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009920 tx=0x7f2d30009c30 comp rx=0 tx=0).stop 2023-09-12T16:21:57.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 -- 172.21.15.130:0/804148439 shutdown_connections 2023-09-12T16:21:57.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/804148439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d401010b0 0x7f2d40107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:57.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/804148439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d400ff520 0x7f2d400ff940 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:57.328 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/804148439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d40100710 0x7f2d40100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:57.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 -- 172.21.15.130:0/804148439 >> 172.21.15.130:0/804148439 conn(0x7f2d400faa90 msgr2=0x7f2d400fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:57.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 -- 172.21.15.130:0/804148439 shutdown_connections 2023-09-12T16:21:57.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 -- 172.21.15.130:0/804148439 wait complete. 2023-09-12T16:21:57.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 Processor -- start 2023-09-12T16:21:57.329 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.325+0000 7f2d45ddb700 1 -- start start 2023-09-12T16:21:57.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d45ddb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d400ff520 0x7f2d400692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:57.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d45ddb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d40100710 0x7f2d4019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:57.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d45ddb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d401010b0 0x7f2d4019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:57.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d45ddb700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4019ded0 con 0x7f2d401010b0 2023-09-12T16:21:57.330 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d45ddb700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4019e040 con 0x7f2d400ff520 2023-09-12T16:21:57.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d45ddb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4019e200 con 0x7f2d40100710 2023-09-12T16:21:57.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d401010b0 0x7f2d4019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:57.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d401010b0 0x7f2d4019d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57102/0 (socket says 172.21.15.130:57102) 2023-09-12T16:21:57.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3ffff700 1 -- 172.21.15.130:0/1213760792 learned_addr learned my addr 172.21.15.130:0/1213760792 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:57.331 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3f7fe700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d400ff520 0x7f2d400692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:57.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3effd700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d40100710 0x7f2d4019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:57.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3ffff700 1 -- 172.21.15.130:0/1213760792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d40100710 msgr2=0x7f2d4019b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:57.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3ffff700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d40100710 0x7f2d4019b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:57.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3ffff700 1 -- 172.21.15.130:0/1213760792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d400ff520 msgr2=0x7f2d400692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:57.332 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3ffff700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d400ff520 0x7f2d400692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:57.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3ffff700 1 -- 172.21.15.130:0/1213760792 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d38009d00 con 0x7f2d401010b0 2023-09-12T16:21:57.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3ffff700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d401010b0 0x7f2d4019d730 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e3c0 tx=0x7f2d3800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:57.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3cff9700 1 -- 172.21.15.130:0/1213760792 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d380090d0 con 0x7f2d401010b0 2023-09-12T16:21:57.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d30009520 con 0x7f2d401010b0 2023-09-12T16:21:57.333 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.326+0000 7f2d3cff9700 1 -- 172.21.15.130:0/1213760792 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d3800f040 con 0x7f2d401010b0 2023-09-12T16:21:57.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.327+0000 7f2d3cff9700 1 -- 172.21.15.130:0/1213760792 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38007c20 con 0x7f2d401010b0 2023-09-12T16:21:57.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.327+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4019e860 con 0x7f2d401010b0 2023-09-12T16:21:57.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.327+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d40106500 con 0x7f2d401010b0 2023-09-12T16:21:57.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.329+0000 7f2d3cff9700 1 -- 172.21.15.130:0/1213760792 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f2d38004ad0 con 0x7f2d401010b0 2023-09-12T16:21:57.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.329+0000 7f2d3cff9700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d280a0640 0x7f2d280a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:57.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.329+0000 7f2d3cff9700 1 -- 172.21.15.130:0/1213760792 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2d380cc040 con 0x7f2d401010b0 2023-09-12T16:21:57.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.329+0000 7f2d3f7fe700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d280a0640 0x7f2d280a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:57.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.330+0000 7f2d3f7fe700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d280a0640 0x7f2d280a2b00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2d300098f0 tx=0x7f2d30019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:57.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.333+0000 7f2d3cff9700 1 -- 172.21.15.130:0/1213760792 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2d38095790 con 0x7f2d401010b0 2023-09-12T16:21:57.532 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:57.532+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d4010ad30 con 0x7f2d401010b0 2023-09-12T16:21:58.179 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.178+0000 7f2d3cff9700 1 -- 172.21.15.130:0/1213760792 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2d38098db0 con 0x7f2d401010b0 2023-09-12T16:21:58.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.180+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d280a0640 msgr2=0x7f2d280a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:58.181 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.180+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d280a0640 0x7f2d280a2b00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2d300098f0 tx=0x7f2d30019040 comp rx=0 tx=0).stop 2023-09-12T16:21:58.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d401010b0 msgr2=0x7f2d4019d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:58.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d401010b0 0x7f2d4019d730 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e3c0 tx=0x7f2d3800e6d0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 shutdown_connections 2023-09-12T16:21:58.182 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f2d280a0640 0x7f2d280a2b00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d401010b0 0x7f2d4019d730 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d400ff520 0x7f2d400692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.183 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 --2- 172.21.15.130:0/1213760792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d40100710 0x7f2d4019b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 >> 172.21.15.130:0/1213760792 conn(0x7f2d400faa90 msgr2=0x7f2d400fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:58.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 shutdown_connections 2023-09-12T16:21:58.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.181+0000 7f2d45ddb700 1 -- 172.21.15.130:0/1213760792 wait complete. 2023-09-12T16:21:58.185 INFO:teuthology.orchestra.run.smithi130.stderr:pool 'cephfs_metadata' removed 2023-09-12T16:21:58.196 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-12T16:21:58.196 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-12T16:21:58.197 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: Cluster is now healthy 2023-09-12T16:21:58.197 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-12T16:21:58.197 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.203:6837/1312719175,v1:172.21.15.203:6839/1312719175] up:boot 2023-09-12T16:21:58.197 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.203:6834/1731514884,v1:172.21.15.203:6835/1731514884] up:boot 2023-09-12T16:21:58.198 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.203:6836/413323462,v1:172.21.15.203:6838/413323462] up:boot 2023-09-12T16:21:58.198 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.159:6836/3382728819,v1:172.21.15.159:6839/3382728819] up:boot 2023-09-12T16:21:58.198 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.159:6834/2641024032,v1:172.21.15.159:6837/2641024032] up:boot 2023-09-12T16:21:58.199 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: mds.? [v2:172.21.15.159:6835/134429907,v1:172.21.15.159:6838/134429907] up:boot 2023-09-12T16:21:58.199 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: fsmap 12 up:standby 2023-09-12T16:21:58.199 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-12T16:21:58.199 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-12T16:21:58.200 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-12T16:21:58.200 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-12T16:21:58.200 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-12T16:21:58.201 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-12T16:21:58.201 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1213760792' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-12T16:21:58.201 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:57 smithi130 ceph-mon[98670]: pgmap v864: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-12T16:21:58.203 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-12T16:21:58.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-12T16:21:58.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-12T16:21:58.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: Cluster is now healthy 2023-09-12T16:21:58.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-12T16:21:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.203:6837/1312719175,v1:172.21.15.203:6839/1312719175] up:boot 2023-09-12T16:21:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.203:6834/1731514884,v1:172.21.15.203:6835/1731514884] up:boot 2023-09-12T16:21:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.203:6836/413323462,v1:172.21.15.203:6838/413323462] up:boot 2023-09-12T16:21:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.159:6836/3382728819,v1:172.21.15.159:6839/3382728819] up:boot 2023-09-12T16:21:58.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.159:6834/2641024032,v1:172.21.15.159:6837/2641024032] up:boot 2023-09-12T16:21:58.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: mds.? [v2:172.21.15.159:6835/134429907,v1:172.21.15.159:6838/134429907] up:boot 2023-09-12T16:21:58.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: fsmap 12 up:standby 2023-09-12T16:21:58.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-12T16:21:58.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-12T16:21:58.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-12T16:21:58.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-12T16:21:58.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-12T16:21:58.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-12T16:21:58.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1213760792' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-12T16:21:58.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:57 smithi159 ceph-mon[102522]: pgmap v864: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-12T16:21:58.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-12T16:21:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-12T16:21:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: Cluster is now healthy 2023-09-12T16:21:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-12T16:21:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.203:6837/1312719175,v1:172.21.15.203:6839/1312719175] up:boot 2023-09-12T16:21:58.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.203:6834/1731514884,v1:172.21.15.203:6835/1731514884] up:boot 2023-09-12T16:21:58.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.203:6836/413323462,v1:172.21.15.203:6838/413323462] up:boot 2023-09-12T16:21:58.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.159:6836/3382728819,v1:172.21.15.159:6839/3382728819] up:boot 2023-09-12T16:21:58.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.159:6834/2641024032,v1:172.21.15.159:6837/2641024032] up:boot 2023-09-12T16:21:58.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: mds.? [v2:172.21.15.159:6835/134429907,v1:172.21.15.159:6838/134429907] up:boot 2023-09-12T16:21:58.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: fsmap 12 up:standby 2023-09-12T16:21:58.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-12T16:21:58.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-12T16:21:58.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-12T16:21:58.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-12T16:21:58.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-12T16:21:58.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: from='mgr.14150 172.21.15.130:0/789637330' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-12T16:21:58.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1213760792' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-12T16:21:58.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:57 smithi203 ceph-mon[103431]: pgmap v864: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-12T16:21:58.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 -- 172.21.15.130:0/1663000320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa4100610 msgr2=0x7f7aa4100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:58.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 --2- 172.21.15.130:0/1663000320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa4100610 0x7f7aa4100a30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7a940099c0 tx=0x7f7a94009cd0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 -- 172.21.15.130:0/1663000320 shutdown_connections 2023-09-12T16:21:58.334 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 --2- 172.21.15.130:0/1663000320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa41021a0 0x7f7aa4108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 --2- 172.21.15.130:0/1663000320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa4100610 0x7f7aa4100a30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 --2- 172.21.15.130:0/1663000320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7aa4101800 0x7f7aa4101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 -- 172.21.15.130:0/1663000320 >> 172.21.15.130:0/1663000320 conn(0x7f7aa40fbb80 msgr2=0x7f7aa40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:58.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 -- 172.21.15.130:0/1663000320 shutdown_connections 2023-09-12T16:21:58.335 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 -- 172.21.15.130:0/1663000320 wait complete. 2023-09-12T16:21:58.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 Processor -- start 2023-09-12T16:21:58.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.331+0000 7f7aac628700 1 -- start start 2023-09-12T16:21:58.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aac628700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7aa4100610 0x7f7aa4196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:58.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aac628700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa4101800 0x7f7aa41968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:58.336 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aac628700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa41021a0 0x7f7aa419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:58.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aac628700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa419df00 con 0x7f7aa41021a0 2023-09-12T16:21:58.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aac628700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa419e040 con 0x7f7aa4101800 2023-09-12T16:21:58.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aac628700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa419e1d0 con 0x7f7aa4100610 2023-09-12T16:21:58.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaabc5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa41021a0 0x7f7aa419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:58.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaabc5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa41021a0 0x7f7aa419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57114/0 (socket says 172.21.15.130:57114) 2023-09-12T16:21:58.337 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaabc5700 1 -- 172.21.15.130:0/2298691044 learned_addr learned my addr 172.21.15.130:0/2298691044 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:58.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aa9bc3700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa4101800 0x7f7aa41968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:58.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaa3c4700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7aa4100610 0x7f7aa4196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:58.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaabc5700 1 -- 172.21.15.130:0/2298691044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7aa4100610 msgr2=0x7f7aa4196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:58.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaabc5700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7aa4100610 0x7f7aa4196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.338 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaabc5700 1 -- 172.21.15.130:0/2298691044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa4101800 msgr2=0x7f7aa41968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:58.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaabc5700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa4101800 0x7f7aa41968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:58.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaabc5700 1 -- 172.21.15.130:0/2298691044 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a940095f0 con 0x7f7aa41021a0 2023-09-12T16:21:58.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aaabc5700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa41021a0 0x7f7aa419d930 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00ef30 tx=0x7f7a9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:58.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7a9b7fe700 1 -- 172.21.15.130:0/2298691044 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c018070 con 0x7f7aa41021a0 2023-09-12T16:21:58.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7a9b7fe700 1 -- 172.21.15.130:0/2298691044 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c00f040 con 0x7f7aa41021a0 2023-09-12T16:21:58.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa419e450 con 0x7f7aa41021a0 2023-09-12T16:21:58.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.332+0000 7f7a9b7fe700 1 -- 172.21.15.130:0/2298691044 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00cd00 con 0x7f7aa41021a0 2023-09-12T16:21:58.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.333+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa419e960 con 0x7f7aa41021a0 2023-09-12T16:21:58.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.333+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa4107590 con 0x7f7aa41021a0 2023-09-12T16:21:58.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.334+0000 7f7a9b7fe700 1 -- 172.21.15.130:0/2298691044 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7f7a9c003a00 con 0x7f7aa41021a0 2023-09-12T16:21:58.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.335+0000 7f7a9b7fe700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a900a06c0 0x7f7a900a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:58.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.335+0000 7f7aaa3c4700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a900a06c0 0x7f7a900a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:58.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.335+0000 7f7a9b7fe700 1 -- 172.21.15.130:0/2298691044 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8782+0+0 (secure 0 0 0) 0x7f7a9c0cc240 con 0x7f7aa41021a0 2023-09-12T16:21:58.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.335+0000 7f7aaa3c4700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a900a06c0 0x7f7a900a2b80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009990 tx=0x7f7a94005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:58.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.338+0000 7f7a9b7fe700 1 -- 172.21.15.130:0/2298691044 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7a9c095be0 con 0x7f7aa41021a0 2023-09-12T16:21:58.537 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:58.536+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7aa410be40 con 0x7f7aa41021a0 2023-09-12T16:21:59.184 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.184+0000 7f7a9b7fe700 1 -- 172.21.15.130:0/2298691044 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7a9c099200 con 0x7f7aa41021a0 2023-09-12T16:21:59.188 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a900a06c0 msgr2=0x7f7a900a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:59.189 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a900a06c0 0x7f7a900a2b80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009990 tx=0x7f7a94005fb0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.189 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa41021a0 msgr2=0x7f7aa419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:59.189 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa41021a0 0x7f7aa419d930 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00ef30 tx=0x7f7a9c00c3a0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 shutdown_connections 2023-09-12T16:21:59.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7f7a900a06c0 0x7f7a900a2b80 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009990 tx=0x7f7a94005fb0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7aa41021a0 0x7f7aa419d930 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.190 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7aa4100610 0x7f7aa4196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 --2- 172.21.15.130:0/2298691044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa4101800 0x7f7aa41968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.188+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 >> 172.21.15.130:0/2298691044 conn(0x7f7aa40fbb80 msgr2=0x7f7aa40fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:59.191 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.190+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 shutdown_connections 2023-09-12T16:21:59.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.190+0000 7f7aac628700 1 -- 172.21.15.130:0/2298691044 wait complete. 2023-09-12T16:21:59.192 INFO:teuthology.orchestra.run.smithi130.stderr:pool 'cephfs_data' removed 2023-09-12T16:21:59.207 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-09-12T16:21:59.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.336+0000 7fe522b7b700 1 -- 172.21.15.130:0/4207097489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c100710 msgr2=0x7fe51c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:59.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.336+0000 7fe522b7b700 1 --2- 172.21.15.130:0/4207097489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c100710 0x7fe51c100b90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe508009990 tx=0x7fe508009ca0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.339 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.336+0000 7fe522b7b700 1 -- 172.21.15.130:0/4207097489 shutdown_connections 2023-09-12T16:21:59.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.336+0000 7fe522b7b700 1 --2- 172.21.15.130:0/4207097489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c1010d0 0x7fe51c107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.336+0000 7fe522b7b700 1 --2- 172.21.15.130:0/4207097489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c0ffa00 0x7fe51c0ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.336+0000 7fe522b7b700 1 --2- 172.21.15.130:0/4207097489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c100710 0x7fe51c100b90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 -- 172.21.15.130:0/4207097489 >> 172.21.15.130:0/4207097489 conn(0x7fe51c0fa1d0 msgr2=0x7fe51c0fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:21:59.340 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 -- 172.21.15.130:0/4207097489 shutdown_connections 2023-09-12T16:21:59.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 -- 172.21.15.130:0/4207097489 wait complete. 2023-09-12T16:21:59.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 Processor -- start 2023-09-12T16:21:59.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 -- start start 2023-09-12T16:21:59.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c0ffa00 0x7fe51c195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:59.341 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c100710 0x7fe51c1957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:59.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c1010d0 0x7fe51c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:59.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19ce10 con 0x7fe51c0ffa00 2023-09-12T16:21:59.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19cf50 con 0x7fe51c100710 2023-09-12T16:21:59.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe522b7b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19d0e0 con 0x7fe51c1010d0 2023-09-12T16:21:59.342 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe520917700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c0ffa00 0x7fe51c195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:59.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe520917700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c0ffa00 0x7fe51c195290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57124/0 (socket says 172.21.15.130:57124) 2023-09-12T16:21:59.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.337+0000 7fe520917700 1 -- 172.21.15.130:0/891086094 learned_addr learned my addr 172.21.15.130:0/891086094 (peer_addr_for_me v2:172.21.15.130:0/0) 2023-09-12T16:21:59.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe51bfff700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c100710 0x7fe51c1957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:59.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe520917700 1 -- 172.21.15.130:0/891086094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c1010d0 msgr2=0x7fe51c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:59.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe520917700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c1010d0 0x7fe51c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.343 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe520917700 1 -- 172.21.15.130:0/891086094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c100710 msgr2=0x7fe51c1957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:21:59.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe520917700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c100710 0x7fe51c1957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:21:59.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe520917700 1 -- 172.21.15.130:0/891086094 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080095f0 con 0x7fe51c0ffa00 2023-09-12T16:21:59.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe520917700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c0ffa00 0x7fe51c195290 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe51000d980 tx=0x7fe51000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:59.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe519ffb700 1 -- 172.21.15.130:0/891086094 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000de60 con 0x7fe51c0ffa00 2023-09-12T16:21:59.344 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe519ffb700 1 -- 172.21.15.130:0/891086094 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5100044e0 con 0x7fe51c0ffa00 2023-09-12T16:21:59.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c19d3c0 con 0x7fe51c0ffa00 2023-09-12T16:21:59.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe519ffb700 1 -- 172.21.15.130:0/891086094 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510003be0 con 0x7fe51c0ffa00 2023-09-12T16:21:59.345 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.338+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c19d920 con 0x7fe51c0ffa00 2023-09-12T16:21:59.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.339+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c10ad20 con 0x7fe51c0ffa00 2023-09-12T16:21:59.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.340+0000 7fe519ffb700 1 -- 172.21.15.130:0/891086094 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 135260+0+0 (secure 0 0 0) 0x7fe510009bb0 con 0x7fe51c0ffa00 2023-09-12T16:21:59.346 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.341+0000 7fe519ffb700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe50c0a0740 0x7fe50c0a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T16:21:59.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.341+0000 7fe51bfff700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe50c0a0740 0x7fe50c0a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T16:21:59.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.341+0000 7fe519ffb700 1 -- 172.21.15.130:0/891086094 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8252+0+0 (secure 0 0 0) 0x7fe5100cb900 con 0x7fe51c0ffa00 2023-09-12T16:21:59.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.341+0000 7fe51bfff700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe50c0a0740 0x7fe50c0a2c00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe5080095c0 tx=0x7fe508009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T16:21:59.347 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.345+0000 7fe519ffb700 1 -- 172.21.15.130:0/891086094 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe5100953c0 con 0x7fe51c0ffa00 2023-09-12T16:21:59.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:59 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/1213760792' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-12T16:21:59.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:59 smithi130 ceph-mon[98670]: osdmap e92: 12 total, 12 up, 12 in 2023-09-12T16:21:59.475 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:21:59 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2298691044' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-12T16:21:59.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:59 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/1213760792' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-12T16:21:59.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:59 smithi159 ceph-mon[102522]: osdmap e92: 12 total, 12 up, 12 in 2023-09-12T16:21:59.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:21:59 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2298691044' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-12T16:21:59.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:59 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/1213760792' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-12T16:21:59.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:59 smithi203 ceph-mon[103431]: osdmap e92: 12 total, 12 up, 12 in 2023-09-12T16:21:59.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:21:59 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2298691044' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-12T16:21:59.544 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:21:59.543+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe51c1064d0 con 0x7fe51c0ffa00 2023-09-12T16:22:00.192 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.191+0000 7fe519ffb700 1 -- 172.21.15.130:0/891086094 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v94) v1 ==== 178+0+0 (secure 0 0 0) 0x7fe510009940 con 0x7fe51c0ffa00 2023-09-12T16:22:00.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe50c0a0740 msgr2=0x7fe50c0a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:22:00.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe50c0a0740 0x7fe50c0a2c00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe5080095c0 tx=0x7fe508009550 comp rx=0 tx=0).stop 2023-09-12T16:22:00.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c0ffa00 msgr2=0x7fe51c195290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T16:22:00.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c0ffa00 0x7fe51c195290 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe51000d980 tx=0x7fe51000dd40 comp rx=0 tx=0).stop 2023-09-12T16:22:00.195 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 shutdown_connections 2023-09-12T16:22:00.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:6800/4273925772,v1:172.21.15.130:6801/4273925772] conn(0x7fe50c0a0740 0x7fe50c0a2c00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:22:00.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c1010d0 0x7fe51c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:22:00.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe51c0ffa00 0x7fe51c195290 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:22:00.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 --2- 172.21.15.130:0/891086094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c100710 0x7fe51c1957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T16:22:00.196 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 >> 172.21.15.130:0/891086094 conn(0x7fe51c0fa1d0 msgr2=0x7fe51c0fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T16:22:00.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 shutdown_connections 2023-09-12T16:22:00.197 INFO:teuthology.orchestra.run.smithi130.stderr:2023-09-12T16:22:00.194+0000 7fe522b7b700 1 -- 172.21.15.130:0/891086094 wait complete. 2023-09-12T16:22:00.197 INFO:teuthology.orchestra.run.smithi130.stderr:pool 'cephfs_data_ec' removed 2023-09-12T16:22:00.216 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-12T16:22:00.269 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-12T16:22:00.309 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-12T16:22:00.353 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-09-12T16:22:00.354 DEBUG:teuthology.orchestra.run.smithi130:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-12T16:22:00.384 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-12T16:22:00.405 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-12T16:22:00.423 INFO:tasks.cephadm:Stopping all daemons... 2023-09-12T16:22:00.424 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-09-12T16:22:00.424 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.a 2023-09-12T16:22:00.451 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:22:00 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/2298691044' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-12T16:22:00.451 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:22:00 smithi130 ceph-mon[98670]: osdmap e93: 12 total, 12 up, 12 in 2023-09-12T16:22:00.452 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:22:00 smithi130 ceph-mon[98670]: from='client.? 172.21.15.130:0/891086094' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-12T16:22:00.452 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:22:00 smithi130 ceph-mon[98670]: pgmap v867: 33 pgs: 33 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:22:00.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:22:00 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/2298691044' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-12T16:22:00.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:22:00 smithi159 ceph-mon[102522]: osdmap e93: 12 total, 12 up, 12 in 2023-09-12T16:22:00.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:22:00 smithi159 ceph-mon[102522]: from='client.? 172.21.15.130:0/891086094' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-12T16:22:00.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:22:00 smithi159 ceph-mon[102522]: pgmap v867: 33 pgs: 33 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:22:00.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:22:00 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/2298691044' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-12T16:22:00.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:22:00 smithi203 ceph-mon[103431]: osdmap e93: 12 total, 12 up, 12 in 2023-09-12T16:22:00.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:22:00 smithi203 ceph-mon[103431]: from='client.? 172.21.15.130:0/891086094' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-12T16:22:00.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:22:00 smithi203 ceph-mon[103431]: pgmap v867: 33 pgs: 33 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-12T16:22:00.725 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:22:00 smithi130 systemd[1]: Stopping Ceph mon.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:22:00 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-a[98646]: 2023-09-12T16:22:00.843+0000 7f5ebd8e2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:22:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:22:00 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-a[98646]: 2023-09-12T16:22:00.843+0000 7f5ebd8e2700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-09-12T16:22:01.226 INFO:journalctl@ceph.mon.a.smithi130.stdout:Sep 12 16:22:01 smithi130 bash[179111]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-a 2023-09-12T16:22:01.633 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.a.service' 2023-09-12T16:22:01.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:01.799 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-09-12T16:22:01.799 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-09-12T16:22:01.800 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.b 2023-09-12T16:22:02.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:22:01 smithi159 systemd[1]: Stopping Ceph mon.b for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:02.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:22:02 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-b[102498]: 2023-09-12T16:22:02.472+0000 7f9354960700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:22:02.726 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:22:02 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-b[102498]: 2023-09-12T16:22:02.472+0000 7f9354960700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-09-12T16:22:03.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 12 16:22:02 smithi159 bash[145514]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-b 2023-09-12T16:22:03.254 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.b.service' 2023-09-12T16:22:03.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:03.391 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-09-12T16:22:03.392 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-09-12T16:22:03.392 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.c 2023-09-12T16:22:03.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:22:03 smithi203 systemd[1]: Stopping Ceph mon.c for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:04.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:22:03 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-c[103407]: 2023-09-12T16:22:03.987+0000 7f883d97c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:22:04.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:22:03 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-c[103407]: 2023-09-12T16:22:03.987+0000 7f883d97c700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-09-12T16:22:05.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 12 16:22:04 smithi203 bash[147138]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mon-c 2023-09-12T16:22:06.827 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mon.c.service' 2023-09-12T16:22:06.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:06.948 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-09-12T16:22:06.949 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-09-12T16:22:06.949 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.x 2023-09-12T16:22:07.475 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 16:22:07 smithi130 systemd[1]: Stopping Ceph mgr.x for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:08.725 INFO:journalctl@ceph.mgr.x.smithi130.stdout:Sep 12 16:22:08 smithi130 bash[179252]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-x 2023-09-12T16:22:08.889 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.x.service' 2023-09-12T16:22:08.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:08.956 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-09-12T16:22:08.957 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-09-12T16:22:08.957 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.y 2023-09-12T16:22:09.376 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 16:22:08 smithi159 systemd[1]: Stopping Ceph mgr.y for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:09.655 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 16:22:09 smithi159 bash[145658]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-y 2023-09-12T16:22:09.834 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 12 16:22:09 smithi159 systemd[1]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-09-12T16:22:09.838 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.y.service' 2023-09-12T16:22:09.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:09.877 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-09-12T16:22:09.877 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-09-12T16:22:09.877 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.z 2023-09-12T16:22:10.246 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 16:22:09 smithi203 systemd[1]: Stopping Ceph mgr.z for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:13.568 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 16:22:13 smithi203 bash[147288]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mgr-z 2023-09-12T16:22:13.568 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 12 16:22:13 smithi203 systemd[1]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-09-12T16:22:13.575 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mgr.z.service' 2023-09-12T16:22:13.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:13.612 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-09-12T16:22:13.612 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-09-12T16:22:13.612 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.0 2023-09-12T16:22:13.975 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 16:22:13 smithi130 systemd[1]: Stopping Ceph osd.0 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:14.725 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 16:22:14 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-0[121577]: 2023-09-12T16:22:14.297+0000 7f6338835700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:22:14.725 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 16:22:14 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-0[121577]: 2023-09-12T16:22:14.297+0000 7f6338835700 -1 osd.0 94 *** Got signal Terminated *** 2023-09-12T16:22:14.725 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 16:22:14 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-0[121577]: 2023-09-12T16:22:14.297+0000 7f6338835700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:22:21.976 INFO:journalctl@ceph.osd.0.smithi130.stdout:Sep 12 16:22:21 smithi130 bash[179401]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-0 2023-09-12T16:22:26.653 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.0.service' 2023-09-12T16:22:26.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:26.696 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-09-12T16:22:26.697 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-09-12T16:22:26.697 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.1 2023-09-12T16:22:27.068 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 16:22:26 smithi159 systemd[1]: Stopping Ceph osd.1 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:27.476 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 16:22:27 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-1[107552]: 2023-09-12T16:22:27.066+0000 7fba05bb7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:22:27.476 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 16:22:27 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-1[107552]: 2023-09-12T16:22:27.066+0000 7fba05bb7700 -1 osd.1 94 *** Got signal Terminated *** 2023-09-12T16:22:27.476 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 16:22:27 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-1[107552]: 2023-09-12T16:22:27.066+0000 7fba05bb7700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:22:33.225 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 12 16:22:32 smithi159 bash[145805]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-1 2023-09-12T16:22:34.691 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.1.service' 2023-09-12T16:22:34.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:34.734 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-09-12T16:22:34.735 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-09-12T16:22:34.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.2 2023-09-12T16:22:35.106 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 16:22:34 smithi203 systemd[1]: Stopping Ceph osd.2 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:35.495 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 16:22:35 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-2[109286]: 2023-09-12T16:22:35.105+0000 7facd0bb2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:22:35.496 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 16:22:35 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-2[109286]: 2023-09-12T16:22:35.105+0000 7facd0bb2700 -1 osd.2 94 *** Got signal Terminated *** 2023-09-12T16:22:35.496 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 16:22:35 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-2[109286]: 2023-09-12T16:22:35.105+0000 7facd0bb2700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:22:41.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:40 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:40.865+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:41.245 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 12 16:22:40 smithi203 bash[147430]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-2 2023-09-12T16:22:41.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:41 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:41.414+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:42.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:41 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:41.879+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:42.639 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.2.service' 2023-09-12T16:22:42.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:42.682 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-09-12T16:22:42.682 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-09-12T16:22:42.683 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.3 2023-09-12T16:22:42.713 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:42 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:42.461+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:42.975 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:42 smithi130 systemd[1]: Stopping Ceph osd.3 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:43.225 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:42 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:42.923+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:43.456 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:43 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:43.030+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:43.457 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:43 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T16:22:43.043+0000 7f9fd909a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:22:43.457 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:43 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T16:22:43.043+0000 7f9fd909a700 -1 osd.3 94 *** Got signal Terminated *** 2023-09-12T16:22:43.457 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:43 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T16:22:43.043+0000 7f9fd909a700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:22:43.457 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:43 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T16:22:43.130+0000 7f9fd1688700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.900368+0000 front 2023-09-12T16:22:16.900416+0000 (oldest deadline 2023-09-12T16:22:42.200587+0000) 2023-09-12T16:22:43.458 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:43 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:43.456+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:43.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:43 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:43.547+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:43.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:43 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:43.790+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:44.225 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:43 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:43.892+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:44.446 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:44 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:44.036+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:44.446 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:44 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T16:22:44.094+0000 7f9fd1688700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.900368+0000 front 2023-09-12T16:22:16.900416+0000 (oldest deadline 2023-09-12T16:22:42.200587+0000) 2023-09-12T16:22:44.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:44 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:44.446+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:44.789 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:44 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:44.497+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:45.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:44 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:44.889+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:45.245 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:44 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:44.786+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:45.475 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:45 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T16:22:45.101+0000 7f9fd1688700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.900368+0000 front 2023-09-12T16:22:16.900416+0000 (oldest deadline 2023-09-12T16:22:42.200587+0000) 2023-09-12T16:22:45.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:45 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:45.071+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:45.975 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:45 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:45.482+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:45.995 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:45.541+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:45.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:45.739+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:46.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:45 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:45.866+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:46.463 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:46 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:46.088+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:46.463 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:46 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T16:22:46.076+0000 7f9fd1688700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.900368+0000 front 2023-09-12T16:22:16.900416+0000 (oldest deadline 2023-09-12T16:22:42.200587+0000) 2023-09-12T16:22:46.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:46 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:46.463+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:46.995 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:46 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:46.559+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:46.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:46 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:46.749+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:47.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:46 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:46.915+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:47.441 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:47 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:47.123+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:47.442 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:47 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3[129670]: 2023-09-12T16:22:47.065+0000 7f9fd1688700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.900368+0000 front 2023-09-12T16:22:16.900416+0000 (oldest deadline 2023-09-12T16:22:42.200587+0000) 2023-09-12T16:22:47.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:47 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:47.442+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:47.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:47 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:47.571+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:47.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:47 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:47.716+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:48.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:47 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:47.938+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:48.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:48 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:48.163+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:48.840 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:48 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:48.479+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:48.995 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:48.525+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:48.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:48.695+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:49.122 INFO:journalctl@ceph.osd.3.smithi130.stdout:Sep 12 16:22:49 smithi130 bash[179684]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-3 2023-09-12T16:22:49.432 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:49 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:49.182+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:49.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:48 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:48.986+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:49.705 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:49 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:49.433+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:49.995 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:49.524+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:49.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:49.689+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:50.380 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:50 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:50.134+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:50.387 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.3.service' 2023-09-12T16:22:50.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:50.469 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-09-12T16:22:50.469 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-09-12T16:22:50.469 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.4 2023-09-12T16:22:50.475 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:49 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:49.981+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:50.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:50 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:50.387+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:50.821 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 16:22:50 smithi159 systemd[1]: Stopping Ceph osd.4 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:50.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:50 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:50.667+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:50.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:50 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:50.570+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:51.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:51 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:51.020+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:51.226 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 16:22:50 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4[115429]: 2023-09-12T16:22:50.821+0000 7ff99c44b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:22:51.226 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 16:22:50 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4[115429]: 2023-09-12T16:22:50.821+0000 7ff99c44b700 -1 osd.4 94 *** Got signal Terminated *** 2023-09-12T16:22:51.227 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 16:22:50 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4[115429]: 2023-09-12T16:22:50.821+0000 7ff99c44b700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:22:51.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:51 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:51.149+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:51.475 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:51 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:51.353+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:51.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:51 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:51.568+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:51.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:51 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:51.697+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:52.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:52 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:52.166+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:52.475 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:52 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:52.312+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:52.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:51 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:51.982+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:52.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:52 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:52.684+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:52.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:52 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:52.547+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:53.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:53 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:53.122+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:53.475 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:53 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:53.327+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:53.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:53 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:53.030+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:53.933 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 16:22:53 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4[115429]: 2023-09-12T16:22:53.660+0000 7ff99523a700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.123627+0000 front 2023-09-12T16:22:28.123662+0000 (oldest deadline 2023-09-12T16:22:52.823736+0000) 2023-09-12T16:22:53.995 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:53 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:53.565+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:53.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:53 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:53.679+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:54.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:54 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:54.044+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:54.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:22:53 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:22:53.933+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:22:54.351 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:54 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:54.091+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:54.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:54 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:54.352+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:54.976 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 16:22:54 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4[115429]: 2023-09-12T16:22:54.693+0000 7ff99523a700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.123627+0000 front 2023-09-12T16:22:28.123662+0000 (oldest deadline 2023-09-12T16:22:52.823736+0000) 2023-09-12T16:22:54.976 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:22:54 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:22:54.932+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:22:54.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:54 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:54.632+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:54.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:54 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:54.579+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:55.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:55 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:55.077+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:55.475 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:55 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:55.310+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:55.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:55 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:55.065+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:55.976 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 16:22:55 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4[115429]: 2023-09-12T16:22:55.668+0000 7ff99523a700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.123627+0000 front 2023-09-12T16:22:28.123662+0000 (oldest deadline 2023-09-12T16:22:52.823736+0000) 2023-09-12T16:22:55.976 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:22:55 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:22:55.891+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:22:55.995 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:55 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:55.612+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:55.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:55 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:55.603+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:55.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:22:55 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:22:55.610+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:22:56.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:56 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:56.074+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:56.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:56 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:56.074+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:22:56.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:56 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:56.281+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:56.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:56.071+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:56.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:22:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:22:56.602+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:22:56.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:56.644+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:56.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:56.612+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:56.997 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:56 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:56.612+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.505882+0000 front 2023-09-12T16:22:30.505823+0000 (oldest deadline 2023-09-12T16:22:56.405874+0000) 2023-09-12T16:22:57.225 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 12 16:22:56 smithi159 bash[146085]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-4 2023-09-12T16:22:57.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:22:56 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:22:56.913+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:22:57.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:57 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:57.059+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:57.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:57 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:57.076+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:57.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:57 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:57.076+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:22:57.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:57 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:57.242+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:57.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:57 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:57.242+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:22:57.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:57 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:57.684+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:57.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:57 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:57.579+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:57.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:57 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:57.579+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.505882+0000 front 2023-09-12T16:22:30.505823+0000 (oldest deadline 2023-09-12T16:22:56.405874+0000) 2023-09-12T16:22:57.997 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:22:57 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:22:57.585+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:22:58.161 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:22:57 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:22:57.926+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:22:58.162 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:58 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:58.050+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:58.167 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.4.service' 2023-09-12T16:22:58.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:22:58.247 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-09-12T16:22:58.248 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-09-12T16:22:58.248 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.5 2023-09-12T16:22:58.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:58 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:58.038+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:58.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:58 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:58.038+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:22:58.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:58 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:58.220+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:58.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:58 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:58.220+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:22:58.623 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:58 smithi203 systemd[1]: Stopping Ceph osd.5 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:22:58.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:22:58 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:22:58.629+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:22:58.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:58 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:58.688+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:58.997 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:58 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:58.622+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:58.997 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:58 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:58.622+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.505882+0000 front 2023-09-12T16:22:30.505823+0000 (oldest deadline 2023-09-12T16:22:56.405874+0000) 2023-09-12T16:22:58.997 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:58 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:58.626+0000 7f2cf5e90700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:22:58.998 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:58 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:58.626+0000 7f2cf5e90700 -1 osd.5 94 *** Got signal Terminated *** 2023-09-12T16:22:58.998 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:58 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:58.626+0000 7f2cf5e90700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:22:59.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:22:58 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:22:58.922+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:22:59.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:22:59 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:22:59.069+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:22:59.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:59 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:59.068+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:22:59.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:22:59 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:22:59.068+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:22:59.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:59 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:59.198+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:22:59.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:22:59 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:22:59.198+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:22:59.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:22:59 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:22:59.650+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:22:59.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:22:59 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:22:59.683+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:22:59.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:59 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:59.651+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:22:59.997 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:22:59 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:22:59.651+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.505882+0000 front 2023-09-12T16:22:30.505823+0000 (oldest deadline 2023-09-12T16:22:56.405874+0000) 2023-09-12T16:23:00.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:22:59 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:22:59.882+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:00.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:00 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:00.052+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:00.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:00 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:00.079+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:00.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:00 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:00.079+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:00.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:00 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:00.191+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:00.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:00 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:00.191+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:00.995 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:00 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:00.688+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:00.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:00 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:00.660+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:00.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:23:00 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:23:00.669+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:23:00.997 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:23:00 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:23:00.669+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.505882+0000 front 2023-09-12T16:22:30.505823+0000 (oldest deadline 2023-09-12T16:22:56.405874+0000) 2023-09-12T16:23:01.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:00 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:00.899+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:01.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:01 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:01.074+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:01.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:01 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:01.083+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:01.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:01 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:01.083+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:01.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:01 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:01.083+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:01.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:01 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:01.209+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:01.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:01 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:01.209+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:01.995 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:23:01 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:23:01.644+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:23:01.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:23:01 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:23:01.644+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.505882+0000 front 2023-09-12T16:22:30.505823+0000 (oldest deadline 2023-09-12T16:22:56.405874+0000) 2023-09-12T16:23:01.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:01 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:01.642+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:01.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:01 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:01.718+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:01.997 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:01 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:01.718+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:02.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:01 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:01.944+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:02.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:02 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:02.028+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:02.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:02 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:02.029+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:02.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:02 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:02.067+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:02.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:02 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:02.067+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:02.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:02 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:02.067+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:02.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:02 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:02.243+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:02.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:02 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:02.243+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:02.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:02 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:02.243+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:02.995 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:23:02 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:23:02.644+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:23:02.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:23:02 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:23:02.644+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.505882+0000 front 2023-09-12T16:22:30.505823+0000 (oldest deadline 2023-09-12T16:22:56.405874+0000) 2023-09-12T16:23:02.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:02 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:02.682+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:02.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:02 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:02.682+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:02.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:02 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:02.691+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:03.225 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:03 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:03.088+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:03.226 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:03 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:03.088+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:03.226 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:03 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:03.088+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:03.227 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:02 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:02.905+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:03.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:03 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:03.035+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:03.228 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:03 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:03.035+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:03.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:03 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:03.281+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:03.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:03 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:03.281+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:03.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:03 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:03.281+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:03.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:03 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:03.693+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:03.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:03 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:03.693+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:03.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:03 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:03.714+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:03.997 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:23:03 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:23:03.619+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.203961+0000 front 2023-09-12T16:22:18.203654+0000 (oldest deadline 2023-09-12T16:22:42.903830+0000) 2023-09-12T16:23:03.997 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:23:03 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5[117799]: 2023-09-12T16:23:03.619+0000 7f2cee47e700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.505882+0000 front 2023-09-12T16:22:30.505823+0000 (oldest deadline 2023-09-12T16:22:56.405874+0000) 2023-09-12T16:23:04.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:03 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:03.932+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:04.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:03 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:03.932+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:04.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:04 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:04.064+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:04.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:04 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:04.064+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:04.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:04 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:04.134+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:04.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:04 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:04.134+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:04.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:04 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:04.134+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:04.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:04 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:04.257+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:04.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:04 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:04.257+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:04.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:04 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:04.257+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:04.996 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 12 16:23:04 smithi203 bash[147701]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-5 2023-09-12T16:23:04.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:04 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:04.686+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:04.997 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:04 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:04.686+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:04.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:04 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:04.701+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:04.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:04 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:04.701+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:05.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:05 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:05.101+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:05.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:05 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:05.101+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:05.229 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:04 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:04.918+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:05.229 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:04 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:04.918+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:05.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:05 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:05.116+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:05.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:05 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:05.116+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:05.476 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:05 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:05.116+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:05.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:05 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:05.294+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:05.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:05 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:05.294+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:05.477 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:05 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:05.294+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:05.946 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:05 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:05.655+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:05.946 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:05 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:05.655+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:05.947 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:05 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:05.702+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:05.947 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:05 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:05.702+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:05.952 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.5.service' 2023-09-12T16:23:05.992 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:23:05.992 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-09-12T16:23:05.993 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-09-12T16:23:05.993 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.6 2023-09-12T16:23:06.225 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:06 smithi130 systemd[1]: Stopping Ceph osd.6 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:23:06.225 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:06 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:06.071+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:06.226 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:06 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:06.071+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:06.226 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:06 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:06.071+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:06.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:05 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:05.892+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:06.227 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:05 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:05.892+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:06.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:06 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:06.085+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:06.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:06 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:06.085+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:06.725 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:06 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:06.372+0000 7fd07fa87700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:23:06.725 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:06 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:06.372+0000 7fd07fa87700 -1 osd.6 94 *** Got signal Terminated *** 2023-09-12T16:23:06.726 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:06 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:06.372+0000 7fd07fa87700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:23:06.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:06 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:06.292+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:06.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:06 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:06.292+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:06.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:06 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:06.292+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:06.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:06 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:06.669+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:06.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:06 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:06.669+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:06.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:06 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:06.690+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:06.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:06 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:06.690+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:07.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:06 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:06.851+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:07.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:06 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:06.851+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:07.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:07 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:07.039+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:07.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:07 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:07.039+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:07.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:07 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:07.061+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:07.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:07 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:07.061+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:07.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:07 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:07.061+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:07.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:07 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:07.262+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:07.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:07 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:07.262+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:07.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:07 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:07.262+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:07.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:07 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:07.654+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:07.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:07 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:07.654+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:07.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:07 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:07.675+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:07.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:07 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:07.675+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:08.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:07 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:07.892+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:08.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:07 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:07.892+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:08.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:08 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:08.047+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:08.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:08 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:08.047+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:08.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:08.110+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:08.476 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:08.110+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:08.476 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:08.110+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:08.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:08.216+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:08.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:08.216+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:08.477 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:08 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:08.216+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:08.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:08.654+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:08.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:08.654+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:08.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:08.647+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:08.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:08 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:08.647+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:09.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:08 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:08.879+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:09.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:08 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:08.879+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:09.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:09 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:09.029+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:09.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:09 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:09.029+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:09.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:09.093+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:09.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:09.093+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:09.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:09.093+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:09.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:09.209+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:09.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:09.209+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:09.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:09 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:09.209+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:09.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:09 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:09.692+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:09.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:09 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:09.692+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:09.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:09 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:09.657+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:09.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:09 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:09.657+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:10.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:10.053+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:10.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:10.053+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:10.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:09 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:09.851+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:10.227 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:09 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:09.851+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:10.227 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:09 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:09.851+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:10.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:10 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:10.050+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:10.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:10 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:10.050+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:10.476 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:10 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:10.050+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:10.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:10 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:10.240+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:10.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:10 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:10.240+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:10.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:10 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:10.240+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:10.995 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:10 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:10.736+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:10.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:10 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:10.736+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:10.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:10 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:10.681+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:10.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:10 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:10.681+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:11.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:10.866+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:11.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:10.866+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:11.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:10 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:10.866+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:11.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:11 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:11.090+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:11.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:11 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:11.090+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:11.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:11 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:11.092+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:18.261528+0000 front 2023-09-12T16:22:18.261722+0000 (oldest deadline 2023-09-12T16:22:42.361694+0000) 2023-09-12T16:23:11.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:11 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:11.092+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:30.464268+0000 front 2023-09-12T16:22:30.464289+0000 (oldest deadline 2023-09-12T16:22:55.164301+0000) 2023-09-12T16:23:11.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:11 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6[137584]: 2023-09-12T16:23:11.092+0000 7fd07a87a700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:35.164869+0000 front 2023-09-12T16:22:35.165006+0000 (oldest deadline 2023-09-12T16:23:01.064754+0000) 2023-09-12T16:23:11.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:11 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:11.257+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:11.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:11 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:11.257+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:11.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:11 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:11.257+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:11.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:11.732+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:11.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:11.732+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:11.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:11.640+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:11.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:11.640+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:11.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:11 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:11.640+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:12.112 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:11 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:11.846+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:12.112 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:11 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:11.846+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:12.113 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:11 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:11.846+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:12.475 INFO:journalctl@ceph.osd.6.smithi130.stdout:Sep 12 16:23:12 smithi130 bash[179971]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-6 2023-09-12T16:23:12.475 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:12 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:12.247+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:12.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:12 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:12.247+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:12.476 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:12 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:12.247+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:12.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:12.111+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:12.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:12.111+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:12.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:12.111+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:12.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:12 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:12.767+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:12.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:12 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:12.767+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:12.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:12 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:12.767+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:12.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:12 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:12.666+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:12.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:12 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:12.666+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:12.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:12 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:12.666+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:13.122 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:12.827+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:13.122 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:12.827+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:13.122 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:12.827+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:13.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:13.121+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:13.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:13.121+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:13.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:13.121+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:13.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:13 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:13.295+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:13.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:13 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:13.295+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:13.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:13 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:13.295+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:13.754 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.6.service' 2023-09-12T16:23:13.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:23:13.795 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-09-12T16:23:13.796 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-09-12T16:23:13.796 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.7 2023-09-12T16:23:13.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:13.732+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:13.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:13.732+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:13.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:13.732+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:13.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:13.704+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:13.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:13.704+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:13.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:13 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:13.704+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:14.122 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:14.121+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:14.123 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:13.783+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:14.123 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:13.783+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:14.123 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:13 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:13.783+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:14.123 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:13 smithi159 systemd[1]: Stopping Ceph osd.7 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:23:14.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:14.121+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:14.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:14.121+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:14.476 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:14.155+0000 7fd0f8ca2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:23:14.477 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:14.155+0000 7fd0f8ca2700 -1 osd.7 94 *** Got signal Terminated *** 2023-09-12T16:23:14.477 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:14.155+0000 7fd0f8ca2700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:23:14.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:14 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:14.324+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:14.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:14 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:14.324+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:14.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:14 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:14.324+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:14.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:14 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:14.324+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:14.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:14 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:14.727+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:14.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:14 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:14.727+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:14.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:14 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:14.727+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:14.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:14 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:14.746+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:14.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:14 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:14.746+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:14.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:14 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:14.746+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:15.105 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:14.806+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:15.105 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:14.806+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:15.106 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:14 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:14.806+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:15.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:15.104+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:15.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:15.104+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:15.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:15.104+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:15.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:15 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:15.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:15.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:15 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:15.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:15.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:15 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:15.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:15.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:15 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:15.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:15.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:15 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:15.736+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:15.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:15 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:15.736+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:15.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:15 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:15.736+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:15.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:15 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:15.760+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:15.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:15 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:15.760+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:15.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:15 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:15.760+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:16.066 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:15.785+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:16.067 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:15.785+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:16.067 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:15 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:15.785+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:16.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:16 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:16.065+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:16.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:16 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:16.065+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:16.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:16 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:16.065+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:16.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:16 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:16.314+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:16.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:16 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:16.314+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:16.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:16 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:16.314+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:16.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:16 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:16.314+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:16.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:16 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:16.775+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:16.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:16 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:16.775+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:16.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:16 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:16.775+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:16.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:16 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:16.735+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:16.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:16 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:16.735+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:16.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:16 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:16.735+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:17.021 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:16 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:16.767+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:17.021 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:16 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:16.767+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:17.022 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:16 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:16.767+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:17.475 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:17 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:17.021+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:17.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:17 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:17.021+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:17.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:17 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:17.021+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:17.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:17 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:17.311+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:17.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:17 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:17.311+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:17.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:17 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:17.311+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:17.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:17 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:17.311+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:17.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:17 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:17.705+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:17.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:17 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:17.705+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:17.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:17 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:17.705+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:17.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:17 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:17.761+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:17.997 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:17 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:17.761+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:17.997 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:17 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:17.761+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:18.048 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:17 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:17.767+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:18.049 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:17 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:17.767+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:18.049 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:17 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:17.767+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:18.049 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:17 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:17.767+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.193695+0000 front 2023-09-12T16:22:54.193734+0000 (oldest deadline 2023-09-12T16:23:17.693819+0000) 2023-09-12T16:23:18.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:18 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:18.048+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:18.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:18 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:18.048+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:18.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:18 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:18.048+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:18.614 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:18 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:18.301+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:18.614 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:18 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:18.301+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:18.615 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:18 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:18.301+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:18.615 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:18 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:18.301+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:18.995 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:18 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:18.762+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:18.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:18 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:18.762+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:18.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:18 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:18.762+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:18.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:18 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:18.706+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:18.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:18 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:18.706+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:18.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:18 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:18.706+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:19.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:18 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:18.783+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:28.088799+0000 front 2023-09-12T16:22:28.088764+0000 (oldest deadline 2023-09-12T16:22:53.289429+0000) 2023-09-12T16:23:19.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:18 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:18.783+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:37.891121+0000 front 2023-09-12T16:22:37.891029+0000 (oldest deadline 2023-09-12T16:23:03.691797+0000) 2023-09-12T16:23:19.226 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:18 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:18.783+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.992758+0000 front 2023-09-12T16:22:45.992858+0000 (oldest deadline 2023-09-12T16:23:09.492879+0000) 2023-09-12T16:23:19.227 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:18 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7[123383]: 2023-09-12T16:23:18.783+0000 7fd0f4a97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.193695+0000 front 2023-09-12T16:22:54.193734+0000 (oldest deadline 2023-09-12T16:23:17.693819+0000) 2023-09-12T16:23:19.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:19 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:19.029+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:19.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:19 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:19.029+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:19.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:19 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:19.029+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:19.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:19 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:19.029+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:19.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:19 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:19.314+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:19.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:19 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:19.314+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:19.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:19 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:19.314+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:19.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:19 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:19.314+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:19.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:19 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:19.314+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:19.995 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:19 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:19.720+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:19.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:19 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:19.720+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:19.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:19 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:19.720+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:19.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:19 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:19.732+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:19.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:19 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:19.732+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:19.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:19 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:19.732+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:20.476 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 12 16:23:20 smithi159 bash[146350]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-7 2023-09-12T16:23:20.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:19 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:19.985+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:20.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:19 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:19.985+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:20.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:19 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:19.985+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:20.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:19 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:19.985+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:20.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:20 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:20.319+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:20.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:20 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:20.319+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:20.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:20 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:20.319+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:20.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:20 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:20.319+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:20.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:20 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:20.319+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:20.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:20 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:20.699+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:20.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:20 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:20.699+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:20.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:20 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:20.699+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:20.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:20 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:20.699+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.531752+0000 front 2023-09-12T16:22:55.531829+0000 (oldest deadline 2023-09-12T16:23:20.231661+0000) 2023-09-12T16:23:20.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:20 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:20.728+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:20.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:20 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:20.728+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:20.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:20 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:20.728+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:20.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:20 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:20.728+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:21.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:20 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:20.963+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:21.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:20 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:20.963+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:21.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:20 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:20.963+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:21.182 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:20 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:20.963+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:21.525 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.7.service' 2023-09-12T16:23:21.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:23:21.566 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-09-12T16:23:21.567 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-09-12T16:23:21.567 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.8 2023-09-12T16:23:21.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:21 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:21.302+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:21.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:21 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:21.302+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:21.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:21 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:21.302+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:21.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:21 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:21.302+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:21.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:21 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:21.302+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:21.939 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:21.695+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:21.939 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:21.695+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:21.940 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:21.695+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:21.940 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:21.695+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:21.940 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:21 smithi203 systemd[1]: Stopping Ceph osd.8 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:23:21.941 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:21.699+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:21.941 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:21.699+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:21.941 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:21.699+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:21.941 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:21.699+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.531752+0000 front 2023-09-12T16:22:55.531829+0000 (oldest deadline 2023-09-12T16:23:20.231661+0000) 2023-09-12T16:23:21.941 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:21.938+0000 7fd37bcb6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:23:21.942 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:21.938+0000 7fd37bcb6700 -1 osd.8 94 *** Got signal Terminated *** 2023-09-12T16:23:22.225 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:21 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:21.938+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:22.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:21 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:21.938+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:22.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:21 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:21.938+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:22.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:21 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:21.938+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:22.245 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:21 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:21.938+0000 7fd37bcb6700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:23:22.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:22.351+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:22.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:22.351+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:22.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:22.351+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:22.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:22.351+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:22.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:22 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:22.351+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:22.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:22 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:22.701+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:22.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:22 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:22.701+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:22.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:22 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:22.701+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:22.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:22 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:22.701+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.531752+0000 front 2023-09-12T16:22:55.531829+0000 (oldest deadline 2023-09-12T16:23:20.231661+0000) 2023-09-12T16:23:22.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:22 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:22.731+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:22.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:22 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:22.731+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:22.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:22 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:22.731+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:22.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:22 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:22.731+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:23.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:22 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:22.950+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:23.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:22 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:22.950+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:23.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:22 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:22.950+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:23.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:22 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:22.950+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:23.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:23.341+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:23.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:23.341+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:23.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:23.341+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:23.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:23.341+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:23.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:23 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:23.341+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:23.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:23 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:23.675+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:23.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:23 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:23.675+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:23.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:23 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:23.675+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:23.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:23 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:23.675+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.531752+0000 front 2023-09-12T16:22:55.531829+0000 (oldest deadline 2023-09-12T16:23:20.231661+0000) 2023-09-12T16:23:23.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:23 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:23.773+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:23.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:23 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:23.773+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:23.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:23 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:23.773+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:23.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:23 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:23.773+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:24.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:23 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:23.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:24.910 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:23 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:23.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:24.911 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:23 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:23.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:24.911 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:23 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:23.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:24.912 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:24 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:24.329+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:24.912 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:24 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:24.329+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:24.912 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:24 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:24.329+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:24.913 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:24 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:24.329+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:24.913 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:24 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:24.329+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:24.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:24 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:24.650+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:24.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:24 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:24.650+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:24.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:24 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:24.650+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:24.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:24 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:24.650+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.531752+0000 front 2023-09-12T16:22:55.531829+0000 (oldest deadline 2023-09-12T16:23:20.231661+0000) 2023-09-12T16:23:24.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:24 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:24.799+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:24.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:24 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:24.799+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:24.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:24 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:24.799+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:24.998 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:24 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:24.799+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:24.998 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:24 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:24.799+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:25.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:24 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:24.995+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:25.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:24 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:24.995+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:25.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:24 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:24.995+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:25.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:24 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:24.995+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:25.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:25.361+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:25.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:25.361+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:25.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:25.361+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:25.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:25.361+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:25.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:25.361+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:25.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:25 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:25.361+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:25.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:25 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:25.673+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:25.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:25 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:25.673+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:25.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:25 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:25.673+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:25.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:25 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:25.673+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.531752+0000 front 2023-09-12T16:22:55.531829+0000 (oldest deadline 2023-09-12T16:23:20.231661+0000) 2023-09-12T16:23:25.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:25 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:25.823+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:25.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:25 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:25.823+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:25.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:25 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:25.823+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:25.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:25 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:25.823+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:25.998 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:25 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:25.823+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:26.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:26 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:26.023+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:26.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:26 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:26.023+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:26.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:26 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:26.023+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:26.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:26 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:26.023+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:26.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:26.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:26.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:26.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:26.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:26.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:26.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:26.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:26.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:26.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:26.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:26 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:26.348+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:26.995 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:26 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:26.710+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.826637+0000 front 2023-09-12T16:22:31.826536+0000 (oldest deadline 2023-09-12T16:22:54.726746+0000) 2023-09-12T16:23:26.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:26 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:26.710+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.928575+0000 front 2023-09-12T16:22:39.928298+0000 (oldest deadline 2023-09-12T16:23:01.028530+0000) 2023-09-12T16:23:26.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:26 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:26.710+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.830008+0000 front 2023-09-12T16:22:46.829977+0000 (oldest deadline 2023-09-12T16:23:12.630600+0000) 2023-09-12T16:23:26.996 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:26 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8[126503]: 2023-09-12T16:23:26.710+0000 7fd3782ac700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.531752+0000 front 2023-09-12T16:22:55.531829+0000 (oldest deadline 2023-09-12T16:23:20.231661+0000) 2023-09-12T16:23:26.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:26 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:26.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:26.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:26 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:26.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:26.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:26 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:26.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:26.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:26 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:26.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:26.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:26 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:26.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:27.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:26 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:26.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:27.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:26 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:26.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:27.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:26 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:26.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:27.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:26 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:26.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:27.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:27.321+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:27.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:27.321+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:27.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:27.321+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:27.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:27.321+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:27.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:27.321+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:27.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:27 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:27.321+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:27.845 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:27 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:27.770+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:27.845 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:27 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:27.770+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:27.845 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:27 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:27.770+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:27.846 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:27 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:27.770+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:27.846 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:27 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:27.770+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:28.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:27 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:27.935+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:28.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:27 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:27.935+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:28.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:27 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:27.935+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:28.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:27 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:27.935+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:28.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:27 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:27.935+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:28.245 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:27 smithi203 bash[147963]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-8 2023-09-12T16:23:28.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:28 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:28.343+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:28.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:28 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:28.343+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:28.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:28 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:28.343+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:28.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:28 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:28.343+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:28.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:28 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:28.343+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:28.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:28 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:28.343+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:29.021 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:28 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:28.756+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:29.021 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:28 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:28.756+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:29.022 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:28 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:28.756+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:29.022 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:28 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:28.756+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:29.022 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:28 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:28.756+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:29.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:28 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:28.942+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:29.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:28 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:28.942+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:29.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:28 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:28.942+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:29.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:28 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:28.942+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:29.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:28 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:28.942+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:29.264 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.8.service' 2023-09-12T16:23:29.290 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:29 smithi203 systemd[1]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.8.service: Succeeded. 2023-09-12T16:23:29.291 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 12 16:23:29 smithi203 systemd[1]: Stopped Ceph osd.8 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da. 2023-09-12T16:23:29.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:23:29.305 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-09-12T16:23:29.306 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-09-12T16:23:29.306 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.9 2023-09-12T16:23:29.678 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:29.305+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:29.679 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:29.305+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:29.679 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:29.305+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:29.679 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:29.305+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:29.679 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:29.305+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:29.680 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:29.305+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:29.680 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 systemd[1]: Stopping Ceph osd.9 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:23:29.975 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:29.677+0000 7fc5895e2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:23:29.975 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:29.677+0000 7fc5895e2700 -1 osd.9 94 *** Got signal Terminated *** 2023-09-12T16:23:29.975 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:29 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:29.677+0000 7fc5895e2700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:23:30.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:29 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:29.893+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:30.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:29 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:29.893+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:30.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:29 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:29.893+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:30.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:29 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:29.893+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:30.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:29 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:29.893+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:30.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:29 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:29.757+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:30.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:29 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:29.757+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:30.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:29 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:29.757+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:30.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:29 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:29.757+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:30.247 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:29 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:29.757+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:30.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:30 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:30.340+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:30.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:30 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:30.340+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:30.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:30 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:30.340+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:30.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:30 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:30.340+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:30.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:30 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:30.340+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:30.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:30 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:30.340+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:31.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:30 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:30.895+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:31.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:30 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:30.895+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:31.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:30 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:30.895+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:31.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:30 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:30.895+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:31.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:30 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:30.895+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:31.245 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:30 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:30.798+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:31.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:30 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:30.798+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:31.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:30 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:30.798+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:31.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:30 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:30.798+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:31.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:30 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:30.798+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:31.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:31 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:31.344+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:31.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:31 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:31.344+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:31.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:31 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:31.344+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:31.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:31 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:31.344+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:31.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:31 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:31.344+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:31.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:31 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:31.344+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:32.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:31 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:31.884+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:32.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:31 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:31.884+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:32.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:31 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:31.884+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:32.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:31 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:31.884+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:32.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:31 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:31.884+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:32.245 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:31 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:31.830+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:32.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:31 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:31.830+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:32.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:31 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:31.830+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:32.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:31 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:31.830+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:32.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:31 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:31.830+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:32.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:32 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:32.354+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:32.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:32 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:32.354+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:32.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:32 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:32.354+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:32.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:32 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:32.354+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:32.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:32 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:32.354+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:32.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:32 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:32.354+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:33.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:32 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:32.887+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:33.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:32 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:32.887+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:33.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:32 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:32.887+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:33.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:32 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:32.887+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:33.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:32 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:32.887+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:33.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:32 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:32.887+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:33.245 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:32 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:32.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:33.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:32 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:32.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:33.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:32 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:32.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:33.246 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:32 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:32.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:33.247 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:32 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:32.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:33.247 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:32 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:32.787+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:33.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:33.342+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:33.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:33.342+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:33.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:33.342+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:33.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:33.342+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:33.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:33.342+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:33.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:33 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:33.342+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:33.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:33 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:33.744+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:33.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:33 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:33.744+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:33.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:33 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:33.744+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:33.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:33 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:33.744+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:33.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:33 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:33.744+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:33.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:33 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:33.744+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:34.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:33 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:33.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:34.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:33 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:33.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:34.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:33 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:33.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:34.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:33 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:33.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:34.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:33 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:33.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:34.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:33 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:33.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:34.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:34 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:34.357+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.718743+0000 front 2023-09-12T16:22:16.718776+0000 (oldest deadline 2023-09-12T16:22:41.319517+0000) 2023-09-12T16:23:34.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:34 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:34.357+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2023-09-12T16:22:31.022938+0000 front 2023-09-12T16:22:31.022865+0000 (oldest deadline 2023-09-12T16:22:56.922831+0000) 2023-09-12T16:23:34.725 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:34 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:34.357+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:36.923554+0000 front 2023-09-12T16:22:36.923387+0000 (oldest deadline 2023-09-12T16:23:01.623248+0000) 2023-09-12T16:23:34.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:34 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:34.357+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:46.026149+0000 front 2023-09-12T16:22:46.026187+0000 (oldest deadline 2023-09-12T16:23:14.226711+0000) 2023-09-12T16:23:34.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:34 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:34.357+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:54.227286+0000 front 2023-09-12T16:22:54.227248+0000 (oldest deadline 2023-09-12T16:23:18.927243+0000) 2023-09-12T16:23:34.726 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:34 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9[146336]: 2023-09-12T16:23:34.357+0000 7fc5833d3700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:02.428530+0000 front 2023-09-12T16:23:02.428427+0000 (oldest deadline 2023-09-12T16:23:25.328282+0000) 2023-09-12T16:23:34.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:34 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:34.721+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:34.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:34 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:34.721+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:34.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:34 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:34.721+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:34.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:34 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:34.721+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:34.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:34 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:34.721+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:34.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:34 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:34.721+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:35.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:34 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:34.943+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:35.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:34 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:34.943+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:35.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:34 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:34.943+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:35.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:34 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:34.943+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:35.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:34 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:34.943+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:35.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:34 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:34.943+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:35.975 INFO:journalctl@ceph.osd.9.smithi130.stdout:Sep 12 16:23:35 smithi130 bash[180272]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-9 2023-09-12T16:23:35.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:35 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:35.692+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:35.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:35 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:35.692+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:35.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:35 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:35.692+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:35.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:35 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:35.692+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:35.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:35 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:35.692+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:35.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:35 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:35.692+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:36.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:35 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:35.919+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:36.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:35 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:35.919+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:36.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:35 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:35.919+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:36.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:35 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:35.919+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:36.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:35 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:35.919+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:36.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:35 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:35.919+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:36.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:36 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:36.675+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:36.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:36 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:36.675+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:36.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:36 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:36.675+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:36.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:36 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:36.675+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:36.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:36 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:36.675+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:36.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:36 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:36.675+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:37.052 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.9.service' 2023-09-12T16:23:37.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:23:37.094 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-09-12T16:23:37.095 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-09-12T16:23:37.095 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.10 2023-09-12T16:23:37.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:36 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:36.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:37.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:36 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:36.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:37.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:36 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:36.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:37.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:36 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:36.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:37.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:36 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:36.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:37.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:36 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:36.916+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:37.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 systemd[1]: Stopping Ceph osd.10 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:23:37.725 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:37.448+0000 7f2ae2e77700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:23:37.726 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:37.448+0000 7f2ae2e77700 -1 osd.10 94 *** Got signal Terminated *** 2023-09-12T16:23:37.726 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:37.448+0000 7f2ae2e77700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:23:37.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:37 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:37.653+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:37.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:37 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:37.653+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:37.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:37 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:37.653+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:37.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:37 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:37.653+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:37.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:37 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:37.653+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:37.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:37 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:37.653+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:38.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:37.930+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:38.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:37.930+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:38.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:37.930+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:38.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:37.930+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:38.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:37.930+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:38.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:37 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:37.930+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:38.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:38 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:38.623+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:38.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:38 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:38.623+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:38.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:38 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:38.624+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:38.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:38 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:38.624+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:38.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:38 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:38.624+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:38.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:38 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:38.624+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:39.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:38 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:38.959+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:39.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:38 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:38.959+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:39.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:38 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:38.959+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:39.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:38 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:38.959+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:39.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:38 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:38.959+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:39.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:38 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:38.959+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:39.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:39 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:39.575+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:39.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:39 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:39.575+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:39.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:39 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:39.575+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:39.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:39 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:39.575+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:39.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:39 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:39.575+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:39.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:39 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:39.575+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:40.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:39 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:39.994+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:40.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:39 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:39.994+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:40.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:39 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:39.994+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:40.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:39 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:39.994+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:40.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:39 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:39.994+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:40.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:39 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:39.994+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:40.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:40 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:40.577+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:40.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:40 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:40.577+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:40.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:40 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:40.577+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:40.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:40 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:40.577+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:40.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:40 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:40.577+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:40.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:40 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:40.577+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:41.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:40 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:40.999+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:41.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:40 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:40.999+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:41.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:40 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:40.999+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:41.476 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:40 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:40.999+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:41.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:40 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:40.999+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:41.477 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:40 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:40.999+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:41.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:41 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:41.569+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:41.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:41 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:41.569+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:41.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:41 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:41.569+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:41.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:41 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:41.569+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:41.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:41 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:41.569+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:41.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:41 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:41.569+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:42.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:41 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:41.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:16.147967+0000 front 2023-09-12T16:22:16.148185+0000 (oldest deadline 2023-09-12T16:22:40.248097+0000) 2023-09-12T16:23:42.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:41 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:41.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:39.650704+0000 front 2023-09-12T16:22:39.650743+0000 (oldest deadline 2023-09-12T16:23:01.350915+0000) 2023-09-12T16:23:42.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:41 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:41.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:45.853476+0000 front 2023-09-12T16:22:45.853610+0000 (oldest deadline 2023-09-12T16:23:11.153294+0000) 2023-09-12T16:23:42.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:41 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:41.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:51.153790+0000 front 2023-09-12T16:22:51.153776+0000 (oldest deadline 2023-09-12T16:23:18.754318+0000) 2023-09-12T16:23:42.226 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:41 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:41.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.256921+0000 front 2023-09-12T16:23:03.256847+0000 (oldest deadline 2023-09-12T16:23:27.356781+0000) 2023-09-12T16:23:42.227 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:41 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10[132126]: 2023-09-12T16:23:41.975+0000 7f2adf46d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:09.657835+0000 front 2023-09-12T16:23:09.657858+0000 (oldest deadline 2023-09-12T16:23:32.557766+0000) 2023-09-12T16:23:42.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:42 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:42.544+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:42.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:42 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:42.544+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:42.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:42 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:42.544+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:42.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:42 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:42.544+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:42.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:42 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:42.544+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:42.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:42 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:42.544+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:42.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:42 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:42.544+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2023-09-12T16:23:19.148384+0000 front 2023-09-12T16:23:19.148315+0000 (oldest deadline 2023-09-12T16:23:42.048313+0000) 2023-09-12T16:23:43.726 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 12 16:23:43 smithi159 bash[146630]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-10 2023-09-12T16:23:43.995 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:43 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:43.579+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:43.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:43 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:43.579+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:43.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:43 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:43.579+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:43.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:43 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:43.579+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:43.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:43 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:43.579+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:43.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:43 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:43.579+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:43.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:43 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:43.579+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2023-09-12T16:23:19.148384+0000 front 2023-09-12T16:23:19.148315+0000 (oldest deadline 2023-09-12T16:23:42.048313+0000) 2023-09-12T16:23:44.693 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.10.service' 2023-09-12T16:23:44.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:23:44.733 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-09-12T16:23:44.733 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-09-12T16:23:44.734 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.11 2023-09-12T16:23:44.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:44 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:44.616+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:44.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:44 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:44.616+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:44.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:44 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:44.616+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:44.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:44 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:44.616+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:44.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:44 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:44.616+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:44.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:44 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:44.616+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:44.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:44 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:44.616+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2023-09-12T16:23:19.148384+0000 front 2023-09-12T16:23:19.148315+0000 (oldest deadline 2023-09-12T16:23:42.048313+0000) 2023-09-12T16:23:44.998 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:44 smithi203 systemd[1]: Stopping Ceph osd.11 for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:23:45.495 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.093+0000 7f8f47f7e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:23:45.496 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.093+0000 7f8f47f7e700 -1 osd.11 94 *** Got signal Terminated *** 2023-09-12T16:23:45.496 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.093+0000 7f8f47f7e700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-12T16:23:45.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.637+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:45.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.637+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:45.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.637+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:45.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.637+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:45.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.637+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:45.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.637+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:45.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:45 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:45.637+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2023-09-12T16:23:19.148384+0000 front 2023-09-12T16:23:19.148315+0000 (oldest deadline 2023-09-12T16:23:42.048313+0000) 2023-09-12T16:23:46.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:46 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:46.621+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:46.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:46 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:46.621+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:46.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:46 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:46.621+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:46.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:46 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:46.621+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:46.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:46 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:46.621+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:46.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:46 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:46.621+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:46.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:46 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:46.621+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2023-09-12T16:23:19.148384+0000 front 2023-09-12T16:23:19.148315+0000 (oldest deadline 2023-09-12T16:23:42.048313+0000) 2023-09-12T16:23:47.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:47 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:47.647+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:47.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:47 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:47.647+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:47.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:47 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:47.647+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:47.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:47 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:47.647+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:47.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:47 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:47.647+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:47.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:47 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:47.647+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:47.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:47 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:47.647+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2023-09-12T16:23:19.148384+0000 front 2023-09-12T16:23:19.148315+0000 (oldest deadline 2023-09-12T16:23:42.048313+0000) 2023-09-12T16:23:48.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:48.614+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:48.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:48.614+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:48.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:48.614+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:48.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:48.614+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:48.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:48.614+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:48.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:48.615+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:48.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:48.615+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2023-09-12T16:23:19.148384+0000 front 2023-09-12T16:23:19.148315+0000 (oldest deadline 2023-09-12T16:23:42.048313+0000) 2023-09-12T16:23:48.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:48 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:48.615+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6820 osd.8 since back 2023-09-12T16:23:22.048710+0000 front 2023-09-12T16:23:22.048740+0000 (oldest deadline 2023-09-12T16:23:47.948737+0000) 2023-09-12T16:23:49.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:49.633+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6806 osd.0 since back 2023-09-12T16:22:17.636660+0000 front 2023-09-12T16:22:17.636831+0000 (oldest deadline 2023-09-12T16:22:42.936737+0000) 2023-09-12T16:23:49.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:49.633+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-09-12T16:22:38.139259+0000 front 2023-09-12T16:22:38.139221+0000 (oldest deadline 2023-09-12T16:23:04.539804+0000) 2023-09-12T16:23:49.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:49.633+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6814 osd.3 since back 2023-09-12T16:22:47.941733+0000 front 2023-09-12T16:22:47.941725+0000 (oldest deadline 2023-09-12T16:23:11.441601+0000) 2023-09-12T16:23:49.996 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:49.633+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2023-09-12T16:22:55.542785+0000 front 2023-09-12T16:22:55.542712+0000 (oldest deadline 2023-09-12T16:23:20.242781+0000) 2023-09-12T16:23:49.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:49.633+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-09-12T16:23:03.044305+0000 front 2023-09-12T16:23:03.044342+0000 (oldest deadline 2023-09-12T16:23:24.744519+0000) 2023-09-12T16:23:49.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:49.633+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.130:6822 osd.6 since back 2023-09-12T16:23:06.445748+0000 front 2023-09-12T16:23:06.445631+0000 (oldest deadline 2023-09-12T16:23:32.345573+0000) 2023-09-12T16:23:49.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:49.633+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2023-09-12T16:23:19.148384+0000 front 2023-09-12T16:23:19.148315+0000 (oldest deadline 2023-09-12T16:23:42.048313+0000) 2023-09-12T16:23:49.997 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:49 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11[134478]: 2023-09-12T16:23:49.633+0000 7f8f4156e700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.203:6820 osd.8 since back 2023-09-12T16:23:22.048710+0000 front 2023-09-12T16:23:22.048740+0000 (oldest deadline 2023-09-12T16:23:47.948737+0000) 2023-09-12T16:23:51.177 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 12 16:23:51 smithi203 bash[148239]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-osd-11 2023-09-12T16:23:52.538 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@osd.11.service' 2023-09-12T16:23:52.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:23:52.579 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-09-12T16:23:52.580 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-09-12T16:23:52.580 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.a 2023-09-12T16:23:52.940 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:23:52 smithi130 systemd[1]: Stopping Ceph mds.a for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:23:53.225 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:23:52 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-a[154322]: 2023-09-12T16:23:52.940+0000 7f16c69c1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:23:53.225 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:23:52 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-a[154322]: 2023-09-12T16:23:52.940+0000 7f16c69c1700 -1 mds.a *** got signal Terminated *** 2023-09-12T16:23:57.309 INFO:journalctl@ceph.mds.a.smithi130.stdout:Sep 12 16:23:57 smithi130 bash[180547]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-a 2023-09-12T16:23:57.496 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.a.service' 2023-09-12T16:23:57.533 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:23:57.533 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-09-12T16:23:57.533 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-09-12T16:23:57.534 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.d 2023-09-12T16:23:57.931 INFO:journalctl@ceph.mds.d.smithi130.stdout:Sep 12 16:23:57 smithi130 systemd[1]: Stopping Ceph mds.d for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:23:58.225 INFO:journalctl@ceph.mds.d.smithi130.stdout:Sep 12 16:23:57 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-d[153330]: 2023-09-12T16:23:57.929+0000 7fa23c878700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:23:58.225 INFO:journalctl@ceph.mds.d.smithi130.stdout:Sep 12 16:23:57 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-d[153330]: 2023-09-12T16:23:57.929+0000 7fa23c878700 -1 mds.d *** got signal Terminated *** 2023-09-12T16:24:02.277 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.d.service' 2023-09-12T16:24:02.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:02.313 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-09-12T16:24:02.313 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-09-12T16:24:02.313 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.g 2023-09-12T16:24:02.689 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:24:02 smithi130 systemd[1]: Stopping Ceph mds.g for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:02.975 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:24:02 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-g[153939]: 2023-09-12T16:24:02.688+0000 7f44c2cc3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:02.975 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:24:02 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-g[153939]: 2023-09-12T16:24:02.688+0000 7f44c2cc3700 -1 mds.g *** got signal Terminated *** 2023-09-12T16:24:07.169 INFO:journalctl@ceph.mds.g.smithi130.stdout:Sep 12 16:24:06 smithi130 bash[180830]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-g 2023-09-12T16:24:07.174 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.g.service' 2023-09-12T16:24:07.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:07.210 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-09-12T16:24:07.210 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-09-12T16:24:07.211 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.j 2023-09-12T16:24:07.475 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:24:07 smithi130 systemd[1]: Stopping Ceph mds.j for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:07.975 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:24:07 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-j[152968]: 2023-09-12T16:24:07.579+0000 7ff343b81700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:07.975 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:24:07 smithi130 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-j[152968]: 2023-09-12T16:24:07.579+0000 7ff343b81700 -1 mds.j *** got signal Terminated *** 2023-09-12T16:24:11.761 INFO:journalctl@ceph.mds.j.smithi130.stdout:Sep 12 16:24:11 smithi130 bash[180977]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-j 2023-09-12T16:24:11.970 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.j.service' 2023-09-12T16:24:12.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:12.007 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-09-12T16:24:12.008 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-09-12T16:24:12.008 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.b 2023-09-12T16:24:12.368 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:24:12 smithi159 systemd[1]: Stopping Ceph mds.b for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:12.725 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:24:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-b[139180]: 2023-09-12T16:24:12.366+0000 7fbcbbdb7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:12.726 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:24:12 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-b[139180]: 2023-09-12T16:24:12.366+0000 7fbcbbdb7700 -1 mds.b *** got signal Terminated *** 2023-09-12T16:24:16.726 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 12 16:24:16 smithi159 bash[146890]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-b 2023-09-12T16:24:16.832 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.b.service' 2023-09-12T16:24:16.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:16.869 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-09-12T16:24:16.869 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-09-12T16:24:16.869 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.e 2023-09-12T16:24:17.226 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:24:16 smithi159 systemd[1]: Stopping Ceph mds.e for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:17.726 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:24:17 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-e[138498]: 2023-09-12T16:24:17.240+0000 7f95072e7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:17.726 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:24:17 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-e[138498]: 2023-09-12T16:24:17.240+0000 7f95072e7700 -1 mds.e *** got signal Terminated *** 2023-09-12T16:24:20.813 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 12 16:24:20 smithi159 bash[147028]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-e 2023-09-12T16:24:21.003 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.e.service' 2023-09-12T16:24:21.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:21.038 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-09-12T16:24:21.039 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-09-12T16:24:21.039 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.h 2023-09-12T16:24:21.416 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:24:21 smithi159 systemd[1]: Stopping Ceph mds.h for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:21.725 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:24:21 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-h[138839]: 2023-09-12T16:24:21.416+0000 7f126e487700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:21.726 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:24:21 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-h[138839]: 2023-09-12T16:24:21.416+0000 7f126e487700 -1 mds.h *** got signal Terminated *** 2023-09-12T16:24:24.783 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 12 16:24:24 smithi159 bash[147170]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-h 2023-09-12T16:24:24.966 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.h.service' 2023-09-12T16:24:25.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:25.001 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-09-12T16:24:25.001 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-09-12T16:24:25.002 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.k 2023-09-12T16:24:25.226 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:24:25 smithi159 systemd[1]: Stopping Ceph mds.k for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:25.726 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:24:25 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-k[138149]: 2023-09-12T16:24:25.390+0000 7f5ce7a8e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:25.726 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:24:25 smithi159 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-k[138149]: 2023-09-12T16:24:25.390+0000 7f5ce7a8e700 -1 mds.k *** got signal Terminated *** 2023-09-12T16:24:29.726 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 12 16:24:29 smithi159 bash[147305]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-k 2023-09-12T16:24:29.950 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.k.service' 2023-09-12T16:24:29.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:29.988 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-09-12T16:24:29.988 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-09-12T16:24:29.988 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.c 2023-09-12T16:24:30.369 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:24:30 smithi203 systemd[1]: Stopping Ceph mds.c for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:30.745 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:24:30 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-c[140578]: 2023-09-12T16:24:30.369+0000 7fd2c2ef2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:30.746 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:24:30 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-c[140578]: 2023-09-12T16:24:30.369+0000 7fd2c2ef2700 -1 mds.c *** got signal Terminated *** 2023-09-12T16:24:32.995 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 12 16:24:32 smithi203 bash[148501]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-c 2023-09-12T16:24:33.004 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.c.service' 2023-09-12T16:24:33.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:33.042 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-09-12T16:24:33.042 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-09-12T16:24:33.042 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.f 2023-09-12T16:24:33.471 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:24:33 smithi203 systemd[1]: Stopping Ceph mds.f for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:33.745 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:24:33 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-f[140235]: 2023-09-12T16:24:33.471+0000 7f248cb24700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:33.746 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:24:33 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-f[140235]: 2023-09-12T16:24:33.471+0000 7f248cb24700 -1 mds.f *** got signal Terminated *** 2023-09-12T16:24:36.996 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 12 16:24:36 smithi203 bash[148642]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-f 2023-09-12T16:24:37.004 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.f.service' 2023-09-12T16:24:37.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:37.042 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-09-12T16:24:37.042 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-09-12T16:24:37.042 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.i 2023-09-12T16:24:37.435 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:24:37 smithi203 systemd[1]: Stopping Ceph mds.i for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:37.745 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:24:37 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-i[140918]: 2023-09-12T16:24:37.435+0000 7f6d8ee93700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:37.746 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:24:37 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-i[140918]: 2023-09-12T16:24:37.435+0000 7f6d8ee93700 -1 mds.i *** got signal Terminated *** 2023-09-12T16:24:40.689 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 12 16:24:40 smithi203 bash[148807]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-i 2023-09-12T16:24:40.879 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.i.service' 2023-09-12T16:24:40.916 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:40.916 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-09-12T16:24:40.917 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-09-12T16:24:40.917 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.l 2023-09-12T16:24:41.245 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:24:40 smithi203 systemd[1]: Stopping Ceph mds.l for 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da... 2023-09-12T16:24:41.746 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:24:41 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-l[139886]: 2023-09-12T16:24:41.317+0000 7f1d57090700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-12T16:24:41.746 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:24:41 smithi203 ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-l[139886]: 2023-09-12T16:24:41.317+0000 7f1d57090700 -1 mds.l *** got signal Terminated *** 2023-09-12T16:24:43.453 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 12 16:24:43 smithi203 bash[148949]: ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da-mds-l 2023-09-12T16:24:43.656 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ef22ea2-5184-11ee-9ab7-7b867c8bd7da@mds.l.service' 2023-09-12T16:24:43.691 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-12T16:24:43.692 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-09-12T16:24:43.692 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da --force --keep-logs 2023-09-12T16:24:43.997 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T16:24:45.748 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da --force --keep-logs 2023-09-12T16:24:46.073 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T16:24:47.698 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da --force --keep-logs 2023-09-12T16:24:48.089 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T16:24:49.734 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-12T16:24:49.772 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-12T16:24:49.808 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-12T16:24:49.838 INFO:tasks.cephadm:Archiving crash dumps... 2023-09-12T16:24:49.840 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/crash to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/remote/smithi130/crash 2023-09-12T16:24:49.841 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/crash -- . 2023-09-12T16:24:49.889 INFO:teuthology.orchestra.run.smithi130.stderr:tar: /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-12T16:24:49.890 INFO:teuthology.orchestra.run.smithi130.stderr:tar: Error is not recoverable: exiting now 2023-09-12T16:24:49.891 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/crash to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/remote/smithi159/crash 2023-09-12T16:24:49.893 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/crash -- . 2023-09-12T16:24:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-12T16:24:49.948 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2023-09-12T16:24:49.949 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/crash to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/remote/smithi203/crash 2023-09-12T16:24:49.950 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/crash -- . 2023-09-12T16:24:49.995 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-12T16:24:49.996 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2023-09-12T16:24:49.997 INFO:tasks.cephadm:Checking cluster log for badness... 2023-09-12T16:24:49.997 DEBUG:teuthology.orchestra.run.smithi130:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-09-12T16:24:50.073 INFO:teuthology.orchestra.run.smithi130.stderr:grep: /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph.log: No such file or directory 2023-09-12T16:24:50.077 INFO:tasks.cephadm:Compressing logs... 2023-09-12T16:24:50.078 DEBUG:teuthology.orchestra.run.smithi130:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-12T16:24:50.080 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-12T16:24:50.083 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-12T16:24:50.118 INFO:teuthology.orchestra.run.smithi130.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-12T16:24:50.118 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.0.log 2023-09-12T16:24:50.118 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.g.log 2023-09-12T16:24:50.119 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.a.log 2023-09-12T16:24:50.120 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.6.log 2023-09-12T16:24:50.120 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.g.log: /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.3.log 2023-09-12T16:24:50.120 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.d.log 2023-09-12T16:24:50.122 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mgr.x.log 2023-09-12T16:24:50.122 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.9.log 2023-09-12T16:24:50.122 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-volume.log 2023-09-12T16:24:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.j.log 2023-09-12T16:24:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mon.a.log 2023-09-12T16:24:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-volume.log: /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-12T16:24:50.123 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mon.a.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-09-12T16:24:50.124 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.log: 94.3% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.a.log.gz 2023-09-12T16:24:50.124 INFO:teuthology.orchestra.run.smithi130.stderr: 94.3% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.g.log.gz 2023-09-12T16:24:50.125 INFO:teuthology.orchestra.run.smithi130.stderr: 94.3% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.d.log.gz 2023-09-12T16:24:50.125 INFO:teuthology.orchestra.run.smithi130.stderr: 94.3% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.j.log.gz 2023-09-12T16:24:50.128 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-12T16:24:50.128 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.5.log 2023-09-12T16:24:50.132 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-12T16:24:50.132 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.10.log 2023-09-12T16:24:50.135 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.h.log 2023-09-12T16:24:50.136 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.e.log 2023-09-12T16:24:50.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.4.log 2023-09-12T16:24:50.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.h.log: /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.1.log 2023-09-12T16:24:50.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.k.log 2023-09-12T16:24:50.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-volume.log 2023-09-12T16:24:50.140 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.7.log 2023-09-12T16:24:50.140 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mon.b.log 2023-09-12T16:24:50.141 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.b.log 2023-09-12T16:24:50.141 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mgr.y.log 2023-09-12T16:24:50.141 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mon.b.log: /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-12T16:24:50.142 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.2.log 2023-09-12T16:24:50.143 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.11.log 2023-09-12T16:24:50.143 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mgr.z.log 2023-09-12T16:24:50.143 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.2.log: /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.8.log 2023-09-12T16:24:50.143 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.f.log 2023-09-12T16:24:50.143 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.c.log 2023-09-12T16:24:50.144 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.i.log 2023-09-12T16:24:50.144 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mon.c.log 2023-09-12T16:24:50.144 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-volume.log 2023-09-12T16:24:50.144 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.l.log 2023-09-12T16:24:50.144 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-12T16:24:50.145 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.l.log: /var/log/ceph/cephadm.log: 93.2% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mgr.z.log.gz 2023-09-12T16:24:50.145 INFO:teuthology.orchestra.run.smithi203.stderr: 94.3% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.l.log.gz 2023-09-12T16:24:50.146 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mgr.y.log: /var/log/ceph/cephadm.log: 94.2% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.k.log.gz 2023-09-12T16:24:50.146 INFO:teuthology.orchestra.run.smithi159.stderr: 93.1% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mgr.y.log.gz 2023-09-12T16:24:50.444 INFO:teuthology.orchestra.run.smithi130.stderr: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-12T16:24:50.456 INFO:teuthology.orchestra.run.smithi203.stderr: 91.0% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-volume.log.gz 2023-09-12T16:24:50.467 INFO:teuthology.orchestra.run.smithi159.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-12T16:24:50.721 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-volume.log.gz 2023-09-12T16:24:50.743 INFO:teuthology.orchestra.run.smithi130.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-09-12T16:24:50.808 INFO:teuthology.orchestra.run.smithi159.stderr: 93.9% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.e.log.gz 2023-09-12T16:24:50.924 INFO:teuthology.orchestra.run.smithi130.stderr: 91.1% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-volume.log.gz 2023-09-12T16:24:51.658 INFO:teuthology.orchestra.run.smithi203.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-12T16:24:51.675 INFO:teuthology.orchestra.run.smithi130.stderr: 90.3% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mgr.x.log.gz 2023-09-12T16:24:52.126 INFO:teuthology.orchestra.run.smithi159.stderr: 95.4% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.h.log.gz 2023-09-12T16:24:52.304 INFO:teuthology.orchestra.run.smithi159.stderr: 91.9% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mon.b.log.gz 2023-09-12T16:24:52.793 INFO:teuthology.orchestra.run.smithi203.stderr: 94.2% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.f.log.gz 2023-09-12T16:24:53.057 INFO:teuthology.orchestra.run.smithi203.stderr: 92.0% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mon.c.log.gz 2023-09-12T16:24:54.924 INFO:teuthology.orchestra.run.smithi130.stderr: 90.3% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mon.a.log.gz 2023-09-12T16:25:45.452 INFO:teuthology.orchestra.run.smithi203.stderr: 94.6% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.i.log.gz 2023-09-12T16:26:54.204 INFO:teuthology.orchestra.run.smithi203.stderr: 97.9% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.c.log.gz 2023-09-12T16:28:50.912 INFO:teuthology.orchestra.run.smithi159.stderr: 98.1% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-mds.b.log.gz 2023-09-12T16:29:18.789 INFO:teuthology.orchestra.run.smithi159.stderr: 91.2% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.10.log.gz 2023-09-12T16:29:42.059 INFO:teuthology.orchestra.run.smithi203.stderr: 91.2% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.8.log.gz 2023-09-12T16:29:46.624 INFO:teuthology.orchestra.run.smithi203.stderr: 91.3% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.11.log.gz 2023-09-12T16:30:02.721 INFO:teuthology.orchestra.run.smithi159.stderr: 91.2% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.1.log.gz 2023-09-12T16:30:10.841 INFO:teuthology.orchestra.run.smithi130.stderr: 91.2% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.0.log.gz 2023-09-12T16:30:36.502 INFO:teuthology.orchestra.run.smithi130.stderr: 91.2% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.3.log.gz 2023-09-12T16:30:44.452 INFO:teuthology.orchestra.run.smithi203.stderr: 91.4% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.2.log.gz 2023-09-12T16:30:52.708 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.7.log.gz 2023-09-12T16:31:09.689 INFO:teuthology.orchestra.run.smithi130.stderr: 91.2% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.6.log.gz 2023-09-12T16:31:59.270 INFO:teuthology.orchestra.run.smithi130.stderr: 91.1% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.9.log.gz 2023-09-12T16:31:59.273 INFO:teuthology.orchestra.run.smithi130.stderr: 2023-09-12T16:31:59.273 INFO:teuthology.orchestra.run.smithi130.stderr:real 7m9.177s 2023-09-12T16:31:59.273 INFO:teuthology.orchestra.run.smithi130.stderr:user 9m32.088s 2023-09-12T16:31:59.274 INFO:teuthology.orchestra.run.smithi130.stderr:sys 0m22.194s 2023-09-12T16:32:03.748 INFO:teuthology.orchestra.run.smithi203.stderr: 91.1% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.5.log.gz 2023-09-12T16:32:03.760 INFO:teuthology.orchestra.run.smithi203.stderr: 2023-09-12T16:32:03.761 INFO:teuthology.orchestra.run.smithi203.stderr:real 7m13.661s 2023-09-12T16:32:03.761 INFO:teuthology.orchestra.run.smithi203.stderr:user 8m58.498s 2023-09-12T16:32:03.761 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m21.587s 2023-09-12T16:32:03.964 INFO:teuthology.orchestra.run.smithi159.stderr: 91.5% -- replaced with /var/log/ceph/9ef22ea2-5184-11ee-9ab7-7b867c8bd7da/ceph-osd.4.log.gz 2023-09-12T16:32:03.967 INFO:teuthology.orchestra.run.smithi159.stderr: 2023-09-12T16:32:03.967 INFO:teuthology.orchestra.run.smithi159.stderr:real 7m13.867s 2023-09-12T16:32:03.967 INFO:teuthology.orchestra.run.smithi159.stderr:user 9m3.903s 2023-09-12T16:32:03.968 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m22.249s 2023-09-12T16:32:03.968 INFO:tasks.cephadm:Archiving logs... 2023-09-12T16:32:03.969 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/remote/smithi130/log 2023-09-12T16:32:03.971 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-12T16:33:47.829 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/remote/smithi159/log 2023-09-12T16:33:47.831 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-12T16:35:19.455 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/remote/smithi203/log 2023-09-12T16:35:19.478 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-12T16:36:53.275 INFO:tasks.cephadm:Removing cluster... 2023-09-12T16:36:53.275 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da --force 2023-09-12T16:36:53.823 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T16:36:55.006 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da --force 2023-09-12T16:36:55.602 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T16:36:56.731 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da --force 2023-09-12T16:36:57.262 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 9ef22ea2-5184-11ee-9ab7-7b867c8bd7da 2023-09-12T16:36:58.369 INFO:tasks.cephadm:Removing cephadm ... 2023-09-12T16:36:58.369 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-12T16:36:58.391 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-12T16:36:58.426 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-12T16:36:58.447 INFO:tasks.cephadm:Teardown complete 2023-09-12T16:36:58.448 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-12T16:36:58.463 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-12T16:36:58.463 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-12T16:36:58.467 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-12T16:36:58.470 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-12T16:36:58.602 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-12T16:36:58.613 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-12T16:36:58.614 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T16:36:58.617 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T16:36:58.620 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T16:36:58.634 INFO:teuthology.orchestra.run.smithi130.stderr:bash: ntpq: command not found 2023-09-12T16:36:58.636 INFO:teuthology.orchestra.run.smithi159.stderr:bash: ntpq: command not found 2023-09-12T16:36:58.637 INFO:teuthology.orchestra.run.smithi203.stderr:bash: ntpq: command not found 2023-09-12T16:36:58.655 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-12T16:36:58.655 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2023-09-12T16:36:58.655 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 128 -248us[ -251us] +/- 91ms 2023-09-12T16:36:58.656 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv02.front.sepia.ceph.com 3 7 377 5 -321us[ -324us] +/- 57ms 2023-09-12T16:36:58.656 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 9 377 69 +658us[ +655us] +/- 86ms 2023-09-12T16:36:58.656 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T16:36:58.660 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-12T16:36:58.660 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2023-09-12T16:36:58.661 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 -134us[ -122us] +/- 92ms 2023-09-12T16:36:58.661 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 -229us[ -217us] +/- 57ms 2023-09-12T16:36:58.661 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +740us[ +751us] +/- 87ms 2023-09-12T16:36:58.661 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T16:36:58.677 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-12T16:36:58.677 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2023-09-12T16:36:58.677 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 6 -129us[ -129us] +/- 92ms 2023-09-12T16:36:58.677 INFO:teuthology.orchestra.run.smithi130.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -244us[ -232us] +/- 57ms 2023-09-12T16:36:58.678 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +728us[ +728us] +/- 87ms 2023-09-12T16:36:58.678 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-12T16:36:58.679 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-12T16:36:58.689 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-12T16:36:58.690 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-12T16:36:58.705 DEBUG:teuthology.orchestra.run.smithi130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-12T16:36:58.921 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-12T16:36:59.159 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-12T16:36:59.354 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-12T16:36:59.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T16:36:59.393 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-12T16:36:59.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T16:36:59.430 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-12T16:36:59.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T16:36:59.465 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-12T16:36:59.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-12T16:36:59.584 INFO:teuthology.task.internal:Duration was 3270.318622 seconds 2023-09-12T16:36:59.585 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-12T16:36:59.635 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-12T16:36:59.635 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-12T16:36:59.640 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-12T16:36:59.644 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-12T16:36:59.720 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-12T16:36:59.721 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-12T16:36:59.754 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-12T16:37:00.006 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-12T16:37:00.006 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2023-09-12T16:37:00.007 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-12T16:37:00.041 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2023-09-12T16:37:00.043 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-12T16:37:00.102 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2023-09-12T16:37:00.103 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-12T16:37:00.147 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-12T16:37:00.148 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-12T16:37:00.155 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-12T16:37:00.168 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-12T16:37:00.250 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-12T16:37:00.251 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-12T16:37:00.262 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-12T16:37:00.270 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-12T16:37:00.513 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-12T16:37:00.580 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-12T16:37:00.581 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-12T16:37:00.647 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-12T16:37:00.692 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-12T16:37:00.721 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-12T16:37:00.744 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-12T16:37:00.756 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-12T16:37:00.759 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-12T16:37:00.785 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2023-09-12T16:37:00.801 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2023-09-12T16:37:00.805 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2023-09-12T16:37:00.838 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-12T16:37:00.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T16:37:00.858 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-12T16:37:00.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T16:37:00.875 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-12T16:37:00.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T16:37:00.894 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-12T16:37:00.923 INFO:teuthology.task.internal:Transferring archived files... 2023-09-12T16:37:00.924 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/remote/smithi130 2023-09-12T16:37:00.925 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-12T16:37:00.981 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/remote/smithi159 2023-09-12T16:37:00.982 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-12T16:37:01.026 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100/remote/smithi203 2023-09-12T16:37:01.026 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-12T16:37:01.067 INFO:teuthology.task.internal:Removing archive directory... 2023-09-12T16:37:01.068 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-12T16:37:01.077 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-12T16:37:01.085 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-12T16:37:01.117 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-12T16:37:01.298 INFO:teuthology.task.internal:Not uploading archives. 2023-09-12T16:37:01.298 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-12T16:37:01.658 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-12T16:37:01.659 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-12T16:37:01.664 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-12T16:37:01.667 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-12T16:37:01.679 INFO:teuthology.orchestra.run.smithi130.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 16:37 /home/ubuntu/cephtest 2023-09-12T16:37:01.682 INFO:teuthology.orchestra.run.smithi159.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 16:37 /home/ubuntu/cephtest 2023-09-12T16:37:01.687 INFO:teuthology.orchestra.run.smithi203.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 16:37 /home/ubuntu/cephtest 2023-09-12T16:37:01.695 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-12T16:37:01.715 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-12T16:37:02.083 INFO:teuthology.nuke:Checking targets against current locks 2023-09-12T16:37:02.112 DEBUG:teuthology.nuke:shortname: smithi130 2023-09-12T16:37:02.113 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-12T16:37:02.138 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 15:36:47.897367', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-12T16:37:02.142 DEBUG:teuthology.nuke:shortname: smithi159 2023-09-12T16:37:02.142 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-12T16:37:02.165 DEBUG:teuthology.nuke:shortname: smithi203 2023-09-12T16:37:02.166 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-12T16:37:02.173 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 15:36:47.896221', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-12T16:37:02.208 INFO:teuthology.orchestra.console.smithi130:Power off 2023-09-12T16:37:02.209 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-12T16:37:02.226 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 15:36:47.894868', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-12T16:37:02.245 DEBUG:teuthology.orchestra.console.smithi130:power off output: Chassis Power Control: Down/Off 2023-09-12T16:37:02.246 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:02.265 INFO:teuthology.orchestra.console.smithi159:Power off 2023-09-12T16:37:02.266 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-12T16:37:02.280 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2023-09-12T16:37:02.304 INFO:teuthology.orchestra.console.smithi203:Power off 2023-09-12T16:37:02.304 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-12T16:37:02.315 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2023-09-12T16:37:02.316 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:02.331 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off 2023-09-12T16:37:02.332 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:02.342 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2023-09-12T16:37:02.363 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-09-12T16:37:06.282 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:06.343 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:06.364 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:06.396 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2023-09-12T16:37:06.454 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2023-09-12T16:37:06.473 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-09-12T16:37:10.399 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:10.456 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:10.475 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:10.510 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2023-09-12T16:37:10.567 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2023-09-12T16:37:10.585 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-09-12T16:37:10.612 INFO:teuthology.orchestra.console.smithi130:Power off completed 2023-09-12T16:37:10.668 INFO:teuthology.orchestra.console.smithi159:Power off completed 2023-09-12T16:37:14.590 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T16:37:14.700 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off 2023-09-12T16:37:14.801 INFO:teuthology.orchestra.console.smithi203:Power off completed 2023-09-12T16:37:14.906 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} duration: 3270.318622112274 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2023-09-12T16:37:14.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T16:37:14.997 INFO:teuthology.run:FAIL